module RocketJob::Batch::ThrottleRunningWorkers

Throttle the number of slices of a specific batch job that are processed at the same time.

Example:

class MyJob < RocketJob::Job
  include RocketJob::Batch

  # Maximum number of slices to process at the same time for each running instance.
  self.throttle_running_workers = 25

  def perform(record)
    # ....
  end
end

It attempts to ensure that the number of workers do not exceed this number. This is not a hard limit and it is possible for the number of workers to slightly exceed this value at times. It can also occur that the number of slices running can drop below this number for a short period.

This value can be modified while a job is running. The change will be picked up at the start of processing slices, or after processing a slice and `re_check_seconds` has been exceeded.

0 or nil : No limits in place

Default: nil

Private Instance Methods

throttle_running_jobs_base_query() click to toggle source

Allows another job with a higher priority to start even though this one is running already @overrides RocketJob::Plugins::Job::ThrottleRunningJobs#throttle_running_jobs_base_query

Calls superclass method
# File lib/rocket_job/batch/throttle_running_workers.rb, line 55
def throttle_running_jobs_base_query
  query                = super
  query[:priority.lte] = priority if throttle_running_workers&.positive?
  query
end
throttle_running_workers_exceeded?(slice) click to toggle source

Returns [true|false] whether the throttle for this job has been exceeded

# File lib/rocket_job/batch/throttle_running_workers.rb, line 45
def throttle_running_workers_exceeded?(slice)
  return false unless throttle_running_workers&.positive?

  input.running.with(read: {mode: :primary}) do |conn|
    conn.where(:id.ne => slice.id).count >= throttle_running_workers
  end
end