GoodJob
¶ ↑
GoodJob
is a multithreaded, Postgres-based, ActiveJob
backend for Ruby on Rails.
Inspired by {Delayed::Job}[https://github.com/collectiveidea/delayed_job] and {Que}[https://github.com/que-rb/que], GoodJob is designed for maximum compatibility with Ruby on Rails, ActiveJob, and Postgres to be simple and performant for most workloads.
-
Designed for ActiveJob. Complete support for async, queues, delays, priorities, timeouts, and retries with near-zero configuration.
-
Built for Rails. Fully adopts Ruby on Rails threading and code execution guidelines with Concurrent::Ruby.
-
Backed by Postgres. Relies upon Postgres integrity, session-level Advisory Locks to provide run-once safety and stay within the limits of
schema.rb
, and LISTEN/NOTIFY to reduce queuing latency. -
For most workloads. Targets full-stack teams, economy-minded solo developers, and applications that enqueue 1-million jobs/day and more.
For more of the story of GoodJob
, read the introductory blog post.
<details markdown=β1β> <summary><strong>π Comparison of GoodJob
with other job queue backends (click to expand)</strong></summary>
Queues, priority, retries | Database | Concurrency | Reliability/Integrity | Latency | |
---|---|---|---|---|---|
**GoodJob** | β Yes | β Postgres | β Multithreaded | β ACID, Advisory Locks | β Postgres LISTEN/NOTIFY |
**Que** | β Yes | πΆοΈ Postgres, requires `structure.sql` | β Multithreaded | β ACID, Advisory Locks | β Postgres LISTEN/NOTIFY |
**Delayed Job** | β Yes | β Postgres | π΄ Single-threaded | β ACID, record-based | πΆ Polling |
**Sidekiq** | β Yes | π΄ Redis | β Multithreaded | π΄ Crashes lose jobs | β Redis BRPOP |
**Sidekiq Pro** | β Yes | π΄ Redis | β Multithreaded | β Redis RPOPLPUSH | β Redis RPOPLPUSH |
</details>
Table of contents¶ ↑
Set up¶ ↑
-
Add
good_job
to your application's Gemfile:gem 'good_job'
-
Install the gem:
$ bundle install
-
Run the
GoodJob
install generator. This will generate a database migration to create a table for GoodJob's job records:$ bin/rails g good_job:install
Run the migration:
$ bin/rails db:migrate
Optional: If using Rails' multiple databases with the migrations_paths
configuration option, use the --database
option:
```bash bin/rails g good_job:install --database animals bin/rails db:migrate:animals ```
-
Configure the
ActiveJob
adapter:# config/application.rb config.active_job.queue_adapter = :good_job
-
Inside of your application, queue your job π:
YourJob.perform_later
GoodJob
supports allActiveJob
features:YourJob.set(queue: :some_queue, wait: 5.minutes, priority: 10).perform_later
-
In development,
GoodJob
executes jobs immediately. In production,GoodJob
provides different options:-
By default,
GoodJob
separates job enqueuing from job execution so that jobs can be scaled independently of the web server. Use theGoodJob
command-line tool to execute jobs:$ bundle exec good_job start
Ideally the command-line tool should be run on a separate machine or container from the web process. For example, on Heroku:
web: rails server worker: bundle exec good_job start
The command-line tool supports a variety of options, see the reference below for command-line configuration.
-
GoodJob
can also be configured to execute jobs within the web server process to save on resources. This is useful for low-workloads when economy is paramount.$ GOOD_JOB_EXECUTION_MODE=async rails server
Additional configuration is likely necessary, see the reference below for configuration.
-
Compatibility¶ ↑
-
Ruby on Rails: 5.2+
-
Ruby: MRI 2.5+. JRuby 9.2.13+ (JRuby's
activerecord-jdbcpostgresql-adapter
gem does not support Postgres LISTEN/NOTIFY). -
Postgres: 9.6+
Configuration¶ ↑
Command-line options¶ ↑
There several top-level commands available through the good_job
command-line tool.
Configuration options are available with help
.
good_job start
¶ ↑
good_job start
executes queued jobs.
$ bundle exec good_job help start Usage: good_job start Options: [--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5) [--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *) [--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1) [--max-cache=COUNT] # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000) [--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever)) [--enable-cron] # Whether to run cron process (default: false) [--daemonize] # Run as a background daemon (default: false) [--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid) Executes queued jobs. All options can be configured with environment variables. See option descriptions for the matching environment variable name. == Configuring queues Separate multiple queues with commas; exclude queues with a leading minus; separate isolated execution pools with semicolons and threads with colons.
good_job cleanup_preserved_jobs
¶ ↑
good_job cleanup_preserved_jobs
deletes preserved job records. See GoodJob.preserve_job_records
for when this command is useful.
$ bundle exec good_job help cleanup_preserved_jobs Usage: good_job cleanup_preserved_jobs Options: [--before-seconds-ago=SECONDS] # Delete records finished more than this many seconds ago (env var: GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO, default: 86400) Deletes preserved job records. By default, GoodJob deletes job records when the job is performed and this command is not necessary. However, when `GoodJob.preserve_job_records = true`, the jobs will be preserved in the database. This is useful when wanting to analyze or inspect job performance. If you are preserving job records this way, use this command regularly to delete old records and preserve space in your database.
Configuration options¶ ↑
To use GoodJob
, you can set config.active_job.queue_adapter
to a :good_job
.
Additional configuration can be provided via config.good_job.OPTION = ...
for example:
# config/application.rb config.active_job.queue_adapter = :good_job # Configure options individually... config.good_job.execution_mode = :async config.good_job.max_threads = 5 config.good_job.poll_interval = 30 # seconds config.good_job.shutdown_timeout = 25 # seconds config.good_job.enable_cron = true config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob' } } config.good_job.queues = '*' # ...or all at once. config.good_job = { execution_mode: :async, max_threads: 5, poll_interval: 30, shutdown_timeout: 25, enable_cron: true, cron: { example: { cron: '0 * * * *', class: 'ExampleJob' }, }, queues: '*', }
Available configuration options are:
-
execution_mode
(symbol) specifies how and where jobs should be executed. You can also set this with the environment variableGOOD_JOB_EXECUTION_MODE
. It can be any one of:-
:inline
executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments. -
:external
causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), youβll need to use the command-line tool to actually execute your jobs. -
:async
(or:async_server
) executes jobs in separate threads within the Rails web server process (bundle exec rails server
). It can be more economical for small workloads because you donβt need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose:external
instead. When not in the Rails web server, jobs will execute in:external
mode to ensure jobs are not executed withinrails console
,rails db:migrate
,rails assets:prepare
, etc. -
:async_all
executes jobs in separate threads in any Rails process.
-
-
max_threads
(integer) sets the maximum number of threads to use whenexecution_mode
is set to:async
. You can also set this with the environment variableGOOD_JOB_MAX_THREADS
. -
queues
(string) determines which queues to execute jobs from whenexecution_mode
is set to:async
. See the description ofgood_job start
for more details on the format of this string. You can also set this with the environment variableGOOD_JOB_QUEUES
. -
poll_interval
(integer) sets the number of seconds between polls for jobs whenexecution_mode
is set to:async
. You can also set this with the environment variableGOOD_JOB_POLL_INTERVAL
. A poll interval of-1
disables polling completely. -
max_cache
(integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variableGOOD_JOB_MAX_CACHE
. -
shutdown_timeout
(float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever:-1
. You can also set this with the environment variableGOOD_JOB_SHUTDOWN_TIMEOUT
. -
enable_cron
(boolean) whether to run cron process. Defaults tofalse
. You can also set this with the environment variableGOOD_JOB_ENABLE_CRON
. -
cron
(hash) cron configuration. Defaults to{}
. You can also set this as a JSON string with the environment variableGOOD_JOB_CRON
By default, GoodJob
configures the following execution modes per environment:
# config/environments/development.rb config.active_job.queue_adapter = :good_job config.good_job.execution_mode = :async # config/environments/test.rb config.active_job.queue_adapter = :good_job config.good_job.execution_mode = :inline # config/environments/production.rb config.active_job.queue_adapter = :good_job config.good_job.execution_mode = :external
Global options¶ ↑
Good Jobβs general behavior can also be configured via several attributes directly on the GoodJob
module:
-
GoodJob.active_record_parent_class
(string) The ActiveRecord parent class inherited by GoodJob's ActiveRecord modelGoodJob::Job
(defaults to"ActiveRecord::Base"
). Configure this when using multiple databases with ActiveRecord or when other custom configuration is necessary for the ActiveRecord model to connect to the Postgres database. The value must be a String to avoid premature initialization of ActiveRecord. -
GoodJob.logger
(Rails Logger) lets you set a custom logger forGoodJob
. It should be an instance of a RailsLogger
. -
GoodJob.preserve_job_records
(boolean) keeps job records in your database even after jobs are completed. (Default:false
) -
GoodJob.retry_on_unhandled_error
(boolean) causes jobs to be re-queued and retried if they raise an instance ofStandardError
. Instances ofException
, like SIGINT, will always be retried, regardless of this attributeβs value. (Default:true
) -
GoodJob.on_thread_error
(proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
Youβll generally want to configure these in config/initializers/good_job.rb
, like so:
# config/initializers/good_job.rb GoodJob.active_record_parent_class = "ApplicationRecord" GoodJob.preserve_job_records = true GoodJob.retry_on_unhandled_error = false GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
Dashboard¶ ↑
π§ GoodJob's dashboard is a work in progress. Please contribute ideas and code on {Github}[https://github.com/bensheldon/good_job/issues].
GoodJob
includes a Dashboard as a mountable Rails::Engine
.
-
Explicitly require the Engine code at the top of your
config/application.rb
file, immediately after Rails is required. This is necessary because the mountable engine is an optional feature ofGoodJob
.# config/application.rb require_relative 'boot' require 'rails/all' require 'good_job/engine' # <= Add this line # ...
-
Mount the engine in your
config/routes.rb
file. The following will mount it athttp://example.com/good_job
.# config/routes.rb # ... mount GoodJob::Engine => 'good_job'
Because jobs can potentially contain sensitive information, you should authorize access. For example, using Devise's
authenticate
helper, that might look like:# config/routes.rb # ... authenticate :user, ->(user) { user.admin? } do mount GoodJob::Engine => 'good_job' end
Another option is using basic auth like this:
# config/initializers/good_job.rb GoodJob::Engine.middleware.use(Rack::Auth::Basic) do |username, password| ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_username, username) && ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_password, password) end
ActiveJob
concurrency¶ ↑
GoodJob
can extend ActiveJob
to provide limits on concurrently running jobs, either at time of enqueue or at perform. Limiting concurrency can help prevent duplicate, double or unecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
Note: Limiting concurrency at enqueue requires Rails 6.0+ because Rails 5.2 cannot halt ActiveJob
callbacks.
class MyJob < ApplicationJob include GoodJob::ActiveJobExtensions::Concurrency good_job_control_concurrency_with( # Maximum number of unfinished jobs to allow with the concurrency key total_limit: 1, # Or, if more control is needed: # Maximum number of jobs with the concurrency key to be concurrently enqueued (excludes performing jobs) enqueue_limit: 2, # Maximum number of jobs with the concurrency key to be concurrently performed (excludes enqueued jobs) perform_limit: 1, # A unique key to be globally locked against. # Can be String or Lambda/Proc that is invoked in the context of the job. # Note: Arguments passed to #perform_later must be accessed through `arguments` method. key: -> { "Unique-#{arguments.first}" } # MyJob.perform_later("Alice") => "Unique-Alice" ) def perform(first_name) # do work end end
When testing, the resulting concurrency key value can be inspected:
job = MyJob.perform_later("Alice") job.good_job_concurrency_key #=> "Unique-Alice"
Cron-style repeating/recurring jobs¶ ↑
GoodJob
can enqueue jobs on a recurring basis that can be used as a replacement for cron.
Cron-style jobs are run on every GoodJob
process (e.g. CLI or async
execution mode) when config.good_job.enable_cron = true
; use GoodJob's ActiveJob concurrency extension to limit the number of jobs that are enqueued.
Cron-format is parsed by the {fugit
} gem, which has support for seconds-level resolution (e.g. * * * * * *
).
# config/environments/application.rb or a specific environment e.g. production.rb # Enable cron in this process; e.g. only run on the first Heroku worker process config.good_job.enable_cron = ENV['DYNO'] == 'worker.1' # or `true` or via $GOOD_JOB_ENABLE_CRON # Configure cron with a hash that has a unique key for each recurring job config.good_job.cron = { # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(42, name: "Alice")` frequent_task: { # each recurring job must have a unique key cron: "*/15 * * * *", # cron-style scheduling format by fugit gem class: "ExampleJob", # reference the Job class with a string args: [42, { name: "Alice" }], # arguments to pass; can also be a proc e.g. `-> { { when: Time.now } }` set: { priority: -10 }, # additional ActiveJob properties; can also be a lambda/proc e.g. `-> { { priority: [1,2].sample } }` description: "Something helpful", # optional description that appears in Dashboard (coming soon!) }, another_task: { cron: "0 0,12 * * *", class: "AnotherJob", }, # etc. }
Updating¶ ↑
GoodJob
follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
Upgrading minor versions¶ ↑
Upgrading between minor versions (e.g. v1.4 to v1.5) should not introduce breaking changes, but can introduce new deprecation warnings and database migration notices.
To perform upgrades to the GoodJob
database tables:
-
Generate new database migration files:
bin/rails g good_job:update
Optional: If using Rails' multiple databases with the migrations_paths
configuration option, use the --database
option:
```bash $ bin/rails g good_job:update --database animals ```
-
Run the database migration locally
bin/rails db:migrate
-
Commit the migration files and resulting
db/schema.rb
changes. -
Deploy the code, run the migrations against the production database, and restart server/worker processes.
Upgrading v1 to v2¶ ↑
GoodJob
v2 introduces a new Advisory Lock key format that is different than the v1 advisory lock key format; it's therefore necessary to perform a simple, but staged production upgrade. If you are already using >= v1.12+
no other changes are necessary.
-
Upgrade your production environment to
v1.99.x
following the minor version upgrade process, including database migrations.v1.99
is a transitional release that is safely compatible with bothv1.x
andv2.0.0
because it uses bothv1
- andv2
-formatted advisory locks. -
Address any deprecation warnings generated by
v1.99
. -
Upgrade your production environment to
v1.99.x
tov2.0.x
again following the minor upgrade process.
Notable changes:
-
Renames
:async_server
execution mode to:async
; renames prior:async
execution mode to:async_all
. -
Sets default Development environment's execution mode to
:async
with disabled polling. -
Excludes performing jobs from
enqueue_limit
's count inGoodJob::ActiveJobExtensions::Concurrency
. -
Triggers
GoodJob.on_thread_error
for unhandledActiveJob
exceptions. -
Renames
GoodJob.reperform_jobs_on_standard_error
accessor toGoodJob.retry_on_unhandled_error
. -
Renames
GoodJob::Adapter.shutdown(wait:)
argument toGoodJob::Adapter.shutdown(timeout:)
. -
Changes Advisory Lock key format from
good_jobs[ROW_ID]
togood_jobs-[ACTIVE_JOB_ID]
. -
Expects presence of columns
good_jobs.active_job_id
,good_jobs.concurrency_key
,good_jobs.concurrency_key
, andgood_jobs.retried_good_job_id
.
Go deeper¶ ↑
Exceptions, retries, and reliability¶ ↑
GoodJob
guarantees that a completely-performed job will run once and only once. GoodJob
fully supports ActiveJob's built-in functionality for error handling, retries and timeouts.
Exceptions¶ ↑
ActiveJob
provides tools for rescuing and retrying exceptions, including retry_on
, discard_on
, rescue_from
that will rescue exceptions before they get to GoodJob
.
If errors do reach GoodJob
, you can assign a callable to GoodJob.on_thread_error
to be notified. For example, to log errors to an exception monitoring service like Sentry (or Bugsnag, Airbrake, Honeybadger, etc.):
# config/initializers/good_job.rb GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
Retries¶ ↑
By default, GoodJob
will automatically and immediately retry a job when an exception is raised to GoodJob
.
However, ActiveJob
can be configured to retry an infinite number of times, with an exponential backoff. Using ActiveJob's retry_on
prevents exceptions from reaching GoodJob:
class ApplicationJob < ActiveJob::Base retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY # ... end
When using retry_on
with a limited number of retries, the final exception will not be rescued and will raise to GoodJob
. GoodJob
can be configured to discard un-handled exceptions instead of retrying them:
# config/initializers/good_job.rb GoodJob.retry_on_unhandled_error = false
Alternatively, pass a block to retry_on
to handle the final exception instead of raising it to GoodJob:
class ApplicationJob < ActiveJob::Base retry_on StandardError, attempts: 5 do |_job, _exception| # Log error, do nothing, etc. end # ... end
When using retry_on
with an infinite number of retries, exceptions will never be raised to GoodJob
, which means GoodJob.on_thread_error
will never be called. To report log or report exceptions to an exception monitoring service (e.g. Sentry, Bugsnag, Airbrake, Honeybadger, etc), create an explicit exception wrapper. For example:
class ApplicationJob < ActiveJob::Base retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY retry_on SpecialError, attempts: 5 do |_job, exception| Raven.capture_exception(exception) end around_perform do |_job, block| block.call rescue StandardError => e Raven.capture_exception(e) raise end # ... end
ActionMailer retries¶ ↑
Any configuration in ApplicationJob
will have to be duplicated on ActionMailer::MailDeliveryJob
(ActionMailer::DeliveryJob
in Rails 5.2 or earlier) because ActionMailer uses a custom class, ActionMailer::MailDeliveryJob
, which inherits from ActiveJob::Base
, rather than your applications ApplicationJob
.
You can use an initializer to configure ActionMailer::MailDeliveryJob
, for example:
# config/initializers/good_job.rb ActionMailer::MailDeliveryJob.retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY # With Sentry (or Bugsnag, Airbrake, Honeybadger, etc.) ActionMailer::MailDeliveryJob.around_perform do |_job, block| block.call rescue StandardError => e Raven.capture_exception(e) raise end
Note, that ActionMailer::MailDeliveryJob
is a default since Rails 6.0. Be sure that your app is using that class, as it might also be configured to use (deprecated now) ActionMailer::DeliveryJob
.
Timeouts¶ ↑
Job timeouts can be configured with an around_perform
:
class ApplicationJob < ActiveJob::Base JobTimeoutError = Class.new(StandardError) around_perform do |_job, block| # Timeout jobs after 10 minutes Timeout.timeout(10.minutes, JobTimeoutError) do block.call end end end
Optimize queues, threads, and processes¶ ↑
By default, GoodJob
creates a single thread execution pool that will execute jobs from any queue. Depending on your application's workload, job types, and service level objectives, you may wish to optimize execution resources. For example, providing dedicated execution resources for transactional emails so they are not delayed by long-running batch jobs. Some options:
-
Multiple execution pools within a single process:
$ bundle exec good_job --queues="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" --max-threads=5
This configuration will result in a single process with 4 isolated thread execution pools. Isolated execution pools are separated with a semicolon (
;
) and queue names and thread counts with a colon (:
)-
transactional_messages:2
: execute jobs enqueued ontransactional_messages
with up to 2 threads. -
batch_processing:1
execute jobs enqueued onbatch_processing
with a single thread. -
-transactional_messages,batch_processing
: execute jobs enqueued on any queue excludingtransactional_messages
orbatch_processing
with up to 2 threads. -
*
: execute jobs on any queue on up to 5 threads, as configured by--max-threads=5
For moderate workloads, multiple isolated thread execution pools offers a good balance between congestion management and economy.
Configuration can be injected by environment variables too:
$ GOOD_JOB_QUEUES="transactional_messages:2;batch_processing:1;-transactional_messages,batch_processing:2;*" GOOD_JOB_MAX_THREADS=5 bundle exec good_job
-
-
Multiple processes; for example, on Heroku:
# Procfile # Separate dyno types worker: bundle exec good_job --max-threads=5 transactional_worker: bundle exec good_job --queues="transactional_messages" --max-threads=2 batch_worker: bundle exec good_job --queues="batch_processing" --max-threads=1 # Combined multi-process dyno combined_worker: bundle exec good_job --max-threads=5 & bundle exec good_job --queues="transactional_messages" --max-threads=2 & bundle exec good_job --queues="batch_processing" --max-threads=1 & wait -n
Running multiple processes can optimize for CPU performance at the expense of greater memory and system resource usage.
Keep in mind, queue operations and management is an advanced discipline. This stuff is complex, especially for heavy workloads and unique processing requirements. Good job π
Database connections¶ ↑
Each GoodJob
execution thread requires its own database connection that is automatically checked out from Railsβ connection pool. Allowing GoodJob to create more threads than available database connections can lead to timeouts and is not recommended. For example:
# config/database.yml pool: <%= [ENV.fetch("RAILS_MAX_THREADS", 5).to_i, ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i].max %>
Execute jobs async / in-process¶ ↑
GoodJob
can execute jobs βasyncβ in the same process as the web server (e.g. bin/rails s
). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
-
Via Rails configuration:
# config/environments/production.rb config.active_job.queue_adapter = :good_job # To change the execution mode config.good_job.execution_mode = :async # Or with more configuration config.good_job = { execution_mode: :async, max_threads: 4, poll_interval: 30 }
-
Or, with environment variables:
$ GOOD_JOB_EXECUTION_MODE=async GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
Depending on your application configuration, you may need to take additional steps:
-
Ensure that you have enough database connections for both web and job execution threads:
# config/database.yml pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i %>
-
When running Puma with workers (
WEB_CONCURRENCY > 0
) or another process-forking web server, GoodJob's threadpool schedulers should be stopped before forking, restarted after fork, and cleanly shut down on exit. Stopping GoodJob's scheduler pre-fork is recommended to ensure thatGoodJob
does not continue executing jobs in the parent/controller process. For example, with Puma:# config/puma.rb before_fork do GoodJob.shutdown end on_worker_boot do GoodJob.restart end on_worker_shutdown do GoodJob.shutdown end MAIN_PID = Process.pid at_exit do GoodJob.shutdown if Process.pid == MAIN_PID end
GoodJob
is compatible with Puma's preload_app!
method.
Migrate to GoodJob
from a different ActiveJob
backend¶ ↑
If your application is already using an ActiveJob
backend, you will need to install GoodJob
to enqueue and perform newly created jobs and finish performing pre-existing jobs on the previous backend.
-
Enqueue newly created jobs on
GoodJob
either entirely by settingActiveJob::Base.queue_adapter = :good_job
or progressively via individual job classes:# jobs/specific_job.rb class SpecificJob < ApplicationJob self.queue_adapter = :good_job # ... end
-
Continue running executors for both backends. For example, on Heroku it's possible to run two processes within the same dyno:
procfile # Procfile # ... worker: bundle exec que ./config/environment.rb & bundle exec good_job & wait -n
-
Once you are confident that no unperformed jobs remain in the previous
ActiveJob
backend, code and configuration for that backend can be completely removed.
Monitor and preserve worked jobs¶ ↑
GoodJob
is fully instrumented with {ActiveSupport::Notifications
}.
By default, GoodJob
will delete job records after they are run, regardless of whether they succeed or not (raising a kind of StandardError
), unless they are interrupted (raising a kind of Exception
).
To preserve job records for later inspection, set an initializer:
# config/initializers/good_job.rb GoodJob.preserve_job_records = true
It is also necessary to delete these preserved jobs from the database after a certain time period:
-
For example, in a Rake task:
GoodJob.cleanup_preserved_jobs # Will keep 1 day of job records by default. GoodJob.cleanup_preserved_jobs(older_than: 7.days) # It also takes custom arguments.
-
For example, using the
good_job
command-line utility:$ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
PgBouncer compatibility¶ ↑
GoodJob
is not compatible with PgBouncer in transaction mode, but is compatible with PgBouncer's connection mode. GoodJob
uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
A workaround to this limitation is to make a direct database connection available to GoodJob
. With Rails 6.0's support for multiple databases, a direct connection to the database can be configured:
-
Define a direct connection to your database that is not proxied through PgBouncer, for example:
# config/database.yml production: primary: url: postgres://pgbouncer_host/my_database primary_direct: url: postgres://database_host/my_database
-
Create a new ActiveRecord base class that uses the direct database connection
# app/models/application_direct_record.rb class ApplicationDirectRecord < ActiveRecord::Base self.abstract_class = true connects_to database: :primary_direct end
-
Configure
GoodJob
to use the newly created ActiveRecord base class:# config/initializers/good_job.rb GoodJob.active_record_parent_class = "ApplicationDirectRecord"
Contribute¶ ↑
Contributions are welcomed and appreciated π
-
Review the Prioritized Project Backlog.
-
Open a new Issue or contribute to an existing Issue. Questions or suggestions are fantastic.
-
Participate according to our Code of Conduct.
Gem development¶ ↑
To run tests:
# Clone the repository locally $ git clone git@github.com:bensheldon/good_job.git # Set up the local environment $ bin/setup # Run the tests $ bin/rspec
This gem uses Appraisal to run tests against multiple versions of Rails:
# Install Appraisal(s) gemfiles $ bundle exec appraisal # Run tests $ bundle exec appraisal bin/rspec
For developing locally within another Ruby on Rails project:
# Within Ruby on Rails directory... $ bundle config local.good_job /path/to/local/git/repository # Confirm that the local copy is used $ bundle install # => Using good_job 0.1.0 from https://github.com/bensheldon/good_job.git (at /Users/You/Projects/good_job@dc57fb0)
Release¶ ↑
Package maintainers can release this gem by running:
# Sign into rubygems $ gem signin # Add a .env file with the following: # CHANGELOG_GITHUB_TOKEN= # Github Personal Access Token # Update version number, changelog, and create git commit: $ bundle exec rake release[minor] # major,minor,patch # ..and follow subsequent directions.
License¶ ↑
The gem is available as open source under the terms of the MIT License.