module TransactionReliability::Helpers

Public Instance Methods

transaction_with_retry(txn_options = {}, retry_options = {}) { || ... } click to toggle source

Execute some code in a DB transaction, with retry

# File lib/transaction_reliability.rb, line 129
def transaction_with_retry(txn_options = {}, retry_options = {})
  base_obj = self.respond_to?(:transaction) ? self : ActiveRecord::Base

  with_transaction_retry(retry_options) do
    base_obj.transaction(txn_options) do
      yield
    end
  end
end
with_transaction_retry(options = {}) { || ... } click to toggle source

Intended to be included in an ActiveRecord model class.

Retries a block (which usually contains a transaction) under certain failure conditions, up to a configurable number of times with an exponential backoff delay between each attempt.

Conditions for retrying:

1. Database connection lost
2. Query or txn failed due to detected deadlock 
   (Mysql/InnoDB and Postgres can both signal this for just about 
    any transaction)
3. Query or txn failed due to serialization failure 
   (Postgres will signal this for transactions in isolation 
    level SERIALIZABLE)

options:

retry_count  - how many retries to make; default 4

backoff      - time period before 1st retry, in fractional seconds.
               will double at every retry. default 0.25 seconds.

exit_on_disconnect 
             - whether to call exit if no retry succeeds and
               the cause is a failed connection

exit_on_fail - whether to call exit if no retry succeeds

defaults:

# File lib/transaction_reliability.rb, line 76
def with_transaction_retry(options = {})
  retry_count         = options.fetch(:retry_count,            4)
  backoff             = options.fetch(:backoff,             0.25)
  exit_on_fail        = options.fetch(:exit_on_fail,       false)
  exit_on_disconnect  = options.fetch(:exit_on_disconnect,  true)

  count               = 0

  # list of exceptions we may catch
  exceptions = ['ActiveRecord::StatementInvalid', 'PG::Error', 'Mysql2::Error'].
                 map {|name| name.safe_constantize}.
                 compact

  #
  # There are times when, for example,
  # a raw PG::Error is throw rather than a wrapped ActiveRecord::StatementInvalid
  #
  # Also, connector-specific classes like PG::Error may not be defined
  #
  begin
    connection_lost = false
    yield
  rescue *exceptions => e
    translated = TransactionReliability.rewrap_exception(e)

    case translated
      when ConnectionLost
        (options[:connection] || ActiveRecord::Base.connection).reconnect!
        connection_lost = true
      when DeadlockDetected, SerializationFailure
      else
        raise translated
    end

    # Retry up to retry_count times
    if count < retry_count
      sleep backoff
      count   += 1
      backoff *= 2
      retry
    else
      if (connection_lost && exit_on_disconnect) || exit_on_fail
        exit
      else 
        raise(translated)
      end
    end
  end
end