module Canistor

Replacement for the HTTP handler in the AWS SDK that mocks all interaction with S3 just above the HTTP level.

The mock implementation is turned on by removing the NetHttp handlers that comes with the library by the Canistor handler.

Aws::S3::Client.remove_plugin(Seahorse::Client::Plugins::NetHttp)
Aws::S3::Client.add_plugin(Canistor::Plugin)

The Canistor instance then needs to be configured with buckets and credentials to be useful. It can be configured using either the config method on the instance or by specifying the buckets one by one.

In the example below Canistor will have two accounts and four buckets. It also specifies which accounts can access the buckets.

Canistor.config(
  logger: Rails.logger,
  credentials: {
    {
      access_key_id: 'AKIAIXXXXXXXXXXXXXX1',
      secret_access_key: 'phRL+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1'
    },
    {
      access_key_id: 'AKIAIXXXXXXXXXXXXXX2',
      secret_access_key: 'phRL+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx2'
    }
  },
  buckets: {
    'us-east-1' => {
      'io-canistor-production-images' => {
        allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1'],
        replicate_to: [
          'eu-central-1:io-canistor-production-images-replica'
        ],
        versioned: true
      },
      'io-canistor-production-books' => {
        allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1', 'AKIAIXXXXXXXXXXXXXX2']
      }
    },
    'eu-central-1' => {
      'io-canistor-production-sales' => {
        allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1']
      },
      'io-canistor-production-images-replica' => {
        allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1'],
        versioned: true
      }
    }
  }
)

Canistor implements basic interaction with buckets and objects. It also verifies authentication information. It does not implement control lists so all accounts have full access to the buckets and objects.

It's possible to turn on replication and versioning. Note that replication is instant in the mock. On actual S3 it takes a while for objects to replicate. Not the entire replication and versioning API is implemented.

The mock can simulate a number of failures. These are triggered by setting the operation which needs to fail on the mock. For more information see [Canistor.fail].

In most cases you should configure the suite to clear the mock before running each example with [Canistor.clear].

Constants

SUPPORTED_FAILURES
VERSION

Attributes

credentials[R]

fail[R]

fail_mutex[R]

logger[RW]

store[RW]

Public Class Methods

buckets=(buckets) click to toggle source

# File lib/canistor.rb, line 149
def self.buckets=(buckets)
  buckets.each do |region, attributes|
    attributes.each do |bucket, options|
      bucket = create_bucket(region, bucket)
      bucket.update_settings(options)
      bucket
    end
  end
end

clear() click to toggle source

Clears the state of the mock. Leaves all the credentials and buckets but removes all objects and mocked responses.

# File lib/canistor.rb, line 239
def self.clear
  @fail = []
  @store.each do |region, buckets|
    buckets.each do |bucket_name, bucket|
      bucket.clear
    end
  end
end

config(config) click to toggle source

# File lib/canistor.rb, line 170
def self.config(config)
  config.each do |section, attributes|
    public_send("#{section}=", attributes)
  end
end

create_bucket(region, bucket_name) click to toggle source

Configures a bucket in the mock implementation. Use update_settings on the Container object returned by this method to configure who may access the bucket.

# File lib/canistor.rb, line 162
def self.create_bucket(region, bucket_name)
  store[region] ||= {}
  store[region][bucket_name] = Canistor::Storage::Bucket.new(
    region: region,
    name: bucket_name
  )
end

credentials=(accounts) click to toggle source

# File lib/canistor.rb, line 136
def self.credentials=(accounts)
  accounts.each do |attributes|
    unless attributes.keys.map(&:to_s) == %w(access_key_id secret_access_key)
      raise(
        ArgumentError,
        "Credentials need to specify access_key_id and secret_access_key, " \
        "got: `#{attributes.keys.inspect}'"
      )
    end
  end
  credentials.merge(accounts)
end

find_bucket(region, bucket) click to toggle source

# File lib/canistor.rb, line 111
def self.find_bucket(region, bucket)
  store.dig(region, bucket) || find_bucket_by_name_and_warn(region, bucket)
end

find_bucket_by_name(bucket) click to toggle source

# File lib/canistor.rb, line 127
def self.find_bucket_by_name(bucket)
  store.each do |_, buckets|
    if found = buckets[bucket]
      return found
    end
  end
  nil
end

find_bucket_by_name_and_warn(region, bucket) click to toggle source

# File lib/canistor.rb, line 115
def self.find_bucket_by_name_and_warn(region, bucket)
  found = find_bucket_by_name(bucket)
  return if found.nil?

  logger.info(
    "S3 client configured for \"#{region}\" but the bucket \"#{bucket}\" " \
    "is in \"#{found.region}\"; Please configure the proper region to " \
    "avoid multiple unnecessary redirects and signing attempts"
  ) if logger
  found
end

find_credentials(authorization) click to toggle source

# File lib/canistor.rb, line 97
def self.find_credentials(authorization)
  if authorization.access_key_id
    credentials.each do |attributes|
      if authorization.access_key_id == attributes[:access_key_id]
        return Aws::Credentials.new(
          attributes[:access_key_id],
          attributes[:secret_access_key]
        )
      end
    end
  end
  nil
end

take_fail(operation, &block) click to toggle source

Executes the block when the operation is in the failure queue and removes one instance of the operation.

# File lib/canistor.rb, line 225
def self.take_fail(operation, &block)
  fail_mutex.synchronize do
    if index = @fail.index(operation)
      begin
        block.call
      ensure
        @fail.delete_at(index)
      end
    end
  end
end