module Canistor
Replacement for the HTTP handler in the AWS SDK that mocks all interaction with S3 just above the HTTP level.
The mock implementation is turned on by removing the NetHttp handlers that comes with the library by the Canistor
handler.
Aws::S3::Client.remove_plugin(Seahorse::Client::Plugins::NetHttp) Aws::S3::Client.add_plugin(Canistor::Plugin)
The Canistor
instance then needs to be configured with buckets and credentials to be useful. It can be configured using either the config method on the instance or by specifying the buckets one by one.
In the example below Canistor
will have two accounts and four buckets. It also specifies which accounts can access the buckets.
Canistor.config( logger: Rails.logger, credentials: { { access_key_id: 'AKIAIXXXXXXXXXXXXXX1', secret_access_key: 'phRL+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1' }, { access_key_id: 'AKIAIXXXXXXXXXXXXXX2', secret_access_key: 'phRL+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx2' } }, buckets: { 'us-east-1' => { 'io-canistor-production-images' => { allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1'], replicate_to: [ 'eu-central-1:io-canistor-production-images-replica' ], versioned: true }, 'io-canistor-production-books' => { allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1', 'AKIAIXXXXXXXXXXXXXX2'] } }, 'eu-central-1' => { 'io-canistor-production-sales' => { allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1'] }, 'io-canistor-production-images-replica' => { allow_access_keys: ['AKIAIXXXXXXXXXXXXXX1'], versioned: true } } } )
Canistor
implements basic interaction with buckets and objects. It also verifies authentication information. It does not implement control lists so all accounts have full access to the buckets and objects.
It's possible to turn on replication and versioning. Note that replication is instant in the mock. On actual S3 it takes a while for objects to replicate. Not the entire replication and versioning API is implemented.
The mock can simulate a number of failures. These are triggered by setting the operation which needs to fail on the mock. For more information see [Canistor.fail].
In most cases you should configure the suite to clear the mock before running each example with [Canistor.clear].
Constants
- SUPPORTED_FAILURES
- VERSION
Attributes
Public Class Methods
# File lib/canistor.rb, line 149 def self.buckets=(buckets) buckets.each do |region, attributes| attributes.each do |bucket, options| bucket = create_bucket(region, bucket) bucket.update_settings(options) bucket end end end
Clears the state of the mock. Leaves all the credentials and buckets but removes all objects and mocked responses.
# File lib/canistor.rb, line 239 def self.clear @fail = [] @store.each do |region, buckets| buckets.each do |bucket_name, bucket| bucket.clear end end end
# File lib/canistor.rb, line 170 def self.config(config) config.each do |section, attributes| public_send("#{section}=", attributes) end end
Configures a bucket in the mock implementation. Use update_settings on the Container object returned by this method to configure who may access the bucket.
# File lib/canistor.rb, line 162 def self.create_bucket(region, bucket_name) store[region] ||= {} store[region][bucket_name] = Canistor::Storage::Bucket.new( region: region, name: bucket_name ) end
# File lib/canistor.rb, line 136 def self.credentials=(accounts) accounts.each do |attributes| unless attributes.keys.map(&:to_s) == %w(access_key_id secret_access_key) raise( ArgumentError, "Credentials need to specify access_key_id and secret_access_key, " \ "got: `#{attributes.keys.inspect}'" ) end end credentials.merge(accounts) end
# File lib/canistor.rb, line 111 def self.find_bucket(region, bucket) store.dig(region, bucket) || find_bucket_by_name_and_warn(region, bucket) end
# File lib/canistor.rb, line 127 def self.find_bucket_by_name(bucket) store.each do |_, buckets| if found = buckets[bucket] return found end end nil end
# File lib/canistor.rb, line 115 def self.find_bucket_by_name_and_warn(region, bucket) found = find_bucket_by_name(bucket) return if found.nil? logger.info( "S3 client configured for \"#{region}\" but the bucket \"#{bucket}\" " \ "is in \"#{found.region}\"; Please configure the proper region to " \ "avoid multiple unnecessary redirects and signing attempts" ) if logger found end
# File lib/canistor.rb, line 97 def self.find_credentials(authorization) if authorization.access_key_id credentials.each do |attributes| if authorization.access_key_id == attributes[:access_key_id] return Aws::Credentials.new( attributes[:access_key_id], attributes[:secret_access_key] ) end end end nil end
Executes the block when the operation is in the failure queue and removes one instance of the operation.
# File lib/canistor.rb, line 225 def self.take_fail(operation, &block) fail_mutex.synchronize do if index = @fail.index(operation) begin block.call ensure @fail.delete_at(index) end end end end