class Stockboy::Configurator

Context for evaluating DSL templates and capturing job options for initializing a job.

Wraps up the DSL methods called in job templates and handles the construction of the job's provider, reader, attributes, and filters.

Attributes

config[R]

Captured job configuration options

@return [Hash]

Public Class Methods

new(dsl='', file=__FILE__, &block) click to toggle source

Evaluate DSL and capture configuration for building a job

@overload new(dsl, file=__FILE__)

Evaluate DSL from a string
@param [String] dsl   job template language for evaluation
@param [String] file  path to original file for reporting errors

@overload new(&block)

Evaluate DSL in a block
# File lib/stockboy/configurator.rb, line 32
def initialize(dsl='', file=__FILE__, &block)
  @config = {}
  @config[:triggers] = Hash.new { |hash, key| hash[key] = [] }
  @config[:filters] = {}
  if block_given?
    instance_eval(&block)
  else
    instance_eval(dsl, file)
  end
end

Public Instance Methods

attribute(key, opts={}) click to toggle source

Add individual attribute mapping rules

@param [Symbol] key Name of the output attribute @param [Hash] opts @option opts [String] from Name of input field from reader @option opts [Array,Proc,Translator] as One or more translators

# File lib/stockboy/configurator.rb, line 144
def attribute(key, opts={})
  @config[:attributes] ||= AttributeMap.new
  @config[:attributes].insert(key, opts)
end
attributes(&block) click to toggle source

Configure the attribute map for data records

This will replace any existing attributes with a new set.

@example

attributes do
  first_name as: ->(raw){ raw["FullName"].split(" ").first }
  email      from: "RawEmail", as: [:string]
  check_in   from: "RawCheckIn", as: [:date]
end
# File lib/stockboy/configurator.rb, line 130
def attributes(&block)
  raise ArgumentError unless block_given?

  @config[:attributes] = AttributeMap.new(&block)
end
connection(key, opts={}, &block)
Alias for: provider
filter(key, callable=nil, *args, &block) click to toggle source

Add a filter to the processing filter chain

  • Must be called with either a callable argument (proc) or a block.

  • Must be called in the order that filters should be applied.

@example

filter :missing_email do |raw, out|
  raw["RawEmail"].empty?
end
filter :past_due do |raw, out|
  out.check_in < Date.today
end
filter :under_age, :check_id
filter :update, proc{ true } # capture all remaining items
# File lib/stockboy/configurator.rb, line 164
def filter(key, callable=nil, *args, &block)
  filter = Filters.build(callable, args, block) || block
  filter or raise ArgumentError, "Missing filter arguments for #{key}"
  @config[:filters][key] = filter
end
format(key, opts={}, &block)
Alias for: reader
on(key, &block) click to toggle source

Register a trigger to notify the job of external events

Useful for adding generic control over the job's resources from your app. For example, if you need to record stats or clean up data after your application has successfully processed the records, these actions can be defined within the context of each job template.

@param [Symbol] key Name of the trigger @yieldparam [Stockboy::Job] @yieldparam [Array] Arguments passed to the action when called

@example

trigger :cleanup do |job, *args|
  job.provider.delete_data
end

# elsewhere:
if MyProjects.find(123).import_records(job.records[:valid])
  job.cleanup
end
# File lib/stockboy/configurator.rb, line 191
def on(key, &block)
  raise(ArgumentError, "no block given") unless block_given?

  @config[:triggers][key] << block
end
provider(key, opts={}, &block) click to toggle source

Configure the provider for fetching data

The optional block is evaluated in the provider's own DSL context.

@param [Symbol, Class, Provider] key

The registered symbol name for the provider, or actual provider

@param [Hash] opts

Provider-specific options passed to the provider initializer

@example

provider :file, file_dir: "/downloads/@client" do
  file_name "example.csv"
end

@return [Provider]

# File lib/stockboy/configurator.rb, line 59
def provider(key, opts={}, &block)
  @config[:provider] = Providers.build(key, opts, block)
end
Also aliased as: connection
reader(key, opts={}, &block) click to toggle source

Configure the reader for parsing data

@param [Symbol, Class, Reader] key

The registered symbol name for the reader, or actual reader instance

@param [Hash] opts

Provider-specific options passed to the provider initializer

@example

reader :csv do
  col_sep "|"
end

@return [Reader]

# File lib/stockboy/configurator.rb, line 114
def reader(key, opts={}, &block)
  @config[:reader] = Readers.build(key, opts, block)
end
Also aliased as: format
repeat(&block) click to toggle source

Configure repeating the provider for fetching multiple parts

If the provider needs to give us all the data as a series of requests, for example multiple HTTP pages or FTP files, the repeat block can be used to define the iteration for fetching each item.

The `<<` interface used here is defined by Ruby's Enumerator.new block syntax. For each page that needs to be fetched, the provider options need to be altered and pushed on to the output. Control will be yielded to the reader at each iteration.

@example

repeat do |output, provider|
  loop do
    output << provider
    break if provider.data.split("\n").size < 100
    provider.query_params["page"] += 1
  end
end

@example

repeat do |output, provider|
  1.upto 10 do |i|
    provider.file_name = "example-#{i}.log"
    output << provider
  end
end
# File lib/stockboy/configurator.rb, line 92
def repeat(&block)
  unless block_given? && block.arity == 2
    raise ArgumentError, "repeat block must accept |output, provider| arguments"
  end

  @config[:repeat] = block
end
to_job() click to toggle source

Initialize a new job with the captured options

@return [Job]

# File lib/stockboy/configurator.rb, line 201
def to_job
  Job.new(config_for_job)
end

Private Instance Methods

config_for_job() click to toggle source
# File lib/stockboy/configurator.rb, line 207
def config_for_job
  config.dup.tap { |config| wrap_provider(config) }
end
wrap_provider(config) click to toggle source
# File lib/stockboy/configurator.rb, line 211
def wrap_provider(config)
  return unless (repeat = config.delete(:repeat))
  config[:provider] = ProviderRepeater.new(config[:provider], &repeat)
end