class RFlow::Configuration

Contains all the configuration data and methods for RFlow. Interacts directly with underlying SQLite database, and keeps a registry of available data types, extensions, and components. Also includes an external DSL, RubyDSL, that can be used in crafting config-like files that load the database.

{Configuration} provides a MVC-like framework for config files, where the models are the {Setting}, {Component}, {Port}, and {Connection} subclasses, the controllers are things like RubyDSL, and the views are defined relative to the controllers.

Public Class Methods

add_available_component(component) click to toggle source

Used when {RFlow::Component} is subclassed to add another available component to the list. @return [void]

# File lib/rflow/configuration.rb, line 108
def add_available_component(component)
  if available_components.include?(component.name)
    raise ArgumentError, "Component already '#{component.name}' already defined"
  end
  available_components[component.name] = component
end
add_available_data_extension(data_type_name, extension) click to toggle source

Add a data extension to the {available_data_extensions} class attribute. The extension parameter should be the name of a ruby module that will extend {RFlow::Message::Data} to provide additional methods/capability. Naive, prefix-based inheritance is possible, see {available_data_extensions} or {DataExtensionCollection}. @return [void]

# File lib/rflow/configuration.rb, line 97
def add_available_data_extension(data_type_name, extension)
  unless extension.is_a? Module
    raise ArgumentError, "Invalid data extension #{extension} for #{data_type_name}.  Only Ruby Modules allowed"
  end

  available_data_extensions.add data_type_name, extension
end
add_available_data_type(name, serialization_type, schema) click to toggle source

Add a schema to the {available_data_types} class attribute. Schema is indexed by name and serialization_type. avro is currently the only supported serialization_type. @return [void]

# File lib/rflow/configuration.rb, line 78
def add_available_data_type(name, serialization_type, schema)
  # TODO: refactor each of these add_available_* into collections to
  # make DRYer.  Also figure out what to do with all to to_syms
  raise ArgumentError, "Data serialization_type must be 'avro' for '#{name}'" unless serialization_type == 'avro'

  if available_data_types[name.to_s].include? serialization_type.to_s
    raise ArgumentError, "Data type '#{name}' already defined for serialization_type '#{serialization_type}'"
  end

  available_data_types[name.to_s][serialization_type.to_s] = schema
end
available_components() click to toggle source

A Hash of defined components, usually automatically populated when a component subclasses {RFlow::Component}. @return [Hash]

# File lib/rflow/configuration.rb, line 70
def available_components
  @available_components ||= {}
end
available_data_extensions() click to toggle source

A {DataExtensionCollection} to hold available extensions that will be applied to the de-serialized data types. @return [DataExtensionCollection]

# File lib/rflow/configuration.rb, line 63
def available_data_extensions
  @available_data_extensions ||= DataExtensionCollection.new
end
available_data_types() click to toggle source

A collection of data types (schemas) indexed by their name and their schema type ('avro'). @return [Hash]

# File lib/rflow/configuration.rb, line 56
def available_data_types
  @available_data_types ||= Hash.new {|hash, key| hash[key] = {}}
end
establish_config_database_connection(database_path) click to toggle source

Connect to the configuration SQLite database, but use {ConfigurationItem} to protect the connection information from other ActiveRecord apps (i.e. Rails). @return [void]

# File lib/rflow/configuration.rb, line 119
def establish_config_database_connection(database_path)
  RFlow.logger.debug "Establishing connection to config database (#{Dir.getwd}) '#{database_path}'"
  ActiveRecord::Base.logger = RFlow.logger
  ConfigurationItem.establish_connection(:adapter => 'sqlite3', :database => database_path)
end
initialize_database(database_path, config_file_path = nil) click to toggle source

Connect to the configuration database, migrate it to the latest version, and process a config file if provided. @return [void]

# File lib/rflow/configuration.rb, line 146
def initialize_database(database_path, config_file_path = nil)
  RFlow.logger.debug "Initializing config database (#{Dir.getwd}) '#{database_path}'"

  # TODO should not need this line
  ActiveRecord::Base.establish_connection(:adapter => 'sqlite3', :database => database_path)

  establish_config_database_connection database_path
  migrate_database

  working_dir = Dir.getwd
  Dir.chdir File.dirname(database_path)

  if config_file_path
    process_config_file File.expand_path(config_file_path)
  end

  RFlow.logger.debug 'Defaulting non-existing config values'
  merge_defaults!

  Dir.chdir working_dir

  self.new(database_path)
end
merge_defaults!() click to toggle source

Make sure that the configuration has all the necessary values set. @return [void]

# File lib/rflow/configuration.rb, line 172
def merge_defaults!
  Setting::DEFAULTS.each do |name, default_value_or_proc|
    value = default_value_or_proc.is_a?(Proc) ? default_value_or_proc.call() : default_value_or_proc
    setting = Setting.find_or_create_by(:name => name, :value => value)
    unless setting.valid?
      raise RuntimeError, setting.errors.map {|_, msg| msg }.join(', ')
    end
  end
end
migrate_database() click to toggle source

Using default ActiveRecord migrations, attempt to migrate the database to the latest version. @return [void]

# File lib/rflow/configuration.rb, line 128
def migrate_database
  RFlow.logger.debug 'Applying default migrations to config database'
  migrations_path = File.join(File.dirname(__FILE__), 'configuration', 'migrations')
  ActiveRecord::Migration.verbose = false
  ActiveRecord::Migrator.migrate migrations_path
end
new(database_path = nil) click to toggle source
# File lib/rflow/configuration.rb, line 183
def initialize(database_path = nil)
  # If there is not a config DB path, assume that an AR
  # connection has already been established
  if database_path
    @database_path = database_path
    Configuration.establish_config_database_connection(database_path)
  end

  # Validate the connected database.
  # TODO: make this more complete, i.e. validate the various columns
  begin
    [Setting, Shard, Component, Port, Connection].each(&:first)
  rescue ActiveRecord::StatementInvalid => e
    raise ArgumentError, "Invalid schema in configuration database: #{e.message}"
  end
end
process_config_file(path) click to toggle source

Load the config file, which should load/process/store all the elements. Only run this after the database has been setup @return [void]

# File lib/rflow/configuration.rb, line 138
def process_config_file(path)
  RFlow.logger.info "Processing config file (#{Dir.getwd}) '#{path}'"
  load path
end

Public Instance Methods

[](name) click to toggle source

Retrieve a setting value by name from the SQLite database. @return [Object]

# File lib/rflow/configuration.rb, line 228
def [](name); Setting.find_by_name(name).value rescue nil; end
available_components() click to toggle source

Retrieve the mapping from component name to {Component}. @return [Hash]

# File lib/rflow/configuration.rb, line 256
def available_components; Configuration.available_components; end
component(uuid) click to toggle source

Retrieve a single {Component} by UUID from the SQLite database. @return [Shard]

# File lib/rflow/configuration.rb, line 252
def component(uuid); Component.find_by_uuid uuid; end
components() click to toggle source

Retrieve all the {Component}s from the SQLite database. @return [Array<Component>]

# File lib/rflow/configuration.rb, line 248
def components; Component.all; end
connections() click to toggle source

Retrieve all the {Connection}s from the SQLite database. @return [Array<Connection>]

# File lib/rflow/configuration.rb, line 240
def connections; Connection.all; end
settings() click to toggle source

Retrieve all the {Setting}s from the SQLite database. @return [Array<Setting>]

# File lib/rflow/configuration.rb, line 232
def settings; Setting.all; end
shard(uuid) click to toggle source

Retrieve a single {Shard} by UUID from the SQLite database. @return [Shard]

# File lib/rflow/configuration.rb, line 244
def shard(uuid); Shard.find_by_uuid uuid; end
shards() click to toggle source

Retrieve all the {Shard}s from the SQLite database. @return [Array<Shard>]

# File lib/rflow/configuration.rb, line 236
def shards; Shard.all; end
to_s() click to toggle source

Output the RFlow configuration to a pretty-printed String. @return [String]

# File lib/rflow/configuration.rb, line 202
def to_s
  string = "Configuration:\n"

  settings.each do |setting|
    string << "Setting: '#{setting.name}' = '#{setting.value}'\n"
  end

  shards.each do |shard|
    string << "Shard #{shard.name} (#{shard.uuid}), type #{shard.class.name}, count #{shard.count}\n"
    shard.components.each do |component|
      string << "  Component '#{component.name}' as #{component.specification} (#{component.uuid})\n"
      component.output_ports.each do |output_port|
        output_port.output_connections.each do |output_connection|
          input_port = output_connection.input_port
          string << "    OutputPort '#{output_port.name}' key '#{output_connection.output_port_key}' (#{output_port.uuid}) =>\n"
          string << "      Connection '#{output_connection.name}' as #{output_connection.type} (#{output_connection.uuid}) =>\n"
          string << "      InputPort '#{input_port.name}' key '#{output_connection.input_port_key}' (#{input_port.uuid}) Component '#{input_port.component.name}' (#{input_port.component.uuid})\n"
        end
      end
    end
  end
  string
end