module Chronicle::ETL::Catalog
Utility methods to catalogue which Extractor
, Transformer
, and Loader
connector classes are available to chronicle-etl
Constants
- BUILTIN
- PHASES
- PLUGINS
Public Class Methods
available_classes()
click to toggle source
Return which ETL
connectors are available, both built in and externally-defined
# File lib/chronicle/etl/catalog.rb, line 15 def self.available_classes # TODO: have a registry of plugins # Attempt to load each chronicle plugin that we might know about so # that we can later search for subclasses to build our list of # available classes PLUGINS.each do |plugin| require "chronicle/#{plugin}" rescue LoadError # this will happen if the gem isn't available globally end parent_klasses = [ ::Chronicle::ETL::Extractor, ::Chronicle::ETL::Transformer, ::Chronicle::ETL::Loader ] klasses = [] parent_klasses.map do |parent| klasses += ::ObjectSpace.each_object(::Class).select { |klass| klass < parent } end klasses.map do |klass| { name: klass.name, built_in: klass.built_in?, provider: klass.provider, phase: klass.phase } end end
identifier_to_klass(identifier:, phase:)
click to toggle source
For a given connector identifier, return the class (either builtin, or from a external chronicle gem)
# File lib/chronicle/etl/catalog.rb, line 54 def self.identifier_to_klass(identifier:, phase:) if BUILTIN[phase].include? identifier load_builtin_klass(name: identifier, phase: phase) else provider, name = identifier.split(':') name ||= '' load_provider_klass(provider: provider, name: name, phase: phase) end end
phase_and_identifier_to_klass(phase, identifier)
click to toggle source
Take a phase (e, t, or l) and an identifier and return the right class
# File lib/chronicle/etl/catalog.rb, line 48 def self.phase_and_identifier_to_klass(phase, identifier) Chronicle::ETL::Catalog.identifier_to_klass(phase: phase, identifier: identifier) end
Private Class Methods
load_builtin_klass(name:, phase:)
click to toggle source
# File lib/chronicle/etl/catalog.rb, line 86 def self.load_builtin_klass(name:, phase:) klass_str = "Chronicle::ETL::#{name.capitalize}#{phase.capitalize}" begin Object.const_get(klass_str) rescue NameError => e raise ConnectorNotAvailableError.new("Connector not found", name: name) end end
load_provider_klass(name: '', phase:, provider:)
click to toggle source
# File lib/chronicle/etl/catalog.rb, line 95 def self.load_provider_klass(name: '', phase:, provider:) begin require "chronicle/#{provider}" klass_str = "Chronicle::#{provider.capitalize}::#{name.capitalize}#{phase.capitalize}" Object.const_get(klass_str) rescue LoadError => e raise ProviderNotAvailableError.new("Provider '#{provider.capitalize}' could not be loaded", provider: provider) rescue NameError => e raise ProviderConnectorNotAvailableError.new("Connector '#{name}' in '#{provider}' could not be found", provider: provider, name: name) end end
Public Instance Methods
built_in?()
click to toggle source
Returns whether this connector is a built-in one
# File lib/chronicle/etl/catalog.rb, line 80 def built_in? to_s.include? 'Chronicle::ETL' end
phase()
click to toggle source
Returns whether a class is an Extractor
, Transformer
, or Loader
# File lib/chronicle/etl/catalog.rb, line 65 def phase ancestors = self.ancestors return :extractor if ancestors.include? Chronicle::ETL::Extractor return :transformer if ancestors.include? Chronicle::ETL::Transformer return :loader if ancestors.include? Chronicle::ETL::Loader end
provider()
click to toggle source
Returns which third-party provider this connector is associated wtih
# File lib/chronicle/etl/catalog.rb, line 73 def provider # TODO: needs better convention for a gem reporting its provider name provider = to_s.split('::')[1].downcase provider == 'etl' ? 'chronicle' : provider end