module Chronicle::ETL::Catalog

Utility methods to catalogue which Extractor, Transformer, and Loader connector classes are available to chronicle-etl

Constants

BUILTIN
PHASES
PLUGINS

Public Class Methods

available_classes() click to toggle source

Return which ETL connectors are available, both built in and externally-defined

# File lib/chronicle/etl/catalog.rb, line 15
def self.available_classes
  # TODO: have a registry of plugins

  # Attempt to load each chronicle plugin that we might know about so
  # that we can later search for subclasses to build our list of
  # available classes
  PLUGINS.each do |plugin|
    require "chronicle/#{plugin}"
  rescue LoadError
    # this will happen if the gem isn't available globally
  end

  parent_klasses = [
    ::Chronicle::ETL::Extractor,
    ::Chronicle::ETL::Transformer,
    ::Chronicle::ETL::Loader
  ]
  klasses = []
  parent_klasses.map do |parent|
    klasses += ::ObjectSpace.each_object(::Class).select { |klass| klass < parent }
  end

  klasses.map do |klass|
    {
      name: klass.name,
      built_in: klass.built_in?,
      provider: klass.provider,
      phase: klass.phase
    }
  end
end
identifier_to_klass(identifier:, phase:) click to toggle source

For a given connector identifier, return the class (either builtin, or from a external chronicle gem)

# File lib/chronicle/etl/catalog.rb, line 54
def self.identifier_to_klass(identifier:, phase:)
  if BUILTIN[phase].include? identifier
    load_builtin_klass(name: identifier, phase: phase)
  else
    provider, name = identifier.split(':')
    name ||= ''
    load_provider_klass(provider: provider, name: name, phase: phase)
  end
end
phase_and_identifier_to_klass(phase, identifier) click to toggle source

Take a phase (e, t, or l) and an identifier and return the right class

# File lib/chronicle/etl/catalog.rb, line 48
def self.phase_and_identifier_to_klass(phase, identifier)
  Chronicle::ETL::Catalog.identifier_to_klass(phase: phase, identifier: identifier)
end

Private Class Methods

load_builtin_klass(name:, phase:) click to toggle source
# File lib/chronicle/etl/catalog.rb, line 86
def self.load_builtin_klass(name:, phase:)
  klass_str = "Chronicle::ETL::#{name.capitalize}#{phase.capitalize}"
  begin
    Object.const_get(klass_str)
  rescue NameError => e
    raise ConnectorNotAvailableError.new("Connector not found", name: name)
  end
end
load_provider_klass(name: '', phase:, provider:) click to toggle source
# File lib/chronicle/etl/catalog.rb, line 95
def self.load_provider_klass(name: '', phase:, provider:)
  begin
    require "chronicle/#{provider}"
    klass_str = "Chronicle::#{provider.capitalize}::#{name.capitalize}#{phase.capitalize}"
    Object.const_get(klass_str)
  rescue LoadError => e
    raise ProviderNotAvailableError.new("Provider '#{provider.capitalize}' could not be loaded", provider: provider)
  rescue NameError => e
    raise ProviderConnectorNotAvailableError.new("Connector '#{name}' in '#{provider}' could not be found", provider: provider, name: name)
  end
end

Public Instance Methods

built_in?() click to toggle source

Returns whether this connector is a built-in one

# File lib/chronicle/etl/catalog.rb, line 80
def built_in?
  to_s.include? 'Chronicle::ETL'
end
phase() click to toggle source

Returns whether a class is an Extractor, Transformer, or Loader

# File lib/chronicle/etl/catalog.rb, line 65
def phase
  ancestors = self.ancestors
  return :extractor if ancestors.include? Chronicle::ETL::Extractor
  return :transformer if ancestors.include? Chronicle::ETL::Transformer
  return :loader if ancestors.include? Chronicle::ETL::Loader
end
provider() click to toggle source

Returns which third-party provider this connector is associated wtih

# File lib/chronicle/etl/catalog.rb, line 73
def provider
  # TODO: needs better convention for a gem reporting its provider name
  provider = to_s.split('::')[1].downcase
  provider == 'etl' ? 'chronicle' : provider
end