class Chronicle::ETL::JobLog
A record of what happened in the running of a job. We're interested in tracking when it ran, if it was successful, and what the latest record we found is (to use as a cursor for the next time)
Attributes
finished_at[RW]
highest_timestamp[RW]
job[RW]
job_id[RW]
last_id[RW]
num_records_processed[RW]
started_at[RW]
success[RW]
Public Class Methods
Private Class Methods
build_from_serialized(attrs)
click to toggle source
Create a new JobLog
and set its instance variables from a serialized hash
# File lib/chronicle/etl/job_log.rb, line 74 def self.build_from_serialized attrs attrs.delete(:id) new do |job_log| attrs.each do |key, value| setter = "#{key.to_s}=".to_sym job_log.send(setter, value) end end end
Public Instance Methods
finish()
click to toggle source
Indicate that a job has finished
# File lib/chronicle/etl/job_log.rb, line 48 def finish @finished_at = Time.now @success = true end
job=(job)
click to toggle source
# File lib/chronicle/etl/job_log.rb, line 53 def job= job @job = job @job_id = job.id end
log_transformation(transformer)
click to toggle source
Log the result of a single transformation in a job @param transformer [Chronicle::ETL::Tranformer] The transformer that ran
# File lib/chronicle/etl/job_log.rb, line 31 def log_transformation(transformer) @last_id = transformer.id if transformer.id # Save the highest timestamp that we've encountered so far @highest_timestamp = [transformer.timestamp, @highest_timestamp].compact.max if transformer.timestamp # TODO: a transformer might yield nil. We might also want certain transformers to explode # records into multiple new ones. Therefore, this this variable will need more subtle behaviour @num_records_processed += 1 end
serialize()
click to toggle source
Take a JobLog's instance variables and turn them into a hash representation
# File lib/chronicle/etl/job_log.rb, line 59 def serialize { job_id: @job_id, last_id: @last_id, highest_timestamp: @highest_timestamp, num_records_processed: @num_records_processed, started_at: @started_at, finished_at: @finished_at, success: @success } end
start()
click to toggle source
Indicate that a job has started
# File lib/chronicle/etl/job_log.rb, line 43 def start @started_at = Time.now end