class Fluent::AddInsertIdsFilter

Fluentd filter plugin for adding insertIds to guarantee log entry order and uniqueness. Sample log entries enriched by this plugin: {

"timestamp": "2017-08-22 13:35:28",
"message": "1",
"logging.googleapis.com/insertId": "aye7eakuf23h41aef0"

} {

"timestamp": "2017-08-22 13:35:28",
"message": "2",
"logging.googleapis.com/insertId": "aye7eakuf23h41aef1"

} {

"timestamp": "2017-08-22 13:35:28",
"message": "3",
"logging.googleapis.com/insertId": "aye7eakuf23h41aef2"

}

Public Instance Methods

filter(tag, time, record) click to toggle source

rubocop:disable Lint/UnusedMethodArgument

# File lib/fluent/plugin/filter_add_insert_ids.rb, line 66
def filter(tag, time, record)
  # Only generate and add an insertId field if the record is a hash and
  # the insert ID field is not already set (or set to an empty string).
  record[@insert_id_key] = increment_insert_id if record.is_a?(Hash) && record[@insert_id_key].to_s.empty?
  record
end
start() click to toggle source
Calls superclass method
# File lib/fluent/plugin/filter_add_insert_ids.rb, line 54
def start
  super
  @log = $log # rubocop:disable Style/GlobalVars

  # Initialize the insertID.
  @log.info "Started the add_insert_ids plugin with #{@insert_id_key}" \
            ' as the insert ID key.'
  @insert_id = generate_initial_insert_id
  @log.info "Initialized the insert ID key to #{@insert_id}."
end

Private Instance Methods

generate_initial_insert_id() click to toggle source

Generate a random string as the initial insertId.

# File lib/fluent/plugin/filter_add_insert_ids.rb, line 77
def generate_initial_insert_id
  Array.new(INSERT_ID_SIZE) { ALLOWED_CHARS.sample }.join
end
increment_insert_id() click to toggle source

Increment the insertId and return the new value.

# File lib/fluent/plugin/filter_add_insert_ids.rb, line 82
def increment_insert_id
  @insert_id = @insert_id.next
end