module LogLineParser

Constants

CombinedLogParser

Parser of NCSA extended/combined log format

CommonLogParser

Parser of Common Log Format (CLF)

ref: www.w3.org/Daemon/User/Config/Logging.html#common-logfile-format

CommonLogWithVHParser

Parser of Common Log Format with Virtual Host

MoeLogFormat

MoeLogFormat and MoeLogParser is added from the personal needs of the original author, and the log format is not a widely used one. You may remove this file if you don't need it. (MOE is the acronym of the name of the organization for which the author is working at the time of the first release of this program.)

MoeLogFormat = CombinedLogFormat + “%D”

MoeLogParser

Parser of MoeLogFormat

PREDEFINED_FORMATS
VERSION

Public Class Methods

each_record(parser: CombinedLogParser, input: ARGF, error_output: STDERR) { |line, record| ... } click to toggle source

Reads each line from input (Apache access log files are expected) and parses it, then yields the line and the parsed result (record) to the associated block.

When it fails to parse a line, the line will be printed to error_output

# File lib/log_line_parser.rb, line 264
def self.each_record(parser: CombinedLogParser,
                     input: ARGF,
                     error_output: STDERR) # :yields: line, record
  input.each_line do |line|
    begin
      yield line, parser.parse(line)
    rescue MalFormedRecordError => e
      error_output.print error_message(input, e)
    end
  end
end
parse(line) click to toggle source
# File lib/log_line_parser.rb, line 216
def self.parse(line)
  stack = LogLineNodeStack.new
  tokens = LogLineTokenizer.tokenize(line.chomp)
  tokens.each {|token| stack.push token }
  stack
  # I'm not checking the reason yet, but the following way of pushing
  # tokens directly into the stack is very slow.
  #
  # LogLineTokenizer.tokenize(line.chomp, stack)
end
parser(log_format) click to toggle source

Creates a parser from a LogFormat.

For example,

parser = LogLineParser.parser("%h %l %u %t \"%r\" %>s %b")

creates the parser of Common Log Format.

# File lib/log_line_parser.rb, line 204
def self.parser(log_format)
  if log_format.kind_of? String
    format_strings = Apache.parse_log_format(log_format)
    field_names = Apache.format_strings_to_symbols(format_strings)
  else
    format_strings = nil
    field_names = log_format
  end

  create_record_type(field_names, format_strings)
end
to_array(line) click to toggle source

Turns a line of Apache access logs into an array of field values.

Escaped characters such as “\t” or “\”“ will be unescaped.

# File lib/log_line_parser.rb, line 232
def self.to_array(line)
  parse(line).to_a
end

Private Class Methods

create_record_type(field_names, format_strings) click to toggle source
# File lib/log_line_parser.rb, line 185
def self.create_record_type(field_names, format_strings)
  record_type = Struct.new(*field_names)
  record_type.extend(ClassMethods)
  record_type.include(InstanceMethods)
  record_type.setup(field_names, format_strings)
  record_type
end
error_message(input, e) click to toggle source
# File lib/log_line_parser.rb, line 276
def self.error_message(input, e)
  if input == ARGF
    "#{ARGF.filename}:#{ARGF.file.lineno}:#{e.message}"
  else
    e.message
  end
end