module LogLineParser
Constants
- CombinedLogParser
Parser of NCSA extended/combined log format
- CommonLogParser
Parser of Common Log Format (CLF)
ref: www.w3.org/Daemon/User/Config/Logging.html#common-logfile-format
- CommonLogWithVHParser
Parser of Common Log Format with Virtual Host
- MoeLogFormat
MoeLogFormat
andMoeLogParser
is added from the personal needs of the original author, and the log format is not a widely used one. You may remove this file if you don't need it. (MOE is the acronym of the name of the organization for which the author is working at the time of the first release of this program.)MoeLogFormat
= CombinedLogFormat + “%D”- MoeLogParser
Parser of
MoeLogFormat
- PREDEFINED_FORMATS
- VERSION
Public Class Methods
Reads each line from input
(Apache
access log files are expected) and parses it, then yields the line and the parsed result (record
) to the associated block.
When it fails to parse a line, the line will be printed to error_output
# File lib/log_line_parser.rb, line 264 def self.each_record(parser: CombinedLogParser, input: ARGF, error_output: STDERR) # :yields: line, record input.each_line do |line| begin yield line, parser.parse(line) rescue MalFormedRecordError => e error_output.print error_message(input, e) end end end
# File lib/log_line_parser.rb, line 216 def self.parse(line) stack = LogLineNodeStack.new tokens = LogLineTokenizer.tokenize(line.chomp) tokens.each {|token| stack.push token } stack # I'm not checking the reason yet, but the following way of pushing # tokens directly into the stack is very slow. # # LogLineTokenizer.tokenize(line.chomp, stack) end
Creates a parser from a LogFormat.
For example,
parser = LogLineParser.parser("%h %l %u %t \"%r\" %>s %b")
creates the parser of Common Log Format.
# File lib/log_line_parser.rb, line 204 def self.parser(log_format) if log_format.kind_of? String format_strings = Apache.parse_log_format(log_format) field_names = Apache.format_strings_to_symbols(format_strings) else format_strings = nil field_names = log_format end create_record_type(field_names, format_strings) end
Turns a line of Apache
access logs into an array of field values.
Escaped characters such as “\t” or “\”“ will be unescaped.
# File lib/log_line_parser.rb, line 232 def self.to_array(line) parse(line).to_a end
Private Class Methods
# File lib/log_line_parser.rb, line 185 def self.create_record_type(field_names, format_strings) record_type = Struct.new(*field_names) record_type.extend(ClassMethods) record_type.include(InstanceMethods) record_type.setup(field_names, format_strings) record_type end
# File lib/log_line_parser.rb, line 276 def self.error_message(input, e) if input == ARGF "#{ARGF.filename}:#{ARGF.file.lineno}:#{e.message}" else e.message end end