module RequestLogAnalyzer::FileFormat

Public Class Methods

all_formats() click to toggle source

Returns an array of all FileFormat instances that are shipped with request-log-analyzer by default.

   # File lib/request_log_analyzer/file_format.rb
65 def self.all_formats
66   @all_formats ||= Dir[File.expand_path('file_format/*.rb', File.dirname(__FILE__))].map do |file|
67     load(File.basename(file, '.rb'))
68   end
69 end
autodetect(file, line_count = 50) click to toggle source

Autodetects the filetype of a given file.

Returns a FileFormat instance, by parsing the first couple of lines of the provided file with avery known file format and return the most promosing file format based on the parser statistics. The autodetect_score method is used to score the fitness of a format.

file

The file to detect the file format for.

line_count

The number of lines to take into consideration

   # File lib/request_log_analyzer/file_format.rb
79 def self.autodetect(file, line_count = 50)
80   parsers = all_formats.map { |f| RequestLogAnalyzer::Source::LogParser.new(f, parse_strategy: 'cautious') }
81 
82   File.open(file, 'rb') do |io|
83     while io.lineno < line_count && (line = io.gets)
84       parsers.each { |parser| parser.parse_line(line) }
85     end
86   end
87 
88   parsers.select { |p| autodetect_score(p) > 0 }.max { |a, b| autodetect_score(a) <=> autodetect_score(b) }.file_format rescue nil
89 end
autodetect_score(parser) click to toggle source

Calculates a file format auto detection score based on the parser statistics.

This method returns a score as an integer. Usually, the score will increase as more lines are parsed. Usually, a file_format with a score of zero or lower should not be considered.

parser

The parsed that was use to parse the initial lines of the log file.

    # File lib/request_log_analyzer/file_format.rb
 98 def self.autodetect_score(parser)
 99   score  = 0
100   score -= parser.file_format.line_definitions.length
101   score -= parser.warnings * 3
102   score += parser.parsed_lines * 1
103   score += parser.parsed_requests * 10
104 
105   # As Apache matches several simular formats, subtracting 1 will make a specific matcher have a higher score
106   score -= 1 if parser.file_format.class == RequestLogAnalyzer::FileFormat::Apache
107 
108   score
109 end
load(file_format, *args) click to toggle source

Loads a FileFormat::Base subclass instance. You can provide:

  • A FileFormat instance (which will return itself)

  • A FileFormat class (of which an imstance will be returned)

  • A filename (from which the FileFormat class is loaded)

  • A symbol of a built-in file format (e.g. :rails)

   # File lib/request_log_analyzer/file_format.rb
29 def self.load(file_format, *args)
30   klass = nil
31   if file_format.is_a?(RequestLogAnalyzer::FileFormat::Base)
32     # this already is a file format! return itself
33     return @current_file_format = file_format
34 
35   elsif file_format.is_a?(Class) && file_format.ancestors.include?(RequestLogAnalyzer::FileFormat::Base)
36     # a usable class is provided. Use this format class.
37     klass = file_format
38 
39   elsif file_format.is_a?(String) && File.exist?(file_format) && File.file?(file_format)
40     # load a format from a ruby file
41     require File.expand_path(file_format)
42 
43     const = RequestLogAnalyzer.to_camelcase(File.basename(file_format, '.rb'))
44     if RequestLogAnalyzer::FileFormat.const_defined?(const)
45       klass = RequestLogAnalyzer::FileFormat.const_get(const)
46     elsif Object.const_defined?(const)
47       klass = Object.const_get(const)
48     else
49       fail "Cannot load class #{const} from #{file_format}!"
50     end
51 
52   else
53     # load a provided file format
54     klass = RequestLogAnalyzer::FileFormat.const_get(RequestLogAnalyzer.to_camelcase(file_format))
55   end
56 
57   # check the returned klass to see if it can be used
58   fail "Could not load a file format from #{file_format.inspect}" if klass.nil?
59   fail "Invalid FileFormat class from #{file_format.inspect}" unless klass.is_a?(Class) && klass.ancestors.include?(RequestLogAnalyzer::FileFormat::Base)
60 
61   @current_file_format = klass.create(*args) # return an instance of the class
62 end