class RequestLogAnalyzer::Aggregator::DatabaseInserter

The database aggregator will create an SQLite3 database with all parsed request information.

The prepare method will create a database schema according to the file format definitions. It will also create ActiveRecord::Base subclasses to interact with the created tables. Then, the aggregate method will be called for every parsed request. The information of these requests is inserted into the tables using the ActiveRecord classes.

A requests table will be created, in which a record is inserted for every parsed request. For every line type, a separate table will be created with a request_id field to point to the request record, and a field for every parsed value. Finally, a warnings table will be created to log all parse warnings.

Attributes

database[R]
request_count[R]
sources[R]

Public Instance Methods

aggregate(request) click to toggle source

Aggregates a request into the database This will create a record in the requests table and create a record for every line that has been parsed, in which the captured values will be stored.

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
32 def aggregate(request)
33   @request_object = RequestLogAnalyzer::Database::Request.new(first_lineno: request.first_lineno, last_lineno: request.last_lineno)
34   request.lines.each do |line|
35     class_columns = database.get_class(line[:line_type]).column_names.reject { |column| %w(id source_id request_id).include?(column) }
36     attributes = Hash[*line.select { |(key, _)| class_columns.include?(key.to_s) }.flatten]
37 
38     # Fix encoding patch for 1.9.2
39     attributes.each do |k, v|
40       attributes[k] = v.force_encoding('UTF-8') if v.is_a?(String)
41     end
42 
43     @request_object.send("#{line[:line_type]}_lines").build(attributes)
44   end
45   @request_object.save!
46 rescue SQLite3::SQLException => e
47   raise Interrupt, e.message
48 end
finalize() click to toggle source

Finalizes the aggregator by closing the connection to the database

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
51 def finalize
52   @request_count = RequestLogAnalyzer::Database::Request.count
53   database.disconnect
54   database.remove_orm_classes!
55 end
prepare() click to toggle source

Establishes a connection to the database and creates the necessary database schema for the current file format

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
18 def prepare
19   require 'request_log_analyzer/database'
20 
21   @sources = {}
22   @database = RequestLogAnalyzer::Database.new(options[:database])
23   @database.file_format = source.file_format
24 
25   database.drop_database_schema! if options[:reset_database]
26   database.create_database_schema!
27 end
report(output) click to toggle source

Prints a short report of what has been inserted into the database

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
75 def report(output)
76   output.title('Request database created')
77 
78   output <<  "A database file has been created with all parsed request information.\n"
79   output <<  "#{@request_count} requests have been added to the database.\n"
80   output << "\n"
81   output <<  "To open a Ruby console to inspect the database, run the following command.\n"
82   output <<  output.colorize("  $ request-log-analyzer console -d #{options[:database]}\n", :bold)
83   output << "\n"
84 end
source_change(change, filename) click to toggle source

Records source changes in the sources table

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
63 def source_change(change, filename)
64   if File.exist?(filename)
65     case change
66     when :started
67       @sources[filename] = RequestLogAnalyzer::Database::Source.create!(filename: filename)
68     when :finished
69       @sources[filename].update_attributes!(filesize: File.size(filename), mtime: File.mtime(filename))
70     end
71   end
72 end
warning(type, message, lineno) click to toggle source

Records w warining in the warnings table.

   # File lib/request_log_analyzer/aggregator/database_inserter.rb
58 def warning(type, message, lineno)
59   RequestLogAnalyzer::Database::Warning.create!(warning_type: type.to_s, message: message, lineno: lineno)
60 end