class DaimonSkycrawlers::Storage::RDB
Storage
for RDBMS
Public Class Methods
new(config_path = "config/database.yml")
click to toggle source
Calls superclass method
DaimonSkycrawlers::ConfigMixin::new
# File lib/daimon_skycrawlers/storage/rdb.rb, line 10 def initialize(config_path = "config/database.yml") super() Base.configurations = YAML.load(ERB.new(::File.read(config_path)).result) Base.establish_connection(DaimonSkycrawlers.env.to_sym) end
Public Instance Methods
read(message = {})
click to toggle source
Fetch page identified by url
@param message [Hash] this hash can include `:url`, `:key` to find page
# File lib/daimon_skycrawlers/storage/rdb.rb, line 44 def read(message = {}) url = message[:url] key = message[:key] if key Page.where(key: key).order(updated_at: :desc).limit(1).first else Page.where(url: url).order(updated_at: :desc).limit(1).first end end
save(data)
click to toggle source
Save data to RDB
@param data [Hash] data has following keys
* `:url`: URL * `:message`: Given message * `:response`: HTTP response
# File lib/daimon_skycrawlers/storage/rdb.rb, line 24 def save(data) url = data[:url] message = data[:message] key = message[:key] || url response = data[:response] headers = response.headers body = response.body Page.create(url: url, key: key, headers: JSON.generate(headers), body: body, last_modified_at: headers["last-modified"], etag: headers["etag"]) end