module NewsCrawler::Storage::RawData
store raw data from website
Public Class Methods
add(url, body)
click to toggle source
Add entry to raw data collection @param [ String ] url @param [ String ] body
# File lib/news_crawler/storage/raw_data.rb, line 52 def add(url, body) @engine.add(url, body) end
clear()
click to toggle source
# File lib/news_crawler/storage/raw_data.rb, line 67 def clear @engine.clear end
count()
click to toggle source
# File lib/news_crawler/storage/raw_data.rb, line 63 def count @engine.count end
find_by_url(url)
click to toggle source
Find document with correspond url @param [ String ] url @return [ String, nil ]
# File lib/news_crawler/storage/raw_data.rb, line 59 def find_by_url(url) @engine.find_by_url url end
set_engine(engine, *opts)
click to toggle source
Set RawData
storage engine @param [ Symbol, Object
] engine specify database engine, pass an object for custom engine @param [ Hash ] opts options pass to engine
This can be * `:mongo`, `:mongodb` for MongoDB backend
# File lib/news_crawler/storage/raw_data.rb, line 37 def set_engine(engine, *opts) if engine.respond_to? :intern engine = engine.intern end engine_class = RawDataEngine.get_engines[engine] if engine_class @engine = engine_class.new(*opts) else @engine = engine end end