class NewsCrawler::Storage::RawData::RawDataEngine

Basic class for RawData engine. Subclass and implement all its method to create new RawData engine, you should keep methods’ singature unchanged

Public Class Methods

get_engines() click to toggle source

Get engine list @return [ Array ] list of url queue engines

# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 35
def self.get_engines
  @engine_list = @engine_list || []
  @engine_list.inject({}) do | memo, klass |
    memo[klass::NAME.intern] = klass
    memo
  end
end
inherited(klass) click to toggle source
# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 29
def self.inherited(klass)
  @engine_list = (@engine_list || []) + [klass]
end

Public Instance Methods

add(url, body) click to toggle source

Add entry to raw data collection @param [ String ] url @param [ String ] body

# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 46
def add(url, body)
  raise NotImplementedError
end
clear() click to toggle source
# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 61
def clear
  raise NotImplementedError
end
count() click to toggle source
# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 57
def count
  raise NotImplementedError
end
find_by_url(url) click to toggle source

Find document with correspond url @param [ String ] url @return [ String, nil ]

# File lib/news_crawler/storage/raw_data/raw_data_engine.rb, line 53
def find_by_url(url)
  raise NotImplementedError
end