class DaimonSkycrawlers::Filter::Base

Base class of filters.

You must implement `#call` in your filter and it must return true or false. If your filter returns true, processors can process given URL after your filter. Otherwise framework skips given URL to skip processors.

Public Class Methods

new(storage: nil) click to toggle source
Calls superclass method DaimonSkycrawlers::ConfigMixin::new
# File lib/daimon_skycrawlers/filter/base.rb, line 18
def initialize(storage: nil)
  super()
  @storage = storage
end

Public Instance Methods

call(message) click to toggle source

Filter message

Override this method in subclass.

@param message [Hash] message can include anything

@return [true|false] process the message if true otherwise skip message.

# File lib/daimon_skycrawlers/filter/base.rb, line 39
def call(message)
  raise NotImplementedError, "Must implement this method in subclass"
end
storage() click to toggle source

Retrieve storage instance

# File lib/daimon_skycrawlers/filter/base.rb, line 26
def storage
  @storage ||= DaimonSkycrawlers::Storage::RDB.new
end

Private Instance Methods

normalize_url(url) click to toggle source
# File lib/daimon_skycrawlers/filter/base.rb, line 45
def normalize_url(url)
  return url unless @base_url
  (URI(@base_url) + url).to_s
end