module DaimonSkycrawlers::Processor
Name space for processors
Public Class Methods
config()
click to toggle source
Shortcut of DaimonSkycrawlers.configuration
@return [DaimonSkycrawlers::Configuration]
# File lib/daimon_skycrawlers/processor.rb, line 41 def config DaimonSkycrawlers.configuration end
default_process_name()
click to toggle source
enqueue_http_response(url, message = {})
click to toggle source
Enqueue a URL to processor queue
@param url [String] Specify absolute URL @param message [Hash] Extra parameters for crawler @return [void]
# File lib/daimon_skycrawlers/processor.rb, line 30 def enqueue_http_response(url, message = {}) message[:url] = url config.logger.debug("#{queue_name}: #{url}") SongkickQueue.publish(queue_name, message) end
queue_name()
click to toggle source
run(process_name: default_process_name)
click to toggle source
Run registered processors
@param process_name [String] Process name
# File lib/daimon_skycrawlers/processor.rb, line 17 def run(process_name: default_process_name) if config.shutdown_interval > 0 DaimonSkycrawlers::Timer.setup_shutdown_timer(config.queue_name_prefix, interval: config.shutdown_interval) end SongkickQueue::Worker.new(process_name, [DaimonSkycrawlers::Consumer::HTTPResponse]).run end