class Pupa::Processor::DocumentStore::RedisStore
Stores JSON documents in Redis.
Pupa
flushes the JSON document store before scraping. If you use Redis, **DO NOT** share a Redis database with Pupa
and other applications. You can select a different database than the default ‘0` for use with Pupa
by passing an argument like `redis://localhost:6379/0`.
@note Redis support depends on the ‘redis-store` gem. You may optionally
use the `hiredis` gem to further improve performance.
Public Class Methods
@param [String] address the address (e.g. ‘redis://localhost:6379/0`)
in which to dump JSON documents
@param [Boolean] pipelined whether to enable
[pipelining](http://redis.io/topics/pipelining)
# File lib/pupa/processor/document_store/redis_store.rb, line 18 def initialize(address, pipelined: false) @pipelined = pipelined options = {marshalling: false} options.update(driver: :hiredis) if defined?(Hiredis) @redis = Redis::Store::Factory.create(address, options) end
Public Instance Methods
Deletes all keys in the database.
# File lib/pupa/processor/document_store/redis_store.rb, line 93 def clear @redis.flushdb end
Delete a key.
@param [String] name a key
# File lib/pupa/processor/document_store/redis_store.rb, line 88 def delete(name) @redis.del(name) end
Returns all keys in the database.
@return [Array<String>] all keys in the store
# File lib/pupa/processor/document_store/redis_store.rb, line 36 def entries @redis.keys('*') end
Returns whether database contains an entry for the given key.
@param [String] name a key @return [Boolean] whether the store contains an entry for the given key
# File lib/pupa/processor/document_store/redis_store.rb, line 29 def exist?(name) @redis.exists(name) end
Collects commands to run all at once.
# File lib/pupa/processor/document_store/redis_store.rb, line 98 def pipelined if @pipelined @redis.pipelined do yield end else yield end end
Returns, as JSON, the value of the given key.
@param [String] name a key @return [Hash] the value of the given key
# File lib/pupa/processor/document_store/redis_store.rb, line 44 def read(name) Oj.load(@redis.get(name)) end
Returns, as JSON, the values of the given keys.
@param [String] names keys @return [Array<Hash>] the values of the given keys
# File lib/pupa/processor/document_store/redis_store.rb, line 52 def read_multi(names) @redis.mget(*names).map{|value| Oj.load(value)} end
Writes, as JSON, the value to a key.
@param [String] name a key @param [Hash] value a value
# File lib/pupa/processor/document_store/redis_store.rb, line 60 def write(name, value) @redis.set(name, Oj.dump(value, mode: :compat, time_format: :ruby)) end
Writes, as JSON, the values to keys.
@param [Hash] pairs key-value pairs
# File lib/pupa/processor/document_store/redis_store.rb, line 76 def write_multi(pairs) args = [] pairs.each do |key,value| args << key args << Oj.dump(value, mode: :compat, time_format: :ruby) end @redis.mset(*args) end
Writes, as JSON, the value to a key, unless the key exists.
@param [String] name a key @param [Hash] value a value @return [Boolean] whether the key was set
# File lib/pupa/processor/document_store/redis_store.rb, line 69 def write_unless_exists(name, value) @redis.setnx(name, Oj.dump(value, mode: :compat, time_format: :ruby)) end