class IncludedInRedis
A specialized class using Redis to track items stored. It supports three operations: new, <<, and include? . Together these can be used to add items to Redis, then determine whether the item has been added.
To use it with Spider
use the check_already_seen_with method:
Spider.start_at('http://example.com/') do |s| s.check_already_seen_with IncludedInRedis.new(host: '127.0.0.1', port: 6379) end
Public Class Methods
new(*a)
click to toggle source
Construct a new IncludedInRedis
instance. All arguments here are passed to Redis (part of the redis gem).
# File lib/spider/included_in_redis.rb, line 18 def initialize(*a) @c = Redis.new(*a) end
Public Instance Methods
<<(v)
click to toggle source
Add an item to Redis
# File lib/spider/included_in_redis.rb, line 23 def <<(v) @c.set(v.to_s, v.to_json) end
include?(v)
click to toggle source
True if the item is in Redis
# File lib/spider/included_in_redis.rb, line 28 def include?(v) @c.get(v.to_s) == v.to_json end