class IncludedInRedis

A specialized class using Redis to track items stored. It supports three operations: new, <<, and include? . Together these can be used to add items to Redis, then determine whether the item has been added.

To use it with Spider use the check_already_seen_with method:

Spider.start_at('http://example.com/') do |s|
  s.check_already_seen_with IncludedInRedis.new(host: '127.0.0.1', port: 6379)
end

Public Class Methods

new(*a) click to toggle source

Construct a new IncludedInRedis instance. All arguments here are passed to Redis (part of the redis gem).

# File lib/spider/included_in_redis.rb, line 18
def initialize(*a)
  @c = Redis.new(*a)
end

Public Instance Methods

<<(v) click to toggle source

Add an item to Redis

# File lib/spider/included_in_redis.rb, line 23
def <<(v)
  @c.set(v.to_s, v.to_json)
end
include?(v) click to toggle source

True if the item is in Redis

# File lib/spider/included_in_redis.rb, line 28
def include?(v)
  @c.get(v.to_s) == v.to_json
end