module ResqueDuplicateless::Ext::Resque::DataStore::QueueAccess

This is the extension for the QueueAccess Object

Public Instance Methods

uniquely_push_to_queue(queue, item) click to toggle source

This function calls our LUA script for evaluation.

# File lib/resque-duplicateless/ext/resque/data_store.rb, line 19
def uniquely_push_to_queue(queue, item)
  out = nil
  # Ensure the script is loaded
  @redis.redis.script(:load, uniquely_push_to_queue_script) unless @redis.redis.script(:exists, uniquely_push_to_queue_script_sha)
  
  # Now pipeline the watch and addition calls
  @redis.pipelined do
    watch_queue queue
    out = @redis.evalsha uniquely_push_to_queue_script_sha, [redis_key_for_queue_also(queue)], [item]
  end
  out.value
end

Protected Instance Methods

redis_key_for_queue_also(queue) click to toggle source

TODO: Resque queue names should really be protected, not private

# File lib/resque-duplicateless/ext/resque/data_store.rb, line 35
def redis_key_for_queue_also(queue)
  "queue:#{queue}"
end
uniquely_push_to_queue_script() click to toggle source

The LUA script to uniquely add to queue

It always runs O(n)

It has two parts

  1. Use RPOPLPUSH to cycle through the entire array. It must complete the cycle, even if it finds something, so that the list's order is maintained

1A. If the list is empty, then we don't need to do this. Non-existant lists are considered empty

  1. If it does not find the item, then add it to the tail of the list

# File lib/resque-duplicateless/ext/resque/data_store.rb, line 55
          def uniquely_push_to_queue_script
<<-FOO
local list_size = redis.call('LLEN',KEYS[1])
local response = 0
if list_size > 0 then
  for i=1,list_size
  do
    redis.call('RPOPLPUSH',KEYS[1],KEYS[1])
    if ARGV[1] == redis.call('LINDEX',KEYS[1],0) then response = 1 end
  end
end
if response == 0 then redis.call('RPUSH',KEYS[1],ARGV[1]) end
return 1
FOO
          end
uniquely_push_to_queue_script_sha() click to toggle source

Create the SHA once and save it to avoid doing it repeatedly for no reason

# File lib/resque-duplicateless/ext/resque/data_store.rb, line 40
def uniquely_push_to_queue_script_sha
  @uniquely_push_to_queue_script_sha = Digest::SHA1.hexdigest uniquely_push_to_queue_script unless @uniquely_push_to_queue_script_sha
  @uniquely_push_to_queue_script_sha
end