module MapReduce::Reduceable

The MapReduce::Reduceable mixin allows to reduce an arbitrary chunk using the specified map-reduce implementation.

Private Instance Methods

reduce_chunk(chunk, implementation) { |prev_item| ... } click to toggle source

Reduces the specified chunk, e.g. some enumerable, using the specified map-reduce implementation using a lookahead of one to detect key changes. The reduce implementation is called up until a key change is detected, because the key change signals that the reduce operation is finished for the particular key, such that it will then be yielded.

@param chunk The chunk to be reduced. Can e.g. be some enumerable. @param implementation The map-reduce implementation.

# File lib/map_reduce/reduceable.rb, line 17
def reduce_chunk(chunk, implementation)
  return enum_for(:reduce_chunk, chunk, implementation) unless block_given?

  last_item = chunk.inject do |prev_item, cur_item|
    prev_key = prev_item[0]

    # Here we can compare without serializing the keys to json first,
    # because we reduce a chunk which includes a deserialization step.

    if prev_key == cur_item[0]
      [prev_key, implementation.reduce(prev_key, prev_item[1], cur_item[1])]
    else
      yield(prev_item)

      cur_item
    end
  end

  yield(last_item) if last_item
end