class DoverToCalais::ResponseData

This class is responsible for creating a response object that will be passed to {Dover}, after the data source has been analysed by OpenCalais. If the response contains valid data, the {#error} attribute will be nil. The response object will then contain the OpenCalais response as an XML string. The user can then call {#filter} to filter the response. If the response doesn’t contain valid, processed data then the {#error} won’t be nil (i.e. will be true). The {#error} attribute can then be read in order to find the cause of the error.

@!attribute [r] error

@return [String, nil] any error that occurred as a result of the OpenCalais API call,
  nil if none occurred

Attributes

entities_store[R]
error[R]
events_store[R]
freds[R]
generic_relations_store[R]

Public Class Methods

new(response_data = nil, error = nil) click to toggle source

creates a new ResponseData object, passing the name of the data source to be processed

@param response_data [ Nokogiri::XML::NodeSet, Hash, nil] the XML or JSON data returned by OpenCalais @param error [ String, nil] an error description if the OpenCalais call has failed

# File lib/dover_to_calais.rb, line 143
def initialize(response_data = nil, error = nil)
  if response_data.class.to_s == "Nokogiri::XML::Document"
    @xml_data = response_data
  elsif response_data.class.to_s == "Hash"
    @json_data = response_data
    prepare_data(response_data)


  else
    @error = error
  end

end

Public Instance Methods

filter(params) click to toggle source

Filters the xml response object to extract relevant data.

@param params [Hash] a filter Hash (see code samples) @return [Array] a list of relevant response items

# File lib/dover_to_calais.rb, line 392
def filter(params)
  unless  @xml_data
    return 'ERR: filter method only works with xml-based output!'

  end

  result = Array.new
  begin
    if @xml_data

      if params[:given]
        found = @xml_data.xpath("//#{params[:given][:entity]}[contains(text(), #{params[:given][:value].inspect})]")
        if found.size > 0
          @xml_data.xpath("//#{params[:entity]}[contains(text(), #{params[:value].inspect})]").each do |node|
            result <<  create_response_item(node)
          end
        end
      else  # no conditional
        @xml_data.xpath("//#{params[:entity]}[contains(text(), #{params[:value].inspect})]").each do |node|
          result <<  create_response_item(node)
        end
      end

      return result
    else # no xml data
      return 'ERR: no valid xml data!'

    end #if

  rescue  Exception=>e
    return "ERR: #filter:  #{e}"

  end

  return result

end
get_relation(ref_key) click to toggle source

Retrieves the relation with the specified key (OpenCalais ID). The method will also de-reference any of its attributes that refer to other entities via an OpenCalais ID and will replace the references with the appropriate Entity structure, if applicable

Only applicable with the JSON (rich) output format

@param String the OpenCalais ID @return an GenericRelation Struct if a match is found, nil otherwise

# File lib/dover_to_calais.rb, line 305
def get_relation(ref_key)
  if @generic_relations_store.key?(ref_key)

    if @generic_relations_store[ref_key]['relationsubject']
      gr_subject = @generic_relations_store[ref_key]['relationsubject'].match('^http://d.opencalais.com') ?
      get_entity(@generic_relations_store[ref_key]['relationsubject']) :
      @generic_relations_store[ref_key]['relationsubject']
    else
      gr_subject = 'N/A'
    end


    if @generic_relations_store[ref_key]['relationobject']
      gr_object = @generic_relations_store[ref_key]['relationobject'].match('^http://d.opencalais.com') ?
      get_entity(@generic_relations_store[ref_key]['relationobject']) :
      @generic_relations_store[ref_key]['relationobject']
    else
      gr_object = 'N/A'
    end

    GenericRelation.new(gr_subject,
    @generic_relations_store[ref_key]['verb'],
    gr_object,
    @generic_relations_store[ref_key]['instances'][0]['exact'] ||= 'N/A')
  else
    nil
  end
end
to_s() click to toggle source

Returns the response data as an XML string or an error, if one has occurred.

@return [String] an XML string

# File lib/dover_to_calais.rb, line 160
def to_s
  if @xml_data
    @xml_data.to_s
  elsif @json_data
    @json_data.to_s
  else
    @error
  end
end

Private Instance Methods

convert_to_hash(an_attribute) click to toggle source

Coverts an attribute to an appropriate Hash

Only applicable with the JSON (rich) output format

@param [String, DoverToCalais::ResponseData::Entity, Hash] an object @return a Hash value

# File lib/dover_to_calais.rb, line 263
def convert_to_hash(an_attribute)
  h = {}
  if an_attribute.class.to_s.eql?('String')
    h[:name] = an_attribute
  end

  if an_attribute.class.to_s.eql?('DoverToCalais::ResponseData::Entity')
    h = an_attribute.to_hash
  end

  if an_attribute.class.to_s.eql?('Hash')
    h = an_attribute
  end

  h
end
create_response_item(node) click to toggle source

Creates a Response Item from an xml node.

@param node [Nokogiri::XML::Node] an XML node @return [ResponseItem] a response item object

# File lib/dover_to_calais.rb, line 434
def create_response_item(node)
  node_relevance =  node.attribute('relevance').text.to_f if node.has_attribute?('relevance')
  node_count =  node.attribute('count').text.to_i if node.has_attribute?('count')
  node_normalized =  node.attribute('normalized').text if node.has_attribute?('normalized')
  node_importance = node.attribute('importance').text.to_i if node.has_attribute?('importance')
  node_orig_value =  node.xpath('originalValue').text if node.name.eql?('SocialTag')

  ResponseItem.new(node.name,
  node.text,
  node_relevance,
  node_count,
  node_normalized,
  node_importance,
  node_orig_value )

end
find_in_events(ref_key) click to toggle source

Selects a Hash of events, where the events’ key matches the specified OpenCalais ID.

Only applicable with the JSON (rich) output format

@param String the OpenCalais ID @return a Hash with the selected matches

# File lib/dover_to_calais.rb, line 382
def find_in_events(ref_key)
  @events_store.select{|key, hash| hash.has_value?(ref_key) }
end
find_in_relations(ref_key) click to toggle source

Selects a Hash of generic relations, where the relations’ subject or object attributes match the specified OpenCalais ID.

Only applicable with the JSON (rich) output format

@param String the OpenCalais ID @return a Hash with the selected matches

# File lib/dover_to_calais.rb, line 369
def find_in_relations(ref_key)
  @generic_relations_store.select{|key, hash| (hash["relationsubject"] == ref_key) ||
  (hash["relationobject"] == ref_key) }

end
get_entity(ref_key) click to toggle source

Retrieves the entity with the specified key (OpenCalais ID)

Only applicable with the JSON (rich) output format

@param String the OpenCalais ID @return an Entity Struct if a match is found, nil otherwise

# File lib/dover_to_calais.rb, line 288
def get_entity(ref_key)
  if @entities_store.has_key?(ref_key)
    Entity.new(@entities_store[ref_key]['_type'],  @entities_store[ref_key]['name'], ref_key)
  else
    nil
  end
end
get_event(ref_key) click to toggle source
# File lib/dover_to_calais.rb, line 334
def get_event(ref_key)

  dereferenced_events = {}

  if @events_store.key?(ref_key)

    @events_store[ref_key].each do |k, v|

      if v.class.to_s.eql?("String") && v.match('^http://d.opencalais.com')
        dereferenced_events[k] = get_entity(v).to_hash
      elsif v.class.to_s.eql?("String") && !v.match('^http://d.opencalais.com')
        h = {}
        h['name'] = v
        dereferenced_events[k] = h
      elsif v.class.to_s.eql?("Array")
        h = {}
        h['name'] = v[0]['exact']
        dereferenced_events[k] = h
      end
    end

    Event.new(dereferenced_events)
  else
    nil
  end
end
prepare_data(results_hash) click to toggle source

The method will first create three Hash instance variables, where it will store the Entities, Generic Relations and Events -respectively- from the OpenCalais response. The key on each Hash instance variable will be the OpenCalais ID and the value will be the values_hash for that ID. Secondly, the method will iterate through each Entity, find all of it’s related Relations and Events and store them -in a relational manner- in Redis, via Ohm.

Only applicable with the JSON (rich) output format

@param Hash the OpenCalais JSON response, as a Hash @return an GenericRelation Struct if a match is found, nil otherwise

# File lib/dover_to_calais.rb, line 185
def prepare_data(results_hash)

  @entities_store = {}
  @generic_relations_store = {}
  @events_store = {}
  # find all Entities in response
  @entities_store = results_hash.select{|key, hash| hash["_typeGroup"] == "entities"}
  # find all GenericRelations in response
  @generic_relations_store = results_hash.select{|key, hash| hash["_typeGroup"] == "relations" &&
  hash["_type"] == "GenericRelations"}
  # find all Events in response
  @events_store = results_hash.select{|key, hash| hash["_typeGroup"] == "relations" &&
  hash["_type"] != "GenericRelations"}

  Ohm.redis = Redic.new(REDIS)


  #for each Entity find all related Relations and Events and store them to Ohm/Redis
  @entities_store.each_pair do |k, v|

    entity_set = EntityModel.find(calais_id: k)

    if entity_set.size > 0 #entity already exists in store
      entity = entity_set.first
      k = entity.calais_id
    else #entity doesn't exist in store
      entity = EntityModel.create(:name => v['name'], :type => v['_type'], :calais_id => k)
      entity.save
    end #if


    #get all referenced relations
    find_in_relations(k).each do |obj|

      found_rel = get_relation(obj[0])
      if found_rel

        found_rel.subject = convert_to_hash(found_rel.subject)
        found_rel.object = convert_to_hash(found_rel.object)

        relation = EntityModel::RelationModel.create(:subject => found_rel.subject,
        :object => found_rel.object,
        :verb => found_rel.verb,
        :detection => found_rel.detection,
        :calais_id => obj[0])
        entity.relations.add(relation)
      end #if
    end #each
    #get all referenced events
    find_in_events(k).each do |obj|
      found_event = get_event(obj[0])
      attribs = {}
      if found_event

        found_event.each_pair do |key, val|

          key = key.to_s.slice(1, key.length-1)
          attribs[key] = val

        end #block

        event = EntityModel::EventModel.create(:calais_id => obj[0], :info_hash => attribs)
        entity.events.add(event)

      end #if

    end #each
  end #each_pair
end