class GnipApi::Adapter

  1. lib/gnip_api/adapter.rb
  2. lib/gnip_api/request.rb
  3. show all
Superclass: Object

The interface to the world. Wraps the net adapter of choice and exposes common methods for the rest of the classes to use, abstracting details of the request/response. A note on the data: HTTParty was selected over Excon for the simple reason that Excon doesn't handle GZip responses. HTTParty uses net/http under the hood and handles this as expected. However, for reasons I don't know at this point, the Zlib responsible of inflating the GZip compressed data waits to fill in a buffer in order to return a decompressed chunk. You can test this on a farily busy stream and you'll see that all chunks you receive decompressed are almsot always 16384 bytes. Additionally, the rn Gnip sends as a keep alive signal seems to be buffered further. As a result I expose 2 additional methods for consuming the stream that uses Curl instead. Curl handles the GZip data better. You can launch it in a shell and the keep alive signals are properly there. These alternatives are a bit of a cheat, but allows delegation of handling the actual connection and the GZip data.

stream_get will work as usual, using HTTParty and Net::HTTP stuff.

io_curl_stream will launch a subprocess and capture output with IO.popen. This method behaves similarly to stream_get regarding the keep alive signals. However, these are returned all at once when a bigger chunk of data is sent over the stream. Still, it reads them well and can be handled.

pty_curl_stream is different. Using the PTY features, a shell is opened and the output captured is as (in my opinion) should be. All keep-alive signals are returned when received with no wait at all.

All this will remain experimental until tested on a real environment. I recomment to use stream_get which was already tested and works as expected.

Methods

Public Class

  1. new

Public Instance

  1. debug
  2. delete
  3. get
  4. io_curl_stream
  5. logger
  6. post
  7. pty_curl_stream
  8. stream_get

Constants

DELETE = 'DELETE'  
GET = 'GET'  
POST = 'POST'  

Attributes

debug [R]
logger [R]

Public Class methods

new ()
[show source]
# File lib/gnip_api/adapter.rb, line 38
def initialize
  raise GnipApi::Errors::MissingCredentials unless GnipApi.credentials?
  @logger = GnipApi.config.logger
  @debug = GnipApi.config.debug
end

Public Instance methods

delete (request)
[show source]
# File lib/gnip_api/adapter.rb, line 56
def delete request
  options = default_options(request)
  data = HTTParty.post request.uri, options
  create_response(request, data.code, data.body, data.headers)
end
get (request)
[show source]
# File lib/gnip_api/adapter.rb, line 44
def get request
  options = default_options(request)
  data = HTTParty.get request.uri, options
  create_response(request, data.code, data.body, data.headers)
end
io_curl_stream (request)
[show source]
# File lib/gnip_api/adapter.rb, line 74
def io_curl_stream request
  request.log!
  auth_header = "Authorization: Basic #{base64_auth}"
  cmd = "curl --compressed -s --header \"#{auth_header}\" \"#{request.uri}\""
  begin
    IO.popen(cmd) do |io|
      while line = io.gets.strip do
        logger.info "Keep alive received" if line == ''
        next if line == ''
        yield(line)
      end
    end
  ensure
    logger.warn "Stream closed"
  end
end
post (request)
[show source]
# File lib/gnip_api/adapter.rb, line 50
def post request
  options = default_options(request)
  data = HTTParty.post request.uri, options
  create_response(request, data.code, data.body, data.headers)
end
pty_curl_stream (request)
[show source]
# File lib/gnip_api/adapter.rb, line 91
def pty_curl_stream request
  request.log!
  auth_header = "Authorization: Basic #{base64_auth}"
  cmd = "curl --compressed -s --header \"#{auth_header}\" \"#{request.uri}\""
  begin
    PTY.spawn(cmd) do |stdout, stdin, pid|
      begin
        stdout.each do |line|
          logger.info "Keep alive received" if line.strip == ''
          next if line.strip == ''
          yield(line.strip)
        end
      rescue Errno::EIO
      end
    end
  rescue PTY::ChildExited
    logger.warn "Stream closed"
  end
end
stream_get (request)
[show source]
# File lib/gnip_api/adapter.rb, line 62
def stream_get request
  request.log!
  begin
    HTTParty.get request.uri, :headers => request.headers, :stream_body => true, :basic_auth => auth do |data|
      yield(data)
    end
  rescue Zlib::BufError => error
    logger.error "STREAM ERROR -> #{error.class} -- #{error.message}\n" + error.backtrace.join("\n")
    raise error
  end
end