GoGetter

Easily send get requests, with a little more sophistication than Net::HTTP.get

Unlike HTTParty, which is a great gem if your class works against a specific website, GoGetter fits the use case where you need to send a bunch of HTTP GETs to several domains, and don’t want to wrap each one in its own class.

Think of it as an alternative to open-uri that doesn’t create any temporary files.

It handles proxies, basic authentication, and HTTP redirects.

Before releasing this code I used it extensively in a proprietary web crawler that sent around a billion GET requests so far, so you could say that it’s quite robust :)

Contributing to gogetter

necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

Copyright © 2011 Elad Kehat. See LICENSE.txt for further details.