class Mechanize::HTTP::Agent
This monkey patch attempts to fix the insidious “too many connection resets” bug described here: github.com/sparklemotion/mechanize/issues/123
The code is taken and modified from this helpful blog article: scottwb.com/blog/2013/11/09/defeating-the-infamous-mechanize-too-many-connection-resets-bug/
Constants
- IDEMPOTENT_HTTP_METHODS
- MAX_CONNECTION_RESET_RETRIES
Public Instance Methods
fetch(uri, http_method = :get, headers = {}, params = [], referer = current_page, redirects = 0)
Also aliased as: fetch_without_retry
Alias for: fetch_with_retry
fetch_with_retry(uri, http_method = :get, headers = {}, params = [], referer = current_page, redirects = 0)
click to toggle source
Replacement for +Mechanize::HTTP::Agent#fetch+. When a “too many connection resets” error is encountered, this method retries the request (upto {MAX_CONNECTION_RESET_RETRIES} times).
# File lib/grubby/mechanize/fetch_with_retry.rb, line 14 def fetch_with_retry(uri, http_method = :get, headers = {}, params = [], referer = current_page, redirects = 0) retry_count = 0 begin fetch_without_retry(uri, http_method, headers, params, referer, redirects) rescue Net::HTTP::Persistent::Error => e # raise if different type of error raise unless e.message.include?("too many connection resets") # raise if non-idempotent http method raise unless IDEMPOTENT_HTTP_METHODS.include?(http_method) # raise if we've tried too many times raise if retry_count >= MAX_CONNECTION_RESET_RETRIES # otherwise, shutdown the persistent HTTP connection and try again retry_count += 1 $log.warn("#{e.message} (#{e.class}). Retry in #{retry_count} seconds.") sleep(retry_count) # incremental backoff to allow server to self-correct $log.warn("Retry #{http_method.to_s.upcase} #{uri}") retry end end
Also aliased as: fetch
fetch_without_retry(uri, http_method = :get, headers = {}, params = [], referer = current_page, redirects = 0)
Alias for: fetch