class PrettyProxy
The PrettyProxy
class aggregate and validate the configuration of a proxy based in simple pretty url oriented rewriting rules. It’s too a rack app, and offers a abstract method for rewrite the responses returned by the proxy. The (X)HTML responses are rewritten to make the hyperlinks point to the proxy version of the page if it exist.
@example A terrible example
# You can run this example with 'rake heresy_example' in the gem folder # and see the result in localhost:9292/proxy/ require 'pretty_proxy' class Heresy < PrettyProxy def sugared_rewrite_response(triplet, requested_to_proxy_env, rewritten_env) status, headers, page = triplet page = page.gsub(/(MTG )?Magic(: The Gathering)?/, 'Yu-Gi-Oh') [status, headers, page] end end run Heresy.new('/proxy/', 'http://magiccards.info', '/')
If you want to make a Rack app who use the proxy to point to another path of the same app you have to use a server in multithread mode, otherwise requests to the proxy will end in a deadlock. The proxy request the original page but the server don’t respond because is waiting the proxy request to be resolved. The proxy request don’t end because need the original page. A timeout error occur.
What this class can’t do but maybe will do in the future: smart handling of 3xx status response and chunked encoding (the chunks are concatened in the proxy and the transfer-encoding header removed); support more than deflate and gzip; exception classes with more than a message;
The exception classes (except Error
) inherit Error
, and Error
inherit ArgumentError. They are empty yet, only have a message.
Glossary:
‘a valid proxy url/path’: The path (or the path of the url) start with the proxy_path and is followed by a original_path.
‘in(side)/out(side) the proxy control’: The url have (or not) the path starting with a original_path, and the scheme, port and host are the same of the original_domain.
CHANGELOG:
4.0.0 * proxify_hyperlink don't take relative paths or urls anymore, only absolute urls. This is because the proxy url was used for a double purpose (know the proxy scheme+host+port and resolve relative hyperlinks). This can lead to the mistake of believing that the base url to resolve relative links in the page is the page url (what's false if the page has a base tag). See more in: http://www.w3.org/TR/html5/infrastructure.html#base-urls * proxify_html (and other methods who use it, as #call) use the base tag from the page to determine the base url, and add the the base tag (if the page don't have one) to simplify the assets proxification. All a[href] are changed to absolute urls. * rspec-html-matchers added as development dependency 3.0.0 * return a String for unproxify_url (and not more a URI) because this is a change in the API (and can break code) the major version is now 3, if you don't use this method you can safely upgrade * depends in addressable gem * handles correctly the URIs without scheme (but with host) like '//duckduckgo.com/' (spec added for that)
@author Henrique Becker
Attributes
Public Class Methods
Create a new PrettyProxy
instance or raise a ConfigError
. Clone the arguments. @param proxy_path [String] Start and end with slashes, represent the
path in the proxy site who map to the proxy app (and, in consequence, to another path in the same or another site).
@param original_domain [String, URI] A URL without path (no trailing slash),
query or fragment (can have scheme (http[s]), domain and port), the site to where the proxy map.
@param original_paths [String, each] The path (or the paths) to be mapped
right inside the proxy_path (has to begin with slash).
@param ignore_html_errors
[TrueClass, FalseClass] If the argument of this
parameter is true the #proxify_html try to ignore some exceptions that can be caused by an malformed (X)HTML and continue. Don't silence #{sugared_,}rewrite_response. Experimental.
@note See the specs {file:../spec/pretty_proxy_spec.rb} for examples and
complete definition of invalid args.
@return [PrettyProxy] a new instance @raise PrettyProxy::ConfigError
# File lib/pretty_proxy.rb, line 108 def initialize(proxy_path, original_domain, original_paths, ignore_html_errors = false) Utils.validate_proxy_path(proxy_path) Utils.validate_original_domain_and_paths(original_domain, original_paths) @ignore_html_errors = ignore_html_errors @proxy_path = proxy_path.clone @original_domain = Addressable::URI.parse(original_domain.clone) @original_paths = Set.new if original_paths.respond_to? :each original_paths.each { | value | @original_paths << value.clone } else @original_paths << original_paths.clone end end
Public Instance Methods
Make this class a Rack app. It’s overriden to repass to the rewrite_response
the original Rack environment (request to the proxy) and the rewritten env (modified to point the original page request). If you don’t know the parameters and return of this method, please read {rack.rubyforge.org/doc/SPEC.html}.
# File lib/pretty_proxy.rb, line 392 def call(env) # in theory we only need to repass the rewritten_env, any original env info # needed can be passed as a environment application variable # example: (env['app_name.original_path'] = env['PATH_INFO']) # but to avoid this to be a common idiom we repass the original env too rewritten_env = rewrite_env(env) rewrite_response(perform_request(rewritten_env), env, rewritten_env) end
Check if the URI::HTTP(S) is a page who can be accessed through the proxy.
# File lib/pretty_proxy.rb, line 408 def inside_proxy_control?(uri) same_domain_as_original?(uri) && valid_path_for_proxy?(@proxy_path + uri.path[1..-1]) end
# File lib/pretty_proxy.rb, line 139 def original_domain=(original_domain) Utils.validate_original_domain_and_paths(original_domain, @original_paths) @original_domain = original_domain end
# File lib/pretty_proxy.rb, line 144 def original_paths=(original_paths) Utils.validate_original_domain_and_paths(@original_domain, original_paths) @original_paths = original_paths end
Take a url and the proxy domain (scheme, host and port) and return if the url point to a valid proxy page.
# File lib/pretty_proxy.rb, line 433 def point_to_a_proxy_page?(hyperlink, proxy_domain) Utils.same_domain?(hyperlink, proxy_domain) && valid_path_for_proxy?(hyperlink.path) end
Take a (X)HTML Document add a base tag (if none) and apply proxify_hyperlink
to the ‘href’ attribute of each ‘a’ element. If the page has a base tag leave it unchanged. If a valid mime_type is passed as argument, but the html argument can’t be parsed by this mime-type it simple returns the first argument unchanged. @param html [String] A (X)HTML document. @param proxy_url [String, URI::HTTP, URI::HTTPS] The url where the
the proxified version of the page will be displayed.
@param mime_type [String] A string containing ‘text/html’ or
'application/xhtml+xml' (insensitive to case and any characters before or after the type). Define if the content will be parsed as xml or html. See this link for more info: http://www.w3.org/TR/xhtml-media-types/. Raise an exception if an invalid value is provided.
@return [String] A copy of the document with the changes applied,
or the original string, if the document can't be parsed.
@raise PrettyProxy::ProxyError
# File lib/pretty_proxy.rb, line 211 def proxify_html(html, proxy_url, mime_type) parsed_html = Utils.parse_html_or_xhtml(html, mime_type) if parsed_html.nil? return html end # This isn't in conformance with the following document # http://www.w3.org/TR/html5/infrastructure.html#base-urls # but support to frames is not a priority document_original_url = unproxify_url(proxy_url) # in theory base must have a href... but to avoid an exception by bad html base_tag = parsed_html.at_css('base[href]') base_url = nil if base_tag base_url = Addressable::URI.parse(document_original_url) .join(base_tag['href']).to_s else base_url = document_original_url end # the href isn't a obrigatory attribute of an anchor element parsed_html.css('a[href]').each do | hyperlink | begin absolute_hyperlink = Addressable::URI.parse(base_url) .join(hyperlink['href']).to_s hyperlink['href'] = proxify_hyperlink(absolute_hyperlink, proxy_url) rescue => e # Here we catch any exception derived from StandardError and do nothing # with it. This is a little risky, but the link in the href can be # wrong in many ways and yet be accepted by nokogiri. So to not # complexify the code we simply ignore when we can't proxify a link. raise e unless @ignore_html_errors end end unless base_tag is_XML = %r{application/xhtml\+xml}.match(mime_type) base_tag = "<base href='#{document_original_url}' #{is_XML ? '/' : ''}>" parsed_html.at_css('head').first_element_child .add_previous_sibling(base_tag) end parsed_html.to_s end
Take a absolute URL and the scheme://host of the proxy page (can have path/?query#fragment, but they are ignored) and return the rewritten hyperlink. The url only is rewritten to point the proxyfied version if it’s under proxy control. If the url is under proxy control, but it’s also a url to the proxy, the url isn’t changed (to not double proxyfy, /proxy/ ~> /proxy/proxy/). @param hyperlink [String, URI::HTTP, URI::HTTPS] A string or URI object
with a absolute url.
@param proxy_site [String, URI::HTTP, URI::HTTPS] A URL with
scheme://host[:port] to use in the hyperlink proxification.
@return [String] A absolute URL. @raise PrettyProxy::ProxyError
# File lib/pretty_proxy.rb, line 182 def proxify_hyperlink(hyperlink, proxy_site) hyperlink = Addressable::URI.parse(hyperlink) proxy_site = Addressable::URI.parse(proxy_site) if inside_proxy_control?(hyperlink) && ! point_to_a_proxy_page?(hyperlink, proxy_site) proxify_uri(hyperlink, proxy_site).to_s else hyperlink.to_s end end
# File lib/pretty_proxy.rb, line 134 def proxy_path=(proxy_path) Utils.validate_proxy_path(proxy_path) @proxy_path = proxy_path end
Modify a Rack environment hash of a request to the proxy version of a page to a request to the original page. As in Rack::proxy is used by call
for require the original page before call rewrite_response
in the response. If you want to use your own rewrite rules maybe is more wise to subclass Rack::Proxy instead subclass this class. The purpose of this class is mainly implement and enforce these rules for you. @param env [Hash{String => String}] A Rack environment hash.
(see: {http://rack.rubyforge.org/doc/SPEC.html})
@return [Hash{String => String}] A unproxified copy of the argument. @raise PrettyProxy::ProxyError
# File lib/pretty_proxy.rb, line 267 def rewrite_env(env) env = env.clone url_requested_to_proxy = Rack::Request.new(env).url # Using URI, and not Addressable::URI because the port value is incorrect in the last unproxified_url = Addressable::URI.parse(unproxify_url(url_requested_to_proxy)) if env['HTTP_HOST'] env['HTTP_HOST'] = unproxified_url.host end env['SERVER_NAME'] = unproxified_url.host env['SERVER_PORT'] = unproxified_url.inferred_port.to_s if env['SCRIPT_NAME'].empty? && !env['PATH_INFO'].empty? env['PATH_INFO'] = unproxified_url.path end if !env['SCRIPT_NAME'].empty? && env['PATH_INFO'].empty? env['SCRIPT_NAME'] = unproxified_url.path end # Seriously, i don't know how to split again the unproxified url, so PATH_INFO gonna have the full path if (!env['SCRIPT_NAME'].empty? && !env['PATH_INFO'].empty?) || (env['SCRIPT_NAME'].empty? && env['PATH_INFO'].empty?) env['PATH_INFO'] = unproxified_url.path env['SCRIPT_NAME'] = '' end env['REQUEST_PATH'] = unproxified_url.path env['REQUEST_URI'] = unproxified_url.path env end
Mainly apply the proxify_html
to the body of the response if it is a html. Raise an error if the ‘content-encoding’ is other than deflate, gzip or identity. Change the ‘content-length’ header for the new body bytesize. Remove the ‘transfer-encoding’ if it is chunked, and act as not chunked. This method is inherited of Rack::Proxy, but in the original it have only the first parameter (the triplet). This version have the Rack env requested to the proxy and the rewritten Rack env as second and third parameters, respectively. @param triplet [Array<(Integer, Hash{String => String}, each)>] A Rack
response (see {http://rack.rubyforge.org/doc/SPEC.html}) for the request to the original site.
@param requested_to_proxy_env [Hash{String => String}] A Rack environment
hash. The requested to the proxy version.
@param rewritten_env [Hash{String => String}] A Rack environment hash.
The rewritten by the proxy to point to the original version.
@return [Array<(Integer, Hash{String => String}, each)>] A unproxified
copy of the first argument.
@raise PrettyProxy::ProxyError
# File lib/pretty_proxy.rb, line 316 def rewrite_response(triplet, requested_to_proxy_env, rewritten_env) status, headers, body = triplet content_type = headers['content-type'] return triplet unless 200 == status && (%r{text/html} =~ content_type || %r{application/xhtml\+xml} =~ content_type) # the #each method of body can't be called twice, but we need to call it here and it is called # after this method return, so we fake the body with a array of one string # we can't return a string (even it responds to #each) see: http://rack.rubyforge.org/doc/SPEC.html (section 'The Body') page = '' body.each do | chunk | page << chunk end case headers['content-encoding'] when 'gzip' then page = Zlib::GzipReader.new(StringIO.new(page)).read when 'deflate' then page = Zlib::Inflate.inflate(page) when 'identity' then page = page when nil then page = page else fail ProxyError, 'unknown content-encoding, only encodings known are gzip, deflate and identity' end request_to_proxy = Rack::Request.new(requested_to_proxy_env) page = proxify_html(page, request_to_proxy.url, content_type) status, headers, page = sugared_rewrite_response([status, headers, page], requested_to_proxy_env, rewritten_env) case headers['content-encoding'] when 'gzip' page_ = '' gzip_stream = Zlib::GzipWriter.new(StringIO.new(page_)) gzip_stream.write page gzip_stream.close page = page_ when 'deflate' then page = Zlib::Deflate.deflate(page) end headers['content-length'] = page.bytesize.to_s if headers['content-length'] # TODO: find a way to make the code work with chunked encoding if 'chunked' == headers['transfer-encoding'] headers.delete('transfer-encoding') headers['content-length'] = page.bytesize.to_s end [status, headers, [page]] end
Check if the scheme, host, and port of the argument are equal to the original_domain ones.
# File lib/pretty_proxy.rb, line 403 def same_domain_as_original?(uri) Utils.same_domain?(@original_domain, uri) end
The simplest way to make use of this class is subclass this class and redefine this method. @abstract This method is called only over (X)HTML responses, after they are
decompressed and the hyperlinks proxified, before they are compressed again and the new content-length calculated.
@note The body of the triplet is a String and not a object who respond to each,
the same has to be true in the return. Return a modified clone of the response, don't change the argument.
@param triplet [Array<(Integer, Hash{String => String}, String)>] Not a
valid Rack response, the third element is a string with the response body.
@param requested_to_proxy_env [Hash{String => String}] A Rack environment
hash. The requested to the proxy version.
@param rewritten_env [Hash{String => String}] A Rack environment hash.
The rewritten by the proxy to point to the original version.
@return [Array<(Integer, Hash{String => String}, String)>] A unproxified
copy of the first argument.
# File lib/pretty_proxy.rb, line 383 def sugared_rewrite_response(triplet, requested_to_proxy_env, rewritten_env) triplet end
Take a proxy url and return the original URL behind the proxy. Preserve the query and fragment, if any. For the rewrite of a request @see rewrite_env. @param url [String, URI::HTTP, URI::HTTPS] A URL. @return [String] The unproxified URI in a string. @raise PrettyProxy::ProxyError
# File lib/pretty_proxy.rb, line 154 def unproxify_url(url) url = Addressable::URI.parse(url.clone) unless valid_path_for_proxy? url.path fail ProxyError, "'#{url.to_s}' isn't inside the proxy control, it can't be unproxified" end url.site = @original_domain.site url.path = url.path.slice((@proxy_path.size-1)..-1) url.to_s rescue Addressable::URI::InvalidURIError raise ArgumentError, "the url argument isn't a valid uri" end
Check if the absolute path begin with a proxy_path and is followed by a original_paths element.
# File lib/pretty_proxy.rb, line 415 def valid_path_for_proxy?(absolute_path) return false unless absolute_path.start_with?(@proxy_path) path_without_proxy_prefix = absolute_path[(@proxy_path.size-1)..-1] @original_paths.any? do | original_path | # if we don't test this '/about' and '/about_us' will match if original_path.end_with? '/' path_without_proxy_prefix.start_with? original_path else path_without_proxy_prefix == original_path || path_without_proxy_prefix.start_with?("#{original_path}/") end end end
Private Instance Methods
api private Don’t use this method.
# File lib/pretty_proxy.rb, line 439 def proxify_uri(absolute_uri, proxy_site) uri = absolute_uri.clone uri.site = proxy_site.site uri.path = @proxy_path[0..-2] + uri.path uri end