class GScraper::Search::AJAXQuery
Represents a Query
through the Google AJAX search API.
Constants
- DEFAULT_KEY
Default key
- DEFAULT_SIG
Default signature
- DEFAULT_VERSION
Default version
- PATH
AJAX API Path
- QUERY
AJAX API
Query
string- RESULTS_PER_PAGE
Maximum results per-page
Attributes
The search key
The search signature
The API version
Public Class Methods
Creates a new AJAX query from the specified URL.
@param [URI::HTTP, String] url
The URL to create the query from.
@param [Hash] options
Additional query options.
@yield [query]
If a block is given, it will be passed the new AJAX query.
@yieldparam [AJAXQuery] query
The new AJAX query.
@return [AJAXQuery]
The new AJAX query.
@see AJAXQuery.new
# File lib/gscraper/search/ajax_query.rb, line 124 def AJAXQuery.from_url(url,options={},&block) url = URI(url.to_s) options[:language] = url.query_params['hl'] options[:query] = url.query_params['q'] options[:sig] = url.query_params['sig'] options[:key] = url.query_params['key'] options[:version] = url.query_params['v'] return AJAXQuery.new(options,&block) end
Creates a new AJAX query.
@param [Hash] options
Query options.
@option options [String] :search_host (www.google.com)
The host to submit queries to.
@option options [String, Symbol] :language (Languages.native
)
The search language.
@option options [String] :sig (‘582c1116317355adf613a6a843f19ece’)
The search signature.
@option options [String, Symbol] :key (‘notsupplied’)
The search key.
@option options [Float] :version (1.0)
The desired API version.
@yield [query]
If a block is given, the new AJAX query will be passed to it.
@yieldparam [AJAXQuery] query
The new AJAX query.
# File lib/gscraper/search/ajax_query.rb, line 94 def initialize(options={},&block) @agent = GScraper.web_agent(options) @sig = options.fetch(:sig,DEFAULT_SIG) @key = options.fetch(:key,DEFAULT_KEY) @version = options.fetch(:version,DEFAULT_VERSION) super(options,&block) end
Public Instance Methods
A page containing results at the specified page index.
@param [Integer] page_index
The index of the page.
@return [Page<Result>]
A page object.
# File lib/gscraper/search/ajax_query.rb, line 200 def page(page_index) Page.new do |new_page| body = @agent.get(page_url(page_index)).body hash = JSON.parse(body.scan(/\{.*\}/).first) rank_offset = result_offset_of(page_index) if (hash.kind_of?(Hash) && hash['results']) hash['results'].each_with_index do |result,index| rank = rank_offset + (index + 1) title = Nokogiri::HTML(result['title']).inner_text url = URI(URI.escape(result['unescapedUrl'])) summary = unless result['content'].empty? Nokogiri::HTML(result['content']).inner_text else '' end cached_url = URI(result['cacheUrl']) new_page << Result.new(rank,title,url,summary,cached_url) end end end end
The URL that represents the query at a specific page index.
@param [Integer] page_index
The page index to create the URL for.
@return [URI::HTTP]
The query URL for the given page index.
# File lib/gscraper/search/ajax_query.rb, line 181 def page_url(page_index) url = search_url if page_index > 1 url.query_params['start'] = result_offset_of(page_index) end return url end
The results per page.
@return [Integer]
The number of results per page.
@see RESULTS_PER_PAGE
# File lib/gscraper/search/ajax_query.rb, line 145 def results_per_page RESULTS_PER_PAGE end
The URL that represents the query.
@return [URI::HTTP]
The URL for the query.
# File lib/gscraper/search/ajax_query.rb, line 155 def search_url search_url = URI::HTTP.build( :host => search_host, :path => PATH, :query => QUERY ) search_url.query_params['hl'] = @language search_url.query_params['gss'] = '.com' search_url.query_params['q'] = expression search_url.query_params['sig'] = @sig search_url.query_params['key'] = @key search_url.query_params['v'] = @version return search_url end