class Puppet::HTTP::Client

The HTTP client provides methods for making `GET`, `POST`, etc requests to HTTP(S) servers. It also provides methods for resolving Puppetserver REST service endpoints using SRV records and settings (such as `server_list`, `server`, `ca_server`, etc). Once a service endpoint has been resolved, there are methods for making REST requests (such as getting a node, sending facts, etc).

The client uses persistent HTTP connections by default unless the `Connection: close` header is specified and supports streaming response bodies.

By default the client only trusts the Puppet CA for HTTPS connections. However, if the `include_system_store` request option is set to true, then Puppet will trust certificates in the puppet-agent CA bundle.

@example To access the HTTP client:

client = Puppet.runtime[:http]

@example To make an HTTP GET request:

response = client.get(URI("http://www.example.com"))

@example To make an HTTPS GET request, trusting the puppet CA and certs in Puppet's CA bundle:

response = client.get(URI("https://www.example.com"), include_system_store: true)

@example To use a URL containing special characters, such as spaces:

response = client.get(URI(Puppet::Util.uri_encode("https://www.example.com/path to file")))

@example To pass query parameters:

response = client.get(URI("https://www.example.com"), query: {'q' => 'puppet'})

@example To pass custom headers:

response = client.get(URI("https://www.example.com"), headers: {'Accept-Content' => 'application/json'})

@example To check if the response is successful (2xx):

response = client.get(URI("http://www.example.com"))
puts response.success?

@example To get the response code and reason:

response = client.get(URI("http://www.example.com"))
unless response.success?
  puts "HTTP #{response.code} #{response.reason}"
 end

@example To read response headers:

response = client.get(URI("http://www.example.com"))
puts response['Content-Type']

@example To stream the response body:

client.get(URI("http://www.example.com")) do |response|
  if response.success?
    response.read_body do |data|
      puts data
    end
  end
end

@example To handle exceptions:

begin
  client.get(URI("https://www.example.com"))
rescue Puppet::HTTP::ResponseError => e
  puts "HTTP #{e.response.code} #{e.response.reason}"
rescue Puppet::HTTP::ConnectionError => e
  puts "Connection error #{e.message}"
rescue Puppet::SSL::SSLError => e
  puts "SSL error #{e.message}"
rescue Puppet::HTTP::HTTPError => e
  puts "General HTTP error #{e.message}"
end

@example To route to the `:puppet` service:

session = client.create_session
service = session.route_to(:puppet)

@example To make a node request:

node = service.get_node(Puppet[:certname], environment: 'production')

@example To submit facts:

facts = Puppet::Indirection::Facts.indirection.find(Puppet[:certname])
service.put_facts(Puppet[:certname], environment: 'production', facts: facts)

@example To submit a report to the `:report` service:

report = Puppet::Transaction::Report.new
service = session.route_to(:report)
service.put_report(Puppet[:certname], report, environment: 'production')

@api public

Attributes

pool[R]

Public Class Methods

new(pool: Puppet::HTTP::Pool.new(Puppet[:http_keepalive_timeout]), ssl_context: nil, system_ssl_context: nil, redirect_limit: 10, retry_limit: 100) click to toggle source

Create a new http client instance. Use `Puppet.runtime` to get the current client instead of creating an instance of this class.

@param [Puppet::HTTP::Pool] pool pool of persistent Net::HTTP

connections

@param [Puppet::SSL::SSLContext] ssl_context ssl context to be used for

connections

@param [Puppet::SSL::SSLContext] system_ssl_context the system ssl context

used if :include_system_store is set to true

@param [Integer] redirect_limit default number of HTTP redirections to allow

in a given request. Can also be specified per-request.

@param [Integer] retry_limit number of HTTP reties allowed in a given

request
    # File lib/puppet/http/client.rb
104 def initialize(pool: Puppet::HTTP::Pool.new(Puppet[:http_keepalive_timeout]), ssl_context: nil, system_ssl_context: nil, redirect_limit: 10, retry_limit: 100)
105   @pool = pool
106   @default_headers = {
107     'X-Puppet-Version' => Puppet.version,
108     'User-Agent' => Puppet[:http_user_agent],
109   }.freeze
110   @default_ssl_context = ssl_context
111   @default_system_ssl_context = system_ssl_context
112   @default_redirect_limit = redirect_limit
113   @retry_after_handler = Puppet::HTTP::RetryAfterHandler.new(retry_limit, Puppet[:runinterval])
114 end

Public Instance Methods

close() click to toggle source

Close persistent connections in the pool.

@return [void]

@api public

    # File lib/puppet/http/client.rb
301 def close
302   @pool.close
303 end
connect(uri, options: {}) { |http| ... } click to toggle source

Open a connection to the given URI. It is typically not necessary to call this method as the client will create connections as needed when a request is made.

@param [URI] uri the connection destination @param [Hash] options @option options [Puppet::SSL::SSLContext] :ssl_context (nil) ssl context to

be used for connections

@option options [Boolean] :include_system_store (false) if we should include

the system store for connection
    # File lib/puppet/http/client.rb
136 def connect(uri, options: {}, &block)
137   start = Time.now
138   verifier = nil
139   connected = false
140 
141   site = Puppet::HTTP::Site.from_uri(uri)
142   if site.use_ssl?
143     ssl_context = options.fetch(:ssl_context, nil)
144     include_system_store = options.fetch(:include_system_store, false)
145     ctx = resolve_ssl_context(ssl_context, include_system_store)
146     verifier = Puppet::SSL::Verifier.new(site.host, ctx)
147   end
148 
149   @pool.with_connection(site, verifier) do |http|
150     connected = true
151     if block_given?
152       yield http
153     end
154   end
155 rescue Net::OpenTimeout => e
156   raise_error(_("Request to %{uri} timed out connect operation after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected)
157 rescue Net::ReadTimeout => e
158   raise_error(_("Request to %{uri} timed out read operation after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected)
159 rescue EOFError => e
160   raise_error(_("Request to %{uri} interrupted after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected)
161 rescue Puppet::SSL::SSLError
162   raise
163 rescue Puppet::HTTP::HTTPError
164   raise
165 rescue => e
166   raise_error(_("Request to %{uri} failed after %{elapsed} seconds: %{message}") %
167               {uri: uri, elapsed: elapsed(start), message: e.message}, e, connected)
168 end
create_session() click to toggle source

Create a new HTTP session. A session is the object through which services may be connected to and accessed.

@return [Puppet::HTTP::Session] the newly created HTTP session

@api public

    # File lib/puppet/http/client.rb
122 def create_session
123   Puppet::HTTP::Session.new(self, build_resolvers)
124 end
delete(url, headers: {}, params: {}, options: {}) click to toggle source

Submits a DELETE HTTP request to the given url.

@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options

@return [Puppet::HTTP::Response] the response

@api public

    # File lib/puppet/http/client.rb
288 def delete(url, headers: {}, params: {}, options: {})
289   url = encode_query(url, params)
290 
291   request = Net::HTTP::Delete.new(url, @default_headers.merge(headers))
292 
293   execute_streaming(request, options: options)
294 end
get(url, headers: {}, params: {}, options: {}, &block) click to toggle source

Submits a GET HTTP request to the given url

@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options

@yield [Puppet::HTTP::Response] if a block is given yields the response

@return [Puppet::HTTP::Response] the response

@api public

    # File lib/puppet/http/client.rb
198 def get(url, headers: {}, params: {}, options: {}, &block)
199   url = encode_query(url, params)
200 
201   request = Net::HTTP::Get.new(url, @default_headers.merge(headers))
202 
203   execute_streaming(request, options: options, &block)
204 end
head(url, headers: {}, params: {}, options: {}) click to toggle source

Submits a HEAD HTTP request to the given url

@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options

@return [Puppet::HTTP::Response] the response

@api public

    # File lib/puppet/http/client.rb
216 def head(url, headers: {}, params: {}, options: {})
217   url = encode_query(url, params)
218 
219   request = Net::HTTP::Head.new(url, @default_headers.merge(headers))
220 
221   execute_streaming(request, options: options)
222 end
post(url, body, headers: {}, params: {}, options: {}, &block) click to toggle source

Submits a POST HTTP request to the given url

@param [URI] url the location to submit the http request @param [String] body the body of the POST request @param [Hash] headers merged with the default headers defined by the client. The

`Content-Type` header is required and should correspond to the type of data passed
as the `body` argument.

@param [Hash] params encoded and set as the url query @!macro request_options

@yield [Puppet::HTTP::Response] if a block is given yields the response

@return [Puppet::HTTP::Response] the response

@api public

    # File lib/puppet/http/client.rb
265 def post(url, body, headers: {}, params: {}, options: {}, &block)
266   raise ArgumentError, "'post' requires a string 'body' argument" unless body.is_a?(String)
267   url = encode_query(url, params)
268 
269   request = Net::HTTP::Post.new(url, @default_headers.merge(headers))
270   request.body = body
271   request.content_length = body.bytesize
272 
273   raise ArgumentError, "'post' requires a 'content-type' header" unless request['Content-Type']
274 
275   execute_streaming(request, options: options, &block)
276 end
put(url, body, headers: {}, params: {}, options: {}) click to toggle source

Submits a PUT HTTP request to the given url

@param [URI] url the location to submit the http request @param [String] body the body of the PUT request @param [Hash] headers merged with the default headers defined by the client. The

`Content-Type` header is required and should correspond to the type of data passed
as the `body` argument.

@param [Hash] params encoded and set as the url query @!macro request_options

@return [Puppet::HTTP::Response] the response

@api public

    # File lib/puppet/http/client.rb
237 def put(url, body, headers: {}, params: {}, options: {})
238   raise ArgumentError, "'put' requires a string 'body' argument" unless body.is_a?(String)
239   url = encode_query(url, params)
240 
241   request = Net::HTTP::Put.new(url, @default_headers.merge(headers))
242   request.body = body
243   request.content_length = body.bytesize
244 
245   raise ArgumentError, "'put' requires a 'content-type' header" unless request['Content-Type']
246 
247   execute_streaming(request, options: options)
248 end

Protected Instance Methods

encode_query(url, params) click to toggle source
    # File lib/puppet/http/client.rb
307 def encode_query(url, params)
308   return url if params.empty?
309 
310   url = url.dup
311   url.query = encode_params(params)
312   url
313 end

Private Instance Methods

apply_auth(request, basic_auth) click to toggle source
    # File lib/puppet/http/client.rb
464 def apply_auth(request, basic_auth)
465   if basic_auth
466     request.basic_auth(basic_auth[:user], basic_auth[:password])
467   end
468 end
build_resolvers() click to toggle source
    # File lib/puppet/http/client.rb
470 def build_resolvers
471   resolvers = []
472 
473   if Puppet[:use_srv_records]
474     resolvers << Puppet::HTTP::Resolver::SRV.new(self, domain: Puppet[:srv_domain])
475   end
476 
477   server_list_setting = Puppet.settings.setting(:server_list)
478   if server_list_setting.value && !server_list_setting.value.empty?
479     # use server list to resolve all services
480     services = Puppet::HTTP::Service::SERVICE_NAMES.dup
481 
482     # except if it's been explicitly set
483     if Puppet.settings.set_by_config?(:ca_server)
484       services.delete(:ca)
485     end
486 
487     if Puppet.settings.set_by_config?(:report_server)
488       services.delete(:report)
489     end
490 
491     resolvers << Puppet::HTTP::Resolver::ServerList.new(self, server_list_setting: server_list_setting, default_port: Puppet[:serverport], services: services)
492   end
493 
494   resolvers << Puppet::HTTP::Resolver::Settings.new(self)
495 
496   resolvers.freeze
497 end
elapsed(start) click to toggle source
    # File lib/puppet/http/client.rb
431 def elapsed(start)
432   (Time.now - start).to_f.round(3)
433 end
encode_params(params) click to toggle source
    # File lib/puppet/http/client.rb
424 def encode_params(params)
425   params = expand_into_parameters(params)
426   params.map do |key, value|
427     "#{key}=#{Puppet::Util.uri_query_encode(value.to_s)}"
428   end.join('&')
429 end
execute_streaming(request, options: {}) { |response| ... } click to toggle source

Connect or borrow a connection from the pool to the host and port associated with the request's URL. Then execute the HTTP request, retrying and following redirects as needed, and return the HTTP response. The response body will always be fully drained/consumed when this method returns.

If a block is provided, then the response will be yielded to the caller, allowing the response body to be streamed.

If the request/response did not result in an exception and the caller did not ask for the connection to be closed (via Connection: close), then the connection will be returned to the pool.

@yieldparam [Puppet::HTTP::Response] response The final response, after following redirects and retrying @return [Puppet::HTTP::Response]

    # File lib/puppet/http/client.rb
332 def execute_streaming(request, options: {}, &block)
333   redirector = Puppet::HTTP::Redirector.new(options.fetch(:redirect_limit, @default_redirect_limit))
334 
335   basic_auth = options.fetch(:basic_auth, nil)
336   unless basic_auth
337     if request.uri.user && request.uri.password
338       basic_auth = { user: request.uri.user, password: request.uri.password }
339     end
340   end
341 
342   redirects = 0
343   retries = 0
344   response = nil
345   done = false
346 
347   while !done do
348     connect(request.uri, options: options) do |http|
349       apply_auth(request, basic_auth)
350 
351       # don't call return within the `request` block
352       http.request(request) do |nethttp|
353         response = Puppet::HTTP::ResponseNetHTTP.new(request.uri, nethttp)
354         begin
355           Puppet.debug("HTTP #{request.method.upcase} #{request.uri} returned #{response.code} #{response.reason}")
356 
357           if redirector.redirect?(request, response)
358             request = redirector.redirect_to(request, response, redirects)
359             redirects += 1
360             next
361           elsif @retry_after_handler.retry_after?(request, response)
362             interval = @retry_after_handler.retry_after_interval(request, response, retries)
363             retries += 1
364             if interval
365               if http.started?
366                 Puppet.debug("Closing connection for #{Puppet::HTTP::Site.from_uri(request.uri)}")
367                 http.finish
368               end
369               Puppet.warning(_("Sleeping for %{interval} seconds before retrying the request") % { interval: interval })
370               ::Kernel.sleep(interval)
371               next
372             end
373           end
374 
375           if block_given?
376             yield response
377           else
378             response.body
379           end
380         ensure
381           # we need to make sure the response body is fully consumed before
382           # the connection is put back in the pool, otherwise the response
383           # for one request could leak into a future response.
384           response.drain
385         end
386 
387         done = true
388       end
389     end
390   end
391 
392   response
393 end
expand_into_parameters(data) click to toggle source
    # File lib/puppet/http/client.rb
395 def expand_into_parameters(data)
396   data.inject([]) do |params, key_value|
397     key, value = key_value
398 
399     expanded_value = case value
400                      when Array
401                        value.collect { |val| [key, val] }
402                      else
403                        [key_value]
404                      end
405 
406     params.concat(expand_primitive_types_into_parameters(expanded_value))
407   end
408 end
expand_primitive_types_into_parameters(data) click to toggle source
    # File lib/puppet/http/client.rb
410 def expand_primitive_types_into_parameters(data)
411   data.inject([]) do |params, key_value|
412     key, value = key_value
413     case value
414     when nil
415       params
416     when true, false, String, Symbol, Integer, Float
417       params << [key, value]
418     else
419       raise Puppet::HTTP::SerializationError, _("HTTP REST queries cannot handle values of type '%{klass}'") % { klass: value.class }
420     end
421   end
422 end
raise_error(message, cause, connected) click to toggle source
    # File lib/puppet/http/client.rb
435 def raise_error(message, cause, connected)
436   if connected
437     raise Puppet::HTTP::HTTPError.new(message, cause)
438   else
439     raise Puppet::HTTP::ConnectionError.new(message, cause)
440   end
441 end
resolve_ssl_context(ssl_context, include_system_store) click to toggle source
    # File lib/puppet/http/client.rb
443 def resolve_ssl_context(ssl_context, include_system_store)
444   if ssl_context
445     raise Puppet::HTTP::HTTPError, "The ssl_context and include_system_store parameters are mutually exclusive" if include_system_store
446     ssl_context
447   elsif include_system_store
448     system_ssl_context
449   else
450     @default_ssl_context || Puppet.lookup(:ssl_context)
451   end
452 end
system_ssl_context() click to toggle source
    # File lib/puppet/http/client.rb
454 def system_ssl_context
455   return @default_system_ssl_context if @default_system_ssl_context
456 
457   cert_provider = Puppet::X509::CertProvider.new
458   cacerts = cert_provider.load_cacerts || []
459 
460   ssl = Puppet::SSL::SSLProvider.new
461   @default_system_ssl_context = ssl.create_system_context(cacerts: cacerts)
462 end