class Puppet::HTTP::Client
The HTTP client provides methods for making `GET`, `POST`, etc requests to HTTP(S) servers. It also provides methods for resolving Puppetserver REST service endpoints using SRV records and settings (such as `server_list`, `server`, `ca_server`, etc). Once a service endpoint has been resolved, there are methods for making REST requests (such as getting a node, sending facts, etc).
The client uses persistent HTTP connections by default unless the `Connection: close` header is specified and supports streaming response bodies.
By default the client only trusts the Puppet
CA for HTTPS connections. However, if the `include_system_store` request option is set to true, then Puppet
will trust certificates in the puppet-agent CA bundle.
@example To access the HTTP client:
client = Puppet.runtime[:http]
@example To make an HTTP GET request:
response = client.get(URI("http://www.example.com"))
@example To make an HTTPS GET request, trusting the puppet CA and certs in Puppet's CA bundle:
response = client.get(URI("https://www.example.com"), include_system_store: true)
@example To use a URL containing special characters, such as spaces:
response = client.get(URI(Puppet::Util.uri_encode("https://www.example.com/path to file")))
@example To pass query parameters:
response = client.get(URI("https://www.example.com"), query: {'q' => 'puppet'})
@example To pass custom headers:
response = client.get(URI("https://www.example.com"), headers: {'Accept-Content' => 'application/json'})
@example To check if the response is successful (2xx):
response = client.get(URI("http://www.example.com")) puts response.success?
@example To get the response code and reason:
response = client.get(URI("http://www.example.com")) unless response.success? puts "HTTP #{response.code} #{response.reason}" end
@example To read response headers:
response = client.get(URI("http://www.example.com")) puts response['Content-Type']
@example To stream the response body:
client.get(URI("http://www.example.com")) do |response| if response.success? response.read_body do |data| puts data end end end
@example To handle exceptions:
begin client.get(URI("https://www.example.com")) rescue Puppet::HTTP::ResponseError => e puts "HTTP #{e.response.code} #{e.response.reason}" rescue Puppet::HTTP::ConnectionError => e puts "Connection error #{e.message}" rescue Puppet::SSL::SSLError => e puts "SSL error #{e.message}" rescue Puppet::HTTP::HTTPError => e puts "General HTTP error #{e.message}" end
@example To route to the `:puppet` service:
session = client.create_session service = session.route_to(:puppet)
@example To make a node request:
node = service.get_node(Puppet[:certname], environment: 'production')
@example To submit facts:
facts = Puppet::Indirection::Facts.indirection.find(Puppet[:certname]) service.put_facts(Puppet[:certname], environment: 'production', facts: facts)
@example To submit a report to the `:report` service:
report = Puppet::Transaction::Report.new service = session.route_to(:report) service.put_report(Puppet[:certname], report, environment: 'production')
@api public
Attributes
Public Class Methods
Create a new http client instance. Use `Puppet.runtime` to get the current client instead of creating an instance of this class.
@param [Puppet::HTTP::Pool] pool pool of persistent Net::HTTP
connections
@param [Puppet::SSL::SSLContext] ssl_context ssl context to be used for
connections
@param [Puppet::SSL::SSLContext] system_ssl_context
the system ssl context
used if :include_system_store is set to true
@param [Integer] redirect_limit default number of HTTP redirections to allow
in a given request. Can also be specified per-request.
@param [Integer] retry_limit number of HTTP reties allowed in a given
request
# File lib/puppet/http/client.rb 104 def initialize(pool: Puppet::HTTP::Pool.new(Puppet[:http_keepalive_timeout]), ssl_context: nil, system_ssl_context: nil, redirect_limit: 10, retry_limit: 100) 105 @pool = pool 106 @default_headers = { 107 'X-Puppet-Version' => Puppet.version, 108 'User-Agent' => Puppet[:http_user_agent], 109 }.freeze 110 @default_ssl_context = ssl_context 111 @default_system_ssl_context = system_ssl_context 112 @default_redirect_limit = redirect_limit 113 @retry_after_handler = Puppet::HTTP::RetryAfterHandler.new(retry_limit, Puppet[:runinterval]) 114 end
Public Instance Methods
Close persistent connections in the pool.
@return [void]
@api public
# File lib/puppet/http/client.rb 301 def close 302 @pool.close 303 end
Open a connection to the given URI. It is typically not necessary to call this method as the client will create connections as needed when a request is made.
@param [URI] uri the connection destination @param [Hash] options @option options [Puppet::SSL::SSLContext] :ssl_context (nil) ssl context to
be used for connections
@option options [Boolean] :include_system_store (false) if we should include
the system store for connection
# File lib/puppet/http/client.rb 136 def connect(uri, options: {}, &block) 137 start = Time.now 138 verifier = nil 139 connected = false 140 141 site = Puppet::HTTP::Site.from_uri(uri) 142 if site.use_ssl? 143 ssl_context = options.fetch(:ssl_context, nil) 144 include_system_store = options.fetch(:include_system_store, false) 145 ctx = resolve_ssl_context(ssl_context, include_system_store) 146 verifier = Puppet::SSL::Verifier.new(site.host, ctx) 147 end 148 149 @pool.with_connection(site, verifier) do |http| 150 connected = true 151 if block_given? 152 yield http 153 end 154 end 155 rescue Net::OpenTimeout => e 156 raise_error(_("Request to %{uri} timed out connect operation after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected) 157 rescue Net::ReadTimeout => e 158 raise_error(_("Request to %{uri} timed out read operation after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected) 159 rescue EOFError => e 160 raise_error(_("Request to %{uri} interrupted after %{elapsed} seconds") % {uri: uri, elapsed: elapsed(start)}, e, connected) 161 rescue Puppet::SSL::SSLError 162 raise 163 rescue Puppet::HTTP::HTTPError 164 raise 165 rescue => e 166 raise_error(_("Request to %{uri} failed after %{elapsed} seconds: %{message}") % 167 {uri: uri, elapsed: elapsed(start), message: e.message}, e, connected) 168 end
Create a new HTTP session. A session is the object through which services may be connected to and accessed.
@return [Puppet::HTTP::Session] the newly created HTTP session
@api public
# File lib/puppet/http/client.rb 122 def create_session 123 Puppet::HTTP::Session.new(self, build_resolvers) 124 end
Submits a DELETE HTTP request to the given url.
@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options
@return [Puppet::HTTP::Response] the response
@api public
# File lib/puppet/http/client.rb 288 def delete(url, headers: {}, params: {}, options: {}) 289 url = encode_query(url, params) 290 291 request = Net::HTTP::Delete.new(url, @default_headers.merge(headers)) 292 293 execute_streaming(request, options: options) 294 end
Submits a GET HTTP request to the given url
@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options
@yield [Puppet::HTTP::Response] if a block is given yields the response
@return [Puppet::HTTP::Response] the response
@api public
# File lib/puppet/http/client.rb 198 def get(url, headers: {}, params: {}, options: {}, &block) 199 url = encode_query(url, params) 200 201 request = Net::HTTP::Get.new(url, @default_headers.merge(headers)) 202 203 execute_streaming(request, options: options, &block) 204 end
Submits a HEAD HTTP request to the given url
@param [URI] url the location to submit the http request @param [Hash] headers merged with the default headers defined by the client @param [Hash] params encoded and set as the url query @!macro request_options
@return [Puppet::HTTP::Response] the response
@api public
# File lib/puppet/http/client.rb 216 def head(url, headers: {}, params: {}, options: {}) 217 url = encode_query(url, params) 218 219 request = Net::HTTP::Head.new(url, @default_headers.merge(headers)) 220 221 execute_streaming(request, options: options) 222 end
Submits a POST HTTP request to the given url
@param [URI] url the location to submit the http request @param [String] body the body of the POST request @param [Hash] headers merged with the default headers defined by the client. The
`Content-Type` header is required and should correspond to the type of data passed as the `body` argument.
@param [Hash] params encoded and set as the url query @!macro request_options
@yield [Puppet::HTTP::Response] if a block is given yields the response
@return [Puppet::HTTP::Response] the response
@api public
# File lib/puppet/http/client.rb 265 def post(url, body, headers: {}, params: {}, options: {}, &block) 266 raise ArgumentError, "'post' requires a string 'body' argument" unless body.is_a?(String) 267 url = encode_query(url, params) 268 269 request = Net::HTTP::Post.new(url, @default_headers.merge(headers)) 270 request.body = body 271 request.content_length = body.bytesize 272 273 raise ArgumentError, "'post' requires a 'content-type' header" unless request['Content-Type'] 274 275 execute_streaming(request, options: options, &block) 276 end
Submits a PUT HTTP request to the given url
@param [URI] url the location to submit the http request @param [String] body the body of the PUT request @param [Hash] headers merged with the default headers defined by the client. The
`Content-Type` header is required and should correspond to the type of data passed as the `body` argument.
@param [Hash] params encoded and set as the url query @!macro request_options
@return [Puppet::HTTP::Response] the response
@api public
# File lib/puppet/http/client.rb 237 def put(url, body, headers: {}, params: {}, options: {}) 238 raise ArgumentError, "'put' requires a string 'body' argument" unless body.is_a?(String) 239 url = encode_query(url, params) 240 241 request = Net::HTTP::Put.new(url, @default_headers.merge(headers)) 242 request.body = body 243 request.content_length = body.bytesize 244 245 raise ArgumentError, "'put' requires a 'content-type' header" unless request['Content-Type'] 246 247 execute_streaming(request, options: options) 248 end
Protected Instance Methods
# File lib/puppet/http/client.rb 307 def encode_query(url, params) 308 return url if params.empty? 309 310 url = url.dup 311 url.query = encode_params(params) 312 url 313 end
Private Instance Methods
# File lib/puppet/http/client.rb 464 def apply_auth(request, basic_auth) 465 if basic_auth 466 request.basic_auth(basic_auth[:user], basic_auth[:password]) 467 end 468 end
# File lib/puppet/http/client.rb 470 def build_resolvers 471 resolvers = [] 472 473 if Puppet[:use_srv_records] 474 resolvers << Puppet::HTTP::Resolver::SRV.new(self, domain: Puppet[:srv_domain]) 475 end 476 477 server_list_setting = Puppet.settings.setting(:server_list) 478 if server_list_setting.value && !server_list_setting.value.empty? 479 # use server list to resolve all services 480 services = Puppet::HTTP::Service::SERVICE_NAMES.dup 481 482 # except if it's been explicitly set 483 if Puppet.settings.set_by_config?(:ca_server) 484 services.delete(:ca) 485 end 486 487 if Puppet.settings.set_by_config?(:report_server) 488 services.delete(:report) 489 end 490 491 resolvers << Puppet::HTTP::Resolver::ServerList.new(self, server_list_setting: server_list_setting, default_port: Puppet[:serverport], services: services) 492 end 493 494 resolvers << Puppet::HTTP::Resolver::Settings.new(self) 495 496 resolvers.freeze 497 end
# File lib/puppet/http/client.rb 431 def elapsed(start) 432 (Time.now - start).to_f.round(3) 433 end
# File lib/puppet/http/client.rb 424 def encode_params(params) 425 params = expand_into_parameters(params) 426 params.map do |key, value| 427 "#{key}=#{Puppet::Util.uri_query_encode(value.to_s)}" 428 end.join('&') 429 end
Connect or borrow a connection from the pool to the host and port associated with the request's URL. Then execute the HTTP request, retrying and following redirects as needed, and return the HTTP response. The response body will always be fully drained/consumed when this method returns.
If a block is provided, then the response will be yielded to the caller, allowing the response body to be streamed.
If the request/response did not result in an exception and the caller did not ask for the connection to be closed (via Connection: close), then the connection will be returned to the pool.
@yieldparam [Puppet::HTTP::Response] response The final response, after following redirects and retrying @return [Puppet::HTTP::Response]
# File lib/puppet/http/client.rb 332 def execute_streaming(request, options: {}, &block) 333 redirector = Puppet::HTTP::Redirector.new(options.fetch(:redirect_limit, @default_redirect_limit)) 334 335 basic_auth = options.fetch(:basic_auth, nil) 336 unless basic_auth 337 if request.uri.user && request.uri.password 338 basic_auth = { user: request.uri.user, password: request.uri.password } 339 end 340 end 341 342 redirects = 0 343 retries = 0 344 response = nil 345 done = false 346 347 while !done do 348 connect(request.uri, options: options) do |http| 349 apply_auth(request, basic_auth) 350 351 # don't call return within the `request` block 352 http.request(request) do |nethttp| 353 response = Puppet::HTTP::ResponseNetHTTP.new(request.uri, nethttp) 354 begin 355 Puppet.debug("HTTP #{request.method.upcase} #{request.uri} returned #{response.code} #{response.reason}") 356 357 if redirector.redirect?(request, response) 358 request = redirector.redirect_to(request, response, redirects) 359 redirects += 1 360 next 361 elsif @retry_after_handler.retry_after?(request, response) 362 interval = @retry_after_handler.retry_after_interval(request, response, retries) 363 retries += 1 364 if interval 365 if http.started? 366 Puppet.debug("Closing connection for #{Puppet::HTTP::Site.from_uri(request.uri)}") 367 http.finish 368 end 369 Puppet.warning(_("Sleeping for %{interval} seconds before retrying the request") % { interval: interval }) 370 ::Kernel.sleep(interval) 371 next 372 end 373 end 374 375 if block_given? 376 yield response 377 else 378 response.body 379 end 380 ensure 381 # we need to make sure the response body is fully consumed before 382 # the connection is put back in the pool, otherwise the response 383 # for one request could leak into a future response. 384 response.drain 385 end 386 387 done = true 388 end 389 end 390 end 391 392 response 393 end
# File lib/puppet/http/client.rb 395 def expand_into_parameters(data) 396 data.inject([]) do |params, key_value| 397 key, value = key_value 398 399 expanded_value = case value 400 when Array 401 value.collect { |val| [key, val] } 402 else 403 [key_value] 404 end 405 406 params.concat(expand_primitive_types_into_parameters(expanded_value)) 407 end 408 end
# File lib/puppet/http/client.rb 410 def expand_primitive_types_into_parameters(data) 411 data.inject([]) do |params, key_value| 412 key, value = key_value 413 case value 414 when nil 415 params 416 when true, false, String, Symbol, Integer, Float 417 params << [key, value] 418 else 419 raise Puppet::HTTP::SerializationError, _("HTTP REST queries cannot handle values of type '%{klass}'") % { klass: value.class } 420 end 421 end 422 end
# File lib/puppet/http/client.rb 435 def raise_error(message, cause, connected) 436 if connected 437 raise Puppet::HTTP::HTTPError.new(message, cause) 438 else 439 raise Puppet::HTTP::ConnectionError.new(message, cause) 440 end 441 end
# File lib/puppet/http/client.rb 443 def resolve_ssl_context(ssl_context, include_system_store) 444 if ssl_context 445 raise Puppet::HTTP::HTTPError, "The ssl_context and include_system_store parameters are mutually exclusive" if include_system_store 446 ssl_context 447 elsif include_system_store 448 system_ssl_context 449 else 450 @default_ssl_context || Puppet.lookup(:ssl_context) 451 end 452 end
# File lib/puppet/http/client.rb 454 def system_ssl_context 455 return @default_system_ssl_context if @default_system_ssl_context 456 457 cert_provider = Puppet::X509::CertProvider.new 458 cacerts = cert_provider.load_cacerts || [] 459 460 ssl = Puppet::SSL::SSLProvider.new 461 @default_system_ssl_context = ssl.create_system_context(cacerts: cacerts) 462 end