module Puppet::Util
MultiMatch
allows multiple values to be tested at once in a case expression. This class is needed since Array does not implement the === operator to mean “each v === other.each v”.
This class is useful in situations when the Puppet
Type
System cannot be used (e.g. in Logging
, since it needs to be able to log very early in the initialization cycle of puppet)
Typically used with the constants NOT_NIL TUPLE TRIPLE
which test against single NOT_NIL value, Array with two NOT_NIL, and Array with three NOT_NIL
Constants
- ALNUM
- ALPHA
From github.com/ruby/ruby/blob/v2_7_3/lib/uri/rfc2396_parser.rb#L24-L46
- AbsolutePathPosix
- AbsolutePathWindows
- DEFAULT_POSIX_MODE
Replace a file, securely. This takes a block, and passes it the file handle of a file open for writing. Write the replacement content inside the block and it will safely replace the target file.
This method will make no changes to the target file until the content is successfully written and the block returns without raising an error.
As far as possible the state of the existing file, such as mode, is preserved. This works hard to avoid loss of any metadata, but will result in an inode change for the file.
Arguments: `filename`, `default_mode`, `staging_location`
The filename is the file we are going to replace.
The default_mode is the mode to use when the target file doesn't already exist; if the file is present we copy the existing mode/owner/group values across. The default_mode can be expressed as an octal integer, a numeric string (ie '0664') or a symbolic file mode.
The staging_location is a location to render the temporary file before moving the file to it's final location.
- DEFAULT_WINDOWS_MODE
- ESCAPED
- HEX
- HttpProxy
for backwards compatibility
- PUPPET_STACK_INSERTION_FRAME
- RESERVED
- RFC_3986_URI_REGEX
- UNRESERVED
- UNSAFE
Public Class Methods
# File lib/puppet/util.rb 298 def absolute_path?(path, platform=nil) 299 unless path.is_a?(String) 300 Puppet.warning("Cannot check if #{path} is an absolute path because it is a '#{path.class}' and not a String'") 301 return false 302 end 303 304 # Ruby only sets File::ALT_SEPARATOR on Windows and the Ruby standard 305 # library uses that to test what platform it's on. Normally in Puppet we 306 # would use Puppet.features.microsoft_windows?, but this method needs to 307 # be called during the initialization of features so it can't depend on 308 # that. 309 # 310 # @deprecated Use ruby's built-in methods to determine if a path is absolute. 311 platform ||= Puppet::Util::Platform.windows? ? :windows : :posix 312 regex = case platform 313 when :windows 314 AbsolutePathWindows 315 when :posix 316 AbsolutePathPosix 317 else 318 raise Puppet::DevError, _("unknown platform %{platform} in absolute_path") % { platform: platform } 319 end 320 321 !! (path =~ regex) 322 end
execute a block of work and based on the logging level provided, log the provided message with the seconds taken The message 'msg' should include string ' in %{seconds} seconds' as part of the message and any content should escape any percent signs '%' so that they are not interpreted as formatting commands
escaped_str = str.gsub(/%/, '%%')
@param msg [String] the message to be formated to assigned the %{seconds} seconds take to execute,
other percent signs '%' need to be escaped
@param level [Symbol] the logging level for this message @param object [Object] The object use for logging the message
# File lib/puppet/util.rb 210 def benchmark(*args) 211 msg = args.pop 212 level = args.pop 213 object = if args.empty? 214 if respond_to?(level) 215 self 216 else 217 Puppet 218 end 219 else 220 args.pop 221 end 222 223 #TRANSLATORS 'benchmark' is a method name and should not be translated 224 raise Puppet::DevError, _("Failed to provide level to benchmark") unless level 225 226 unless level == :none or object.respond_to? level 227 raise Puppet::DevError, _("Benchmarked object does not respond to %{value}") % { value: level } 228 end 229 230 # Only benchmark if our log level is high enough 231 if level != :none and Puppet::Util::Log.sendlevel?(level) 232 seconds = Benchmark.realtime { 233 yield 234 } 235 object.send(level, msg % { seconds: "%0.2f" % seconds }) 236 return seconds 237 else 238 yield 239 end 240 end
Change the process to a different user
# File lib/puppet/util.rb 150 def self.chuser 151 group = Puppet[:group] 152 if group 153 begin 154 Puppet::Util::SUIDManager.change_group(group, true) 155 rescue => detail 156 Puppet.warning _("could not change to group %{group}: %{detail}") % { group: group.inspect, detail: detail } 157 $stderr.puts _("could not change to group %{group}") % { group: group.inspect } 158 159 # Don't exit on failed group changes, since it's 160 # not fatal 161 #exit(74) 162 end 163 end 164 165 user = Puppet[:user] 166 if user 167 begin 168 Puppet::Util::SUIDManager.change_user(user, true) 169 rescue => detail 170 $stderr.puts _("Could not change to user %{user}: %{detail}") % { user: user, detail: detail } 171 exit(74) 172 end 173 end 174 end
Removes all environment variables @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 72 def clear_environment(mode = default_env) 73 case mode 74 when :posix 75 ENV.clear 76 when :windows 77 Puppet::Util::Windows::Process.get_environment_strings.each do |key, _| 78 Puppet::Util::Windows::Process.set_environment_variable(key, nil) 79 end 80 else 81 raise _("Unable to clear the environment for mode %{mode}") % { mode: mode } 82 end 83 end
# File lib/puppet/util.rb 29 def default_env 30 Puppet.features.microsoft_windows? ? 31 :windows : 32 :posix 33 end
# File lib/puppet/util.rb 762 def deterministic_rand(seed,max) 763 deterministic_rand_int(seed, max).to_s 764 end
# File lib/puppet/util.rb 767 def deterministic_rand_int(seed,max) 768 Random.new(seed).rand(max) 769 end
Executes a block of code, wrapped with some special exception handling. Causes the ruby interpreter to
exit if the block throws an exception.
@api public @param [String] message a message to log if the block fails @param [Integer] code the exit code that the ruby interpreter should return if the block fails @yield
# File lib/puppet/util.rb 743 def exit_on_fail(message, code = 1) 744 yield 745 # First, we need to check and see if we are catching a SystemExit error. These will be raised 746 # when we daemonize/fork, and they do not necessarily indicate a failure case. 747 rescue SystemExit => err 748 raise err 749 750 # Now we need to catch *any* other kind of exception, because we may be calling third-party 751 # code (e.g. webrick), and we have no idea what they might throw. 752 rescue Exception => err 753 ## NOTE: when debugging spec failures, these two lines can be very useful 754 #puts err.inspect 755 #puts Puppet::Util.pretty_backtrace(err.backtrace) 756 Puppet.log_exception(err, "#{message}: #{err}") 757 Puppet::Util::Log.force_flushqueue() 758 exit(code) 759 end
arguments may be a Ruby stack, with an optional Puppet
stack argument, or just a Puppet
stack. stacks may be an Array of Strings “/foo.rb:0 in `blah'” or an Array of Arrays that represent a frame: [“/foo.pp”, 0]
# File lib/puppet/util.rb 575 def self.format_backtrace_array(primary_stack, puppetstack = []) 576 primary_stack.flat_map do |frame| 577 frame = format_puppetstack_frame(frame) if frame.is_a?(Array) 578 primary_frame = resolve_stackframe(frame) 579 580 if primary_frame =~ PUPPET_STACK_INSERTION_FRAME && !puppetstack.empty? 581 [resolve_stackframe(format_puppetstack_frame(puppetstack.shift)), 582 primary_frame] 583 else 584 primary_frame 585 end 586 end 587 end
# File lib/puppet/util.rb 599 def self.format_puppetstack_frame(file_and_lineno) 600 file_and_lineno.join(':') 601 end
@param name [String] The name of the environment variable to retrieve @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [String] Value of the specified environment variable. nil if it does not exist @api private
# File lib/puppet/util.rb 40 def get_env(name, mode = default_env) 41 if mode == :windows 42 Puppet::Util::Windows::Process.get_environment_strings.each do |key, value | 43 if name.casecmp(key) == 0 then 44 return value 45 end 46 end 47 return nil 48 else 49 ENV[name] 50 end 51 end
@param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [Hash] A hashtable of all environment variables @api private
# File lib/puppet/util.rb 57 def get_environment(mode = default_env) 58 case mode 59 when :posix 60 ENV.to_hash 61 when :windows 62 Puppet::Util::Windows::Process.get_environment_strings 63 else 64 raise _("Unable to retrieve the environment for mode %{mode}") % { mode: mode } 65 end 66 end
Create instance methods for each of the log levels. This allows the messages to be a little richer. Most classes will be calling this method.
# File lib/puppet/util.rb 179 def self.logmethods(klass, useself = true) 180 Puppet::Util::Log.eachlevel { |level| 181 klass.send(:define_method, level, proc { |args| 182 args = args.join(" ") if args.is_a?(Array) 183 if useself 184 185 Puppet::Util::Log.create( 186 :level => level, 187 :source => self, 188 :message => args 189 ) 190 else 191 192 Puppet::Util::Log.create( 193 :level => level, 194 :message => args 195 ) 196 end 197 }) 198 } 199 end
@param name [Hash] Environment variables to merge into the existing environment. nil values will remove the variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 105 def merge_environment(env_hash, mode = default_env) 106 case mode 107 when :posix 108 env_hash.each { |name, val| ENV[name.to_s] = val } 109 when :windows 110 env_hash.each do |name, val| 111 Puppet::Util::Windows::Process.set_environment_variable(name.to_s, val) 112 end 113 else 114 raise _("Unable to merge given values into the current environment for mode %{mode}") % { mode: mode } 115 end 116 end
Convert a path to a file URI
# File lib/puppet/util.rb 326 def path_to_uri(path) 327 return unless path 328 329 params = { :scheme => 'file' } 330 331 if Puppet::Util::Platform.windows? 332 path = path.tr('\\', '/') 333 334 unc = /^\/\/([^\/]+)(\/.+)/.match(path) 335 if unc 336 params[:host] = unc[1] 337 path = unc[2] 338 elsif path =~ /^[a-z]:\//i 339 path = '/' + path 340 end 341 end 342 343 # have to split *after* any relevant escaping 344 params[:path], params[:query] = uri_encode(path).split('?') 345 search_for_fragment = params[:query] ? :query : :path 346 if params[search_for_fragment].include?('#') 347 params[search_for_fragment], _, params[:fragment] = params[search_for_fragment].rpartition('#') 348 end 349 350 begin 351 URI::Generic.build(params) 352 rescue => detail 353 raise Puppet::Error, _("Failed to convert '%{path}' to URI: %{detail}") % { path: path, detail: detail }, detail.backtrace 354 end 355 end
utility method to get the current call stack and format it to a human-readable string (which some IDEs/editors will recognize as links to the line numbers in the trace)
# File lib/puppet/util.rb 567 def self.pretty_backtrace(backtrace = caller(1), puppetstack = []) 568 format_backtrace_array(backtrace, puppetstack).join("\n") 569 end
# File lib/puppet/util.rb 629 def replace_file(file, default_mode, staging_location: nil, validate_callback: nil, &block) 630 raise Puppet::DevError, _("replace_file requires a block") unless block_given? 631 632 if default_mode 633 unless valid_symbolic_mode?(default_mode) 634 raise Puppet::DevError, _("replace_file default_mode: %{default_mode} is invalid") % { default_mode: default_mode } 635 end 636 637 mode = symbolic_mode_to_int(normalize_symbolic_mode(default_mode)) 638 else 639 if Puppet::Util::Platform.windows? 640 mode = DEFAULT_WINDOWS_MODE 641 else 642 mode = DEFAULT_POSIX_MODE 643 end 644 end 645 646 begin 647 file = Puppet::FileSystem.pathname(file) 648 649 # encoding for Uniquefile is not important here because the caller writes to it as it sees fit 650 if staging_location 651 tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), staging_location) 652 else 653 tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), Puppet::FileSystem.dir_string(file)) 654 end 655 656 657 effective_mode = 658 if !Puppet::Util::Platform.windows? 659 # Grab the current file mode, and fall back to the defaults. 660 661 if Puppet::FileSystem.exist?(file) 662 stat = Puppet::FileSystem.lstat(file) 663 tempfile.chown(stat.uid, stat.gid) 664 stat.mode 665 else 666 mode 667 end 668 end 669 670 # OK, now allow the caller to write the content of the file. 671 yield tempfile 672 673 if effective_mode 674 # We only care about the bottom four slots, which make the real mode, 675 # and not the rest of the platform stat call fluff and stuff. 676 tempfile.chmod(effective_mode & 07777) 677 end 678 679 # Now, make sure the data (which includes the mode) is safe on disk. 680 tempfile.flush 681 begin 682 tempfile.fsync 683 rescue NotImplementedError 684 # fsync may not be implemented by Ruby on all platforms, but 685 # there is absolutely no recovery path if we detect that. So, we just 686 # ignore the return code. 687 # 688 # However, don't be fooled: that is accepting that we are running in 689 # an unsafe fashion. If you are porting to a new platform don't stub 690 # that out. 691 end 692 693 tempfile.close 694 695 if validate_callback 696 validate_callback.call(tempfile.path) 697 end 698 699 if Puppet::Util::Platform.windows? 700 # Windows ReplaceFile needs a file to exist, so touch handles this 701 if !Puppet::FileSystem.exist?(file) 702 Puppet::FileSystem.touch(file) 703 if mode 704 Puppet::Util::Windows::Security.set_mode(mode, Puppet::FileSystem.path_string(file)) 705 end 706 end 707 # Yes, the arguments are reversed compared to the rename in the rest 708 # of the world. 709 Puppet::Util::Windows::File.replace_file(FileSystem.path_string(file), tempfile.path) 710 711 else 712 # MRI Ruby checks for this and raises an error, while JRuby removes the directory 713 # and replaces it with a file. This makes the our version of replace_file() consistent 714 if Puppet::FileSystem.exist?(file) && Puppet::FileSystem.directory?(file) 715 raise Errno::EISDIR, _("Is a directory: %{directory}") % { directory: file } 716 end 717 File.rename(tempfile.path, Puppet::FileSystem.path_string(file)) 718 end 719 ensure 720 # in case an error occurred before we renamed the temp file, make sure it 721 # gets deleted 722 if tempfile 723 tempfile.close! 724 end 725 end 726 727 728 # Ideally, we would now fsync the directory as well, but Ruby doesn't 729 # have support for that, and it doesn't matter /that/ much... 730 731 # Return something true, and possibly useful. 732 file 733 end
# File lib/puppet/util.rb 589 def self.resolve_stackframe(frame) 590 _, path, rest = /^(.*):(\d+.*)$/.match(frame).to_a 591 if path 592 path = Pathname(path).realpath rescue path 593 "#{path}:#{rest}" 594 else 595 frame 596 end 597 end
# File lib/puppet/util.rb 498 def rfc2396_escape(str) 499 str.gsub(UNSAFE) do |match| 500 tmp = '' 501 match.each_byte do |uc| 502 tmp << sprintf('%%%02X', uc) 503 end 504 tmp 505 end.force_encoding(Encoding::US_ASCII) 506 end
# File lib/puppet/util.rb 516 def safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block) 517 child_pid = Kernel.fork do 518 STDIN.reopen(stdin) 519 STDOUT.reopen(stdout) 520 STDERR.reopen(stderr) 521 522 $stdin = STDIN 523 $stdout = STDOUT 524 $stderr = STDERR 525 526 begin 527 Dir.foreach('/proc/self/fd') do |f| 528 if f != '.' && f != '..' && f.to_i >= 3 529 IO::new(f.to_i).close rescue nil 530 end 531 end 532 rescue Errno::ENOENT # /proc/self/fd not found 533 3.upto(256){|fd| IO::new(fd).close rescue nil} 534 end 535 536 block.call if block 537 end 538 child_pid 539 end
@param name [String] The name of the environment variable to set @param value [String] The value to set the variable to. nil deletes the environment variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 90 def set_env(name, value = nil, mode = default_env) 91 case mode 92 when :posix 93 ENV[name] = value 94 when :windows 95 Puppet::Util::Windows::Process.set_environment_variable(name,value) 96 else 97 raise _("Unable to set the environment variable %{name} for mode %{mode}") % { name: name, mode: mode } 98 end 99 end
Executes a block of code, wrapped around Facter.load_external(false) and Facter.load_external(true) which will cause Facter to not evaluate external facts.
# File lib/puppet/util.rb 774 def skip_external_facts 775 return yield unless Facter.respond_to? :load_external 776 begin 777 Facter.load_external(false) 778 yield 779 ensure 780 Facter.load_external(true) 781 end 782 end
# File lib/puppet/util.rb 542 def symbolizehash(hash) 543 newhash = {} 544 hash.each do |name, val| 545 name = name.intern if name.respond_to? :intern 546 newhash[name] = val 547 end 548 newhash 549 end
Just benchmark, with no logging.
# File lib/puppet/util.rb 553 def thinmark 554 seconds = Benchmark.realtime { 555 yield 556 } 557 558 seconds 559 end
Percent-encodes a URI string per RFC3986 - tools.ietf.org/html/rfc3986
Properly handles escaping rules for paths, query strings and fragments independently
The output is safe to pass to URI.parse or URI::Generic.build and will correctly round-trip through URI.unescape
@param [String path] A URI string that may be in the form of:
http://foo.com/bar?query file://tmp/foo bar //foo.com/bar?query /bar?query bar?query bar . C:\Windows\Temp Note that with no specified scheme, authority or query parameter delimiter ? that a naked string will be treated as a path. Note that if query parameters need to contain data such as & or = that this method should not be used, as there is no way to differentiate query parameter data from query delimiters when multiple parameters are specified
@param [Hash{Symbol=>String} opts] Options to alter encoding @option opts [Array<Symbol>] :allow_fragment defaults to false. When false
will treat # as part of a path or query and not a fragment delimiter
@return [String] a new string containing appropriate portions of the URI
encoded per the rules of RFC3986. In particular, path will not encode +, but will encode space as %20 query will encode + as %2B and space as %20 fragment behaves like query
# File lib/puppet/util.rb 452 def uri_encode(path, opts = { :allow_fragment => false }) 453 raise ArgumentError.new(_('path may not be nil')) if path.nil? 454 455 # ensure string starts as UTF-8 for the sake of Ruby 1.9.3 456 encoded = ''.encode!(Encoding::UTF_8) 457 458 # parse uri into named matches, then reassemble properly encoded 459 parts = path.match(RFC_3986_URI_REGEX) 460 461 encoded += parts[:scheme] unless parts[:scheme].nil? 462 encoded += parts[:authority] unless parts[:authority].nil? 463 464 # path requires space to be encoded as %20 (NEVER +) 465 # + should be left unencoded 466 # URI::parse and URI::Generic.build don't like paths encoded with CGI.escape 467 # URI.escape does not change / to %2F and : to %3A like CGI.escape 468 # 469 encoded += rfc2396_escape(parts[:path]) unless parts[:path].nil? 470 471 # each query parameter 472 if !parts[:query].nil? 473 query_string = parts[:query].split('&').map do |pair| 474 # can optionally be separated by an = 475 pair.split('=').map do |v| 476 uri_query_encode(v) 477 end.join('=') 478 end.join('&') 479 encoded += '?' + query_string 480 end 481 482 encoded += ((opts[:allow_fragment] ? '#' : '%23') + uri_query_encode(parts[:fragment])) unless parts[:fragment].nil? 483 484 encoded 485 end
Percent-encodes a URI query parameter per RFC3986 - tools.ietf.org/html/rfc3986
The output will correctly round-trip through URI.unescape
@param [String query_string] A URI query parameter that may contain reserved
characters that must be percent encoded for the key or value to be properly decoded as part of a larger query string: query encodes as : query query_with_special=chars like&and * and# plus+this encodes as: query_with_special%3Dchars%20like%26and%20%2A%20and%23%20plus%2Bthis Note: Also usable by fragments, but not suitable for paths
@return [String] a new string containing an encoded query string per the
rules of RFC3986. In particular, query will encode + as %2B and space as %20
# File lib/puppet/util.rb 402 def uri_query_encode(query_string) 403 return nil if query_string.nil? 404 405 # query can encode space to %20 OR + 406 # + MUST be encoded as %2B 407 # in RFC3968 both query and fragment are defined as: 408 # = *( pchar / "/" / "?" ) 409 # CGI.escape turns space into + which is the most backward compatible 410 # however it doesn't roundtrip through URI.unescape which prefers %20 411 CGI.escape(query_string).gsub('+', '%20') 412 end
Get the path component of a URI
# File lib/puppet/util.rb 359 def uri_to_path(uri) 360 return unless uri.is_a?(URI) 361 362 # CGI.unescape doesn't handle space rules properly in uri paths 363 # URI.unescape does, but returns strings in their original encoding 364 path = uri_unescape(uri.path.encode(Encoding::UTF_8)) 365 366 if Puppet::Util::Platform.windows? && uri.scheme == 'file' 367 if uri.host && !uri.host.empty? 368 path = "//#{uri.host}" + path # UNC 369 else 370 path.sub!(/^\//, '') 371 end 372 end 373 374 path 375 end
# File lib/puppet/util.rb 509 def uri_unescape(str) 510 enc = str.encoding 511 enc = Encoding::UTF_8 if enc == Encoding::US_ASCII 512 str.gsub(ESCAPED) { [$&[1, 2]].pack('H2').force_encoding(enc) } 513 end
Resolve a path for an executable to the absolute path. This tries to behave in the same manner as the unix `which` command and uses the `PATH` environment variable.
@api public @param bin [String] the name of the executable to find. @return [String] the absolute path to the found executable.
# File lib/puppet/util.rb 250 def which(bin) 251 if absolute_path?(bin) 252 return bin if FileTest.file? bin and FileTest.executable? bin 253 else 254 exts = Puppet::Util.get_env('PATHEXT') 255 exts = exts ? exts.split(File::PATH_SEPARATOR) : %w[.COM .EXE .BAT .CMD] 256 Puppet::Util.get_env('PATH').split(File::PATH_SEPARATOR).each do |dir| 257 begin 258 dest = File.expand_path(File.join(dir, bin)) 259 rescue ArgumentError => e 260 # if the user's PATH contains a literal tilde (~) character and HOME is not set, we may get 261 # an ArgumentError here. Let's check to see if that is the case; if not, re-raise whatever error 262 # was thrown. 263 if e.to_s =~ /HOME/ and (Puppet::Util.get_env('HOME').nil? || Puppet::Util.get_env('HOME') == "") 264 # if we get here they have a tilde in their PATH. We'll issue a single warning about this and then 265 # ignore this path element and carry on with our lives. 266 #TRANSLATORS PATH and HOME are environment variables and should not be translated 267 Puppet::Util::Warnings.warnonce(_("PATH contains a ~ character, and HOME is not set; ignoring PATH element '%{dir}'.") % { dir: dir }) 268 elsif e.to_s =~ /doesn't exist|can't find user/ 269 # ...otherwise, we just skip the non-existent entry, and do nothing. 270 #TRANSLATORS PATH is an environment variable and should not be translated 271 Puppet::Util::Warnings.warnonce(_("Couldn't expand PATH containing a ~ character; ignoring PATH element '%{dir}'.") % { dir: dir }) 272 else 273 raise 274 end 275 else 276 if Puppet::Util::Platform.windows? && File.extname(dest).empty? 277 exts.each do |ext| 278 destext = File.expand_path(dest + ext) 279 return destext if FileTest.file? destext and FileTest.executable? destext 280 end 281 end 282 return dest if FileTest.file? dest and FileTest.executable? dest 283 end 284 end 285 end 286 nil 287 end
Run some code with a specific environment. Resets the environment back to what it was at the end of the code. Windows
can store Unicode chars in the environment as keys or values, but Ruby's ENV tries to roundtrip them through the local codepage, which can cause encoding problems - underlying helpers use Windows
APIs on Windows
see bugs.ruby-lang.org/issues/8822
# File lib/puppet/util.rb 125 def withenv(hash, mode = :posix) 126 saved = get_environment(mode) 127 merge_environment(hash, mode) 128 yield 129 ensure 130 if saved 131 clear_environment(mode) 132 merge_environment(saved, mode) 133 end 134 end
Execute a given chunk of code with a new umask.
# File lib/puppet/util.rb 138 def self.withumask(mask) 139 cur = File.umask(mask) 140 141 begin 142 yield 143 ensure 144 File.umask(cur) 145 end 146 end
Private Instance Methods
# File lib/puppet/util.rb 298 def absolute_path?(path, platform=nil) 299 unless path.is_a?(String) 300 Puppet.warning("Cannot check if #{path} is an absolute path because it is a '#{path.class}' and not a String'") 301 return false 302 end 303 304 # Ruby only sets File::ALT_SEPARATOR on Windows and the Ruby standard 305 # library uses that to test what platform it's on. Normally in Puppet we 306 # would use Puppet.features.microsoft_windows?, but this method needs to 307 # be called during the initialization of features so it can't depend on 308 # that. 309 # 310 # @deprecated Use ruby's built-in methods to determine if a path is absolute. 311 platform ||= Puppet::Util::Platform.windows? ? :windows : :posix 312 regex = case platform 313 when :windows 314 AbsolutePathWindows 315 when :posix 316 AbsolutePathPosix 317 else 318 raise Puppet::DevError, _("unknown platform %{platform} in absolute_path") % { platform: platform } 319 end 320 321 !! (path =~ regex) 322 end
execute a block of work and based on the logging level provided, log the provided message with the seconds taken The message 'msg' should include string ' in %{seconds} seconds' as part of the message and any content should escape any percent signs '%' so that they are not interpreted as formatting commands
escaped_str = str.gsub(/%/, '%%')
@param msg [String] the message to be formated to assigned the %{seconds} seconds take to execute,
other percent signs '%' need to be escaped
@param level [Symbol] the logging level for this message @param object [Object] The object use for logging the message
# File lib/puppet/util.rb 210 def benchmark(*args) 211 msg = args.pop 212 level = args.pop 213 object = if args.empty? 214 if respond_to?(level) 215 self 216 else 217 Puppet 218 end 219 else 220 args.pop 221 end 222 223 #TRANSLATORS 'benchmark' is a method name and should not be translated 224 raise Puppet::DevError, _("Failed to provide level to benchmark") unless level 225 226 unless level == :none or object.respond_to? level 227 raise Puppet::DevError, _("Benchmarked object does not respond to %{value}") % { value: level } 228 end 229 230 # Only benchmark if our log level is high enough 231 if level != :none and Puppet::Util::Log.sendlevel?(level) 232 seconds = Benchmark.realtime { 233 yield 234 } 235 object.send(level, msg % { seconds: "%0.2f" % seconds }) 236 return seconds 237 else 238 yield 239 end 240 end
Removes all environment variables @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 72 def clear_environment(mode = default_env) 73 case mode 74 when :posix 75 ENV.clear 76 when :windows 77 Puppet::Util::Windows::Process.get_environment_strings.each do |key, _| 78 Puppet::Util::Windows::Process.set_environment_variable(key, nil) 79 end 80 else 81 raise _("Unable to clear the environment for mode %{mode}") % { mode: mode } 82 end 83 end
# File lib/puppet/util.rb 29 def default_env 30 Puppet.features.microsoft_windows? ? 31 :windows : 32 :posix 33 end
# File lib/puppet/util.rb 762 def deterministic_rand(seed,max) 763 deterministic_rand_int(seed, max).to_s 764 end
# File lib/puppet/util.rb 767 def deterministic_rand_int(seed,max) 768 Random.new(seed).rand(max) 769 end
Executes a block of code, wrapped with some special exception handling. Causes the ruby interpreter to
exit if the block throws an exception.
@api public @param [String] message a message to log if the block fails @param [Integer] code the exit code that the ruby interpreter should return if the block fails @yield
# File lib/puppet/util.rb 743 def exit_on_fail(message, code = 1) 744 yield 745 # First, we need to check and see if we are catching a SystemExit error. These will be raised 746 # when we daemonize/fork, and they do not necessarily indicate a failure case. 747 rescue SystemExit => err 748 raise err 749 750 # Now we need to catch *any* other kind of exception, because we may be calling third-party 751 # code (e.g. webrick), and we have no idea what they might throw. 752 rescue Exception => err 753 ## NOTE: when debugging spec failures, these two lines can be very useful 754 #puts err.inspect 755 #puts Puppet::Util.pretty_backtrace(err.backtrace) 756 Puppet.log_exception(err, "#{message}: #{err}") 757 Puppet::Util::Log.force_flushqueue() 758 exit(code) 759 end
@param name [String] The name of the environment variable to retrieve @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [String] Value of the specified environment variable. nil if it does not exist @api private
# File lib/puppet/util.rb 40 def get_env(name, mode = default_env) 41 if mode == :windows 42 Puppet::Util::Windows::Process.get_environment_strings.each do |key, value | 43 if name.casecmp(key) == 0 then 44 return value 45 end 46 end 47 return nil 48 else 49 ENV[name] 50 end 51 end
@param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [Hash] A hashtable of all environment variables @api private
# File lib/puppet/util.rb 57 def get_environment(mode = default_env) 58 case mode 59 when :posix 60 ENV.to_hash 61 when :windows 62 Puppet::Util::Windows::Process.get_environment_strings 63 else 64 raise _("Unable to retrieve the environment for mode %{mode}") % { mode: mode } 65 end 66 end
@param name [Hash] Environment variables to merge into the existing environment. nil values will remove the variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 105 def merge_environment(env_hash, mode = default_env) 106 case mode 107 when :posix 108 env_hash.each { |name, val| ENV[name.to_s] = val } 109 when :windows 110 env_hash.each do |name, val| 111 Puppet::Util::Windows::Process.set_environment_variable(name.to_s, val) 112 end 113 else 114 raise _("Unable to merge given values into the current environment for mode %{mode}") % { mode: mode } 115 end 116 end
Convert a path to a file URI
# File lib/puppet/util.rb 326 def path_to_uri(path) 327 return unless path 328 329 params = { :scheme => 'file' } 330 331 if Puppet::Util::Platform.windows? 332 path = path.tr('\\', '/') 333 334 unc = /^\/\/([^\/]+)(\/.+)/.match(path) 335 if unc 336 params[:host] = unc[1] 337 path = unc[2] 338 elsif path =~ /^[a-z]:\//i 339 path = '/' + path 340 end 341 end 342 343 # have to split *after* any relevant escaping 344 params[:path], params[:query] = uri_encode(path).split('?') 345 search_for_fragment = params[:query] ? :query : :path 346 if params[search_for_fragment].include?('#') 347 params[search_for_fragment], _, params[:fragment] = params[search_for_fragment].rpartition('#') 348 end 349 350 begin 351 URI::Generic.build(params) 352 rescue => detail 353 raise Puppet::Error, _("Failed to convert '%{path}' to URI: %{detail}") % { path: path, detail: detail }, detail.backtrace 354 end 355 end
# File lib/puppet/util.rb 629 def replace_file(file, default_mode, staging_location: nil, validate_callback: nil, &block) 630 raise Puppet::DevError, _("replace_file requires a block") unless block_given? 631 632 if default_mode 633 unless valid_symbolic_mode?(default_mode) 634 raise Puppet::DevError, _("replace_file default_mode: %{default_mode} is invalid") % { default_mode: default_mode } 635 end 636 637 mode = symbolic_mode_to_int(normalize_symbolic_mode(default_mode)) 638 else 639 if Puppet::Util::Platform.windows? 640 mode = DEFAULT_WINDOWS_MODE 641 else 642 mode = DEFAULT_POSIX_MODE 643 end 644 end 645 646 begin 647 file = Puppet::FileSystem.pathname(file) 648 649 # encoding for Uniquefile is not important here because the caller writes to it as it sees fit 650 if staging_location 651 tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), staging_location) 652 else 653 tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), Puppet::FileSystem.dir_string(file)) 654 end 655 656 657 effective_mode = 658 if !Puppet::Util::Platform.windows? 659 # Grab the current file mode, and fall back to the defaults. 660 661 if Puppet::FileSystem.exist?(file) 662 stat = Puppet::FileSystem.lstat(file) 663 tempfile.chown(stat.uid, stat.gid) 664 stat.mode 665 else 666 mode 667 end 668 end 669 670 # OK, now allow the caller to write the content of the file. 671 yield tempfile 672 673 if effective_mode 674 # We only care about the bottom four slots, which make the real mode, 675 # and not the rest of the platform stat call fluff and stuff. 676 tempfile.chmod(effective_mode & 07777) 677 end 678 679 # Now, make sure the data (which includes the mode) is safe on disk. 680 tempfile.flush 681 begin 682 tempfile.fsync 683 rescue NotImplementedError 684 # fsync may not be implemented by Ruby on all platforms, but 685 # there is absolutely no recovery path if we detect that. So, we just 686 # ignore the return code. 687 # 688 # However, don't be fooled: that is accepting that we are running in 689 # an unsafe fashion. If you are porting to a new platform don't stub 690 # that out. 691 end 692 693 tempfile.close 694 695 if validate_callback 696 validate_callback.call(tempfile.path) 697 end 698 699 if Puppet::Util::Platform.windows? 700 # Windows ReplaceFile needs a file to exist, so touch handles this 701 if !Puppet::FileSystem.exist?(file) 702 Puppet::FileSystem.touch(file) 703 if mode 704 Puppet::Util::Windows::Security.set_mode(mode, Puppet::FileSystem.path_string(file)) 705 end 706 end 707 # Yes, the arguments are reversed compared to the rename in the rest 708 # of the world. 709 Puppet::Util::Windows::File.replace_file(FileSystem.path_string(file), tempfile.path) 710 711 else 712 # MRI Ruby checks for this and raises an error, while JRuby removes the directory 713 # and replaces it with a file. This makes the our version of replace_file() consistent 714 if Puppet::FileSystem.exist?(file) && Puppet::FileSystem.directory?(file) 715 raise Errno::EISDIR, _("Is a directory: %{directory}") % { directory: file } 716 end 717 File.rename(tempfile.path, Puppet::FileSystem.path_string(file)) 718 end 719 ensure 720 # in case an error occurred before we renamed the temp file, make sure it 721 # gets deleted 722 if tempfile 723 tempfile.close! 724 end 725 end 726 727 728 # Ideally, we would now fsync the directory as well, but Ruby doesn't 729 # have support for that, and it doesn't matter /that/ much... 730 731 # Return something true, and possibly useful. 732 file 733 end
# File lib/puppet/util.rb 498 def rfc2396_escape(str) 499 str.gsub(UNSAFE) do |match| 500 tmp = '' 501 match.each_byte do |uc| 502 tmp << sprintf('%%%02X', uc) 503 end 504 tmp 505 end.force_encoding(Encoding::US_ASCII) 506 end
# File lib/puppet/util.rb 516 def safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block) 517 child_pid = Kernel.fork do 518 STDIN.reopen(stdin) 519 STDOUT.reopen(stdout) 520 STDERR.reopen(stderr) 521 522 $stdin = STDIN 523 $stdout = STDOUT 524 $stderr = STDERR 525 526 begin 527 Dir.foreach('/proc/self/fd') do |f| 528 if f != '.' && f != '..' && f.to_i >= 3 529 IO::new(f.to_i).close rescue nil 530 end 531 end 532 rescue Errno::ENOENT # /proc/self/fd not found 533 3.upto(256){|fd| IO::new(fd).close rescue nil} 534 end 535 536 block.call if block 537 end 538 child_pid 539 end
@param name [String] The name of the environment variable to set @param value [String] The value to set the variable to. nil deletes the environment variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private
# File lib/puppet/util.rb 90 def set_env(name, value = nil, mode = default_env) 91 case mode 92 when :posix 93 ENV[name] = value 94 when :windows 95 Puppet::Util::Windows::Process.set_environment_variable(name,value) 96 else 97 raise _("Unable to set the environment variable %{name} for mode %{mode}") % { name: name, mode: mode } 98 end 99 end
Executes a block of code, wrapped around Facter.load_external(false) and Facter.load_external(true) which will cause Facter to not evaluate external facts.
# File lib/puppet/util.rb 774 def skip_external_facts 775 return yield unless Facter.respond_to? :load_external 776 begin 777 Facter.load_external(false) 778 yield 779 ensure 780 Facter.load_external(true) 781 end 782 end
# File lib/puppet/util.rb 542 def symbolizehash(hash) 543 newhash = {} 544 hash.each do |name, val| 545 name = name.intern if name.respond_to? :intern 546 newhash[name] = val 547 end 548 newhash 549 end
Just benchmark, with no logging.
# File lib/puppet/util.rb 553 def thinmark 554 seconds = Benchmark.realtime { 555 yield 556 } 557 558 seconds 559 end
Percent-encodes a URI string per RFC3986 - tools.ietf.org/html/rfc3986
Properly handles escaping rules for paths, query strings and fragments independently
The output is safe to pass to URI.parse or URI::Generic.build and will correctly round-trip through URI.unescape
@param [String path] A URI string that may be in the form of:
http://foo.com/bar?query file://tmp/foo bar //foo.com/bar?query /bar?query bar?query bar . C:\Windows\Temp Note that with no specified scheme, authority or query parameter delimiter ? that a naked string will be treated as a path. Note that if query parameters need to contain data such as & or = that this method should not be used, as there is no way to differentiate query parameter data from query delimiters when multiple parameters are specified
@param [Hash{Symbol=>String} opts] Options to alter encoding @option opts [Array<Symbol>] :allow_fragment defaults to false. When false
will treat # as part of a path or query and not a fragment delimiter
@return [String] a new string containing appropriate portions of the URI
encoded per the rules of RFC3986. In particular, path will not encode +, but will encode space as %20 query will encode + as %2B and space as %20 fragment behaves like query
# File lib/puppet/util.rb 452 def uri_encode(path, opts = { :allow_fragment => false }) 453 raise ArgumentError.new(_('path may not be nil')) if path.nil? 454 455 # ensure string starts as UTF-8 for the sake of Ruby 1.9.3 456 encoded = ''.encode!(Encoding::UTF_8) 457 458 # parse uri into named matches, then reassemble properly encoded 459 parts = path.match(RFC_3986_URI_REGEX) 460 461 encoded += parts[:scheme] unless parts[:scheme].nil? 462 encoded += parts[:authority] unless parts[:authority].nil? 463 464 # path requires space to be encoded as %20 (NEVER +) 465 # + should be left unencoded 466 # URI::parse and URI::Generic.build don't like paths encoded with CGI.escape 467 # URI.escape does not change / to %2F and : to %3A like CGI.escape 468 # 469 encoded += rfc2396_escape(parts[:path]) unless parts[:path].nil? 470 471 # each query parameter 472 if !parts[:query].nil? 473 query_string = parts[:query].split('&').map do |pair| 474 # can optionally be separated by an = 475 pair.split('=').map do |v| 476 uri_query_encode(v) 477 end.join('=') 478 end.join('&') 479 encoded += '?' + query_string 480 end 481 482 encoded += ((opts[:allow_fragment] ? '#' : '%23') + uri_query_encode(parts[:fragment])) unless parts[:fragment].nil? 483 484 encoded 485 end
Percent-encodes a URI query parameter per RFC3986 - tools.ietf.org/html/rfc3986
The output will correctly round-trip through URI.unescape
@param [String query_string] A URI query parameter that may contain reserved
characters that must be percent encoded for the key or value to be properly decoded as part of a larger query string: query encodes as : query query_with_special=chars like&and * and# plus+this encodes as: query_with_special%3Dchars%20like%26and%20%2A%20and%23%20plus%2Bthis Note: Also usable by fragments, but not suitable for paths
@return [String] a new string containing an encoded query string per the
rules of RFC3986. In particular, query will encode + as %2B and space as %20
# File lib/puppet/util.rb 402 def uri_query_encode(query_string) 403 return nil if query_string.nil? 404 405 # query can encode space to %20 OR + 406 # + MUST be encoded as %2B 407 # in RFC3968 both query and fragment are defined as: 408 # = *( pchar / "/" / "?" ) 409 # CGI.escape turns space into + which is the most backward compatible 410 # however it doesn't roundtrip through URI.unescape which prefers %20 411 CGI.escape(query_string).gsub('+', '%20') 412 end
Get the path component of a URI
# File lib/puppet/util.rb 359 def uri_to_path(uri) 360 return unless uri.is_a?(URI) 361 362 # CGI.unescape doesn't handle space rules properly in uri paths 363 # URI.unescape does, but returns strings in their original encoding 364 path = uri_unescape(uri.path.encode(Encoding::UTF_8)) 365 366 if Puppet::Util::Platform.windows? && uri.scheme == 'file' 367 if uri.host && !uri.host.empty? 368 path = "//#{uri.host}" + path # UNC 369 else 370 path.sub!(/^\//, '') 371 end 372 end 373 374 path 375 end
# File lib/puppet/util.rb 509 def uri_unescape(str) 510 enc = str.encoding 511 enc = Encoding::UTF_8 if enc == Encoding::US_ASCII 512 str.gsub(ESCAPED) { [$&[1, 2]].pack('H2').force_encoding(enc) } 513 end
Resolve a path for an executable to the absolute path. This tries to behave in the same manner as the unix `which` command and uses the `PATH` environment variable.
@api public @param bin [String] the name of the executable to find. @return [String] the absolute path to the found executable.
# File lib/puppet/util.rb 250 def which(bin) 251 if absolute_path?(bin) 252 return bin if FileTest.file? bin and FileTest.executable? bin 253 else 254 exts = Puppet::Util.get_env('PATHEXT') 255 exts = exts ? exts.split(File::PATH_SEPARATOR) : %w[.COM .EXE .BAT .CMD] 256 Puppet::Util.get_env('PATH').split(File::PATH_SEPARATOR).each do |dir| 257 begin 258 dest = File.expand_path(File.join(dir, bin)) 259 rescue ArgumentError => e 260 # if the user's PATH contains a literal tilde (~) character and HOME is not set, we may get 261 # an ArgumentError here. Let's check to see if that is the case; if not, re-raise whatever error 262 # was thrown. 263 if e.to_s =~ /HOME/ and (Puppet::Util.get_env('HOME').nil? || Puppet::Util.get_env('HOME') == "") 264 # if we get here they have a tilde in their PATH. We'll issue a single warning about this and then 265 # ignore this path element and carry on with our lives. 266 #TRANSLATORS PATH and HOME are environment variables and should not be translated 267 Puppet::Util::Warnings.warnonce(_("PATH contains a ~ character, and HOME is not set; ignoring PATH element '%{dir}'.") % { dir: dir }) 268 elsif e.to_s =~ /doesn't exist|can't find user/ 269 # ...otherwise, we just skip the non-existent entry, and do nothing. 270 #TRANSLATORS PATH is an environment variable and should not be translated 271 Puppet::Util::Warnings.warnonce(_("Couldn't expand PATH containing a ~ character; ignoring PATH element '%{dir}'.") % { dir: dir }) 272 else 273 raise 274 end 275 else 276 if Puppet::Util::Platform.windows? && File.extname(dest).empty? 277 exts.each do |ext| 278 destext = File.expand_path(dest + ext) 279 return destext if FileTest.file? destext and FileTest.executable? destext 280 end 281 end 282 return dest if FileTest.file? dest and FileTest.executable? dest 283 end 284 end 285 end 286 nil 287 end
Run some code with a specific environment. Resets the environment back to what it was at the end of the code. Windows
can store Unicode chars in the environment as keys or values, but Ruby's ENV tries to roundtrip them through the local codepage, which can cause encoding problems - underlying helpers use Windows
APIs on Windows
see bugs.ruby-lang.org/issues/8822
# File lib/puppet/util.rb 125 def withenv(hash, mode = :posix) 126 saved = get_environment(mode) 127 merge_environment(hash, mode) 128 yield 129 ensure 130 if saved 131 clear_environment(mode) 132 merge_environment(saved, mode) 133 end 134 end