module Puppet::Util

MultiMatch allows multiple values to be tested at once in a case expression. This class is needed since Array does not implement the === operator to mean “each v === other.each v”.

This class is useful in situations when the Puppet Type System cannot be used (e.g. in Logging, since it needs to be able to log very early in the initialization cycle of puppet)

Typically used with the constants NOT_NIL TUPLE TRIPLE

which test against single NOT_NIL value, Array with two NOT_NIL, and Array with three NOT_NIL

Constants

ALNUM
ALPHA

From github.com/ruby/ruby/blob/v2_7_3/lib/uri/rfc2396_parser.rb#L24-L46

AbsolutePathPosix
AbsolutePathWindows
DEFAULT_POSIX_MODE

Replace a file, securely. This takes a block, and passes it the file handle of a file open for writing. Write the replacement content inside the block and it will safely replace the target file.

This method will make no changes to the target file until the content is successfully written and the block returns without raising an error.

As far as possible the state of the existing file, such as mode, is preserved. This works hard to avoid loss of any metadata, but will result in an inode change for the file.

Arguments: `filename`, `default_mode`, `staging_location`

The filename is the file we are going to replace.

The default_mode is the mode to use when the target file doesn't already exist; if the file is present we copy the existing mode/owner/group values across. The default_mode can be expressed as an octal integer, a numeric string (ie '0664') or a symbolic file mode.

The staging_location is a location to render the temporary file before moving the file to it's final location.

DEFAULT_WINDOWS_MODE
ESCAPED
HEX
HttpProxy

for backwards compatibility

PUPPET_STACK_INSERTION_FRAME
RESERVED
RFC_3986_URI_REGEX
UNRESERVED
UNSAFE

Public Class Methods

absolute_path?(path, platform=nil) click to toggle source
    # File lib/puppet/util.rb
298 def absolute_path?(path, platform=nil)
299   unless path.is_a?(String)
300     Puppet.warning("Cannot check if #{path} is an absolute path because it is a '#{path.class}' and not a String'")
301     return false
302   end
303 
304   # Ruby only sets File::ALT_SEPARATOR on Windows and the Ruby standard
305   # library uses that to test what platform it's on.  Normally in Puppet we
306   # would use Puppet.features.microsoft_windows?, but this method needs to
307   # be called during the initialization of features so it can't depend on
308   # that.
309   #
310   # @deprecated Use ruby's built-in methods to determine if a path is absolute.
311   platform ||= Puppet::Util::Platform.windows? ? :windows : :posix
312   regex = case platform
313           when :windows
314             AbsolutePathWindows
315           when :posix
316             AbsolutePathPosix
317           else
318             raise Puppet::DevError, _("unknown platform %{platform} in absolute_path") % { platform: platform }
319           end
320 
321   !! (path =~ regex)
322 end
benchmark(*args) { || ... } click to toggle source

execute a block of work and based on the logging level provided, log the provided message with the seconds taken The message 'msg' should include string ' in %{seconds} seconds' as part of the message and any content should escape any percent signs '%' so that they are not interpreted as formatting commands

escaped_str = str.gsub(/%/, '%%')

@param msg [String] the message to be formated to assigned the %{seconds} seconds take to execute,

other percent signs '%' need to be escaped

@param level [Symbol] the logging level for this message @param object [Object] The object use for logging the message

    # File lib/puppet/util.rb
210 def benchmark(*args)
211   msg = args.pop
212   level = args.pop
213   object = if args.empty?
214              if respond_to?(level)
215                self
216              else
217                Puppet
218              end
219            else
220              args.pop
221            end
222 
223   #TRANSLATORS 'benchmark' is a method name and should not be translated
224   raise Puppet::DevError, _("Failed to provide level to benchmark") unless level
225 
226   unless level == :none or object.respond_to? level
227     raise Puppet::DevError, _("Benchmarked object does not respond to %{value}") % { value: level }
228   end
229 
230   # Only benchmark if our log level is high enough
231   if level != :none and Puppet::Util::Log.sendlevel?(level)
232     seconds = Benchmark.realtime {
233       yield
234     }
235     object.send(level, msg % { seconds: "%0.2f" % seconds })
236     return seconds
237   else
238     yield
239   end
240 end
chuser() click to toggle source

Change the process to a different user

    # File lib/puppet/util.rb
150 def self.chuser
151   group = Puppet[:group]
152   if group
153     begin
154       Puppet::Util::SUIDManager.change_group(group, true)
155     rescue => detail
156       Puppet.warning _("could not change to group %{group}: %{detail}") % { group: group.inspect, detail: detail }
157       $stderr.puts _("could not change to group %{group}") % { group: group.inspect }
158 
159       # Don't exit on failed group changes, since it's
160       # not fatal
161       #exit(74)
162     end
163   end
164 
165   user = Puppet[:user]
166   if user
167     begin
168       Puppet::Util::SUIDManager.change_user(user, true)
169     rescue => detail
170       $stderr.puts _("Could not change to user %{user}: %{detail}") % { user: user, detail: detail }
171       exit(74)
172     end
173   end
174 end
clear_environment(mode = default_env) click to toggle source

Removes all environment variables @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

   # File lib/puppet/util.rb
72 def clear_environment(mode = default_env)
73   case mode
74     when :posix
75       ENV.clear
76     when :windows
77       Puppet::Util::Windows::Process.get_environment_strings.each do |key, _|
78         Puppet::Util::Windows::Process.set_environment_variable(key, nil)
79       end
80     else
81       raise _("Unable to clear the environment for mode %{mode}") % { mode: mode }
82   end
83 end
default_env() click to toggle source
   # File lib/puppet/util.rb
29 def default_env
30   Puppet.features.microsoft_windows? ?
31     :windows :
32     :posix
33 end
deterministic_rand(seed,max) click to toggle source
    # File lib/puppet/util.rb
762 def deterministic_rand(seed,max)
763   deterministic_rand_int(seed, max).to_s
764 end
deterministic_rand_int(seed,max) click to toggle source
    # File lib/puppet/util.rb
767 def deterministic_rand_int(seed,max)
768   Random.new(seed).rand(max)
769 end
exit_on_fail(message, code = 1) { || ... } click to toggle source

Executes a block of code, wrapped with some special exception handling. Causes the ruby interpreter to

exit if the block throws an exception.

@api public @param [String] message a message to log if the block fails @param [Integer] code the exit code that the ruby interpreter should return if the block fails @yield

    # File lib/puppet/util.rb
743 def exit_on_fail(message, code = 1)
744   yield
745 # First, we need to check and see if we are catching a SystemExit error.  These will be raised
746 #  when we daemonize/fork, and they do not necessarily indicate a failure case.
747 rescue SystemExit => err
748   raise err
749 
750 # Now we need to catch *any* other kind of exception, because we may be calling third-party
751 #  code (e.g. webrick), and we have no idea what they might throw.
752 rescue Exception => err
753   ## NOTE: when debugging spec failures, these two lines can be very useful
754   #puts err.inspect
755   #puts Puppet::Util.pretty_backtrace(err.backtrace)
756   Puppet.log_exception(err, "#{message}: #{err}")
757   Puppet::Util::Log.force_flushqueue()
758   exit(code)
759 end
format_backtrace_array(primary_stack, puppetstack = []) click to toggle source

arguments may be a Ruby stack, with an optional Puppet stack argument, or just a Puppet stack. stacks may be an Array of Strings “/foo.rb:0 in `blah'” or an Array of Arrays that represent a frame: [“/foo.pp”, 0]

    # File lib/puppet/util.rb
575 def self.format_backtrace_array(primary_stack, puppetstack = [])
576   primary_stack.flat_map do |frame|
577     frame = format_puppetstack_frame(frame) if frame.is_a?(Array)
578     primary_frame = resolve_stackframe(frame)
579 
580     if primary_frame =~ PUPPET_STACK_INSERTION_FRAME && !puppetstack.empty?
581       [resolve_stackframe(format_puppetstack_frame(puppetstack.shift)),
582        primary_frame]
583     else
584       primary_frame
585     end
586   end
587 end
format_puppetstack_frame(file_and_lineno) click to toggle source
    # File lib/puppet/util.rb
599 def self.format_puppetstack_frame(file_and_lineno)
600   file_and_lineno.join(':')
601 end
get_env(name, mode = default_env) click to toggle source

@param name [String] The name of the environment variable to retrieve @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [String] Value of the specified environment variable. nil if it does not exist @api private

   # File lib/puppet/util.rb
40 def get_env(name, mode = default_env)
41   if mode == :windows
42     Puppet::Util::Windows::Process.get_environment_strings.each do |key, value |
43       if name.casecmp(key) == 0 then
44         return value
45       end
46     end
47     return nil
48   else
49     ENV[name]
50   end
51 end
get_environment(mode = default_env) click to toggle source

@param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [Hash] A hashtable of all environment variables @api private

   # File lib/puppet/util.rb
57 def get_environment(mode = default_env)
58   case mode
59     when :posix
60       ENV.to_hash
61     when :windows
62       Puppet::Util::Windows::Process.get_environment_strings
63     else
64       raise _("Unable to retrieve the environment for mode %{mode}") % { mode: mode }
65   end
66 end
logmethods(klass, useself = true) click to toggle source

Create instance methods for each of the log levels. This allows the messages to be a little richer. Most classes will be calling this method.

    # File lib/puppet/util.rb
179 def self.logmethods(klass, useself = true)
180   Puppet::Util::Log.eachlevel { |level|
181     klass.send(:define_method, level, proc { |args|
182       args = args.join(" ") if args.is_a?(Array)
183       if useself
184 
185         Puppet::Util::Log.create(
186           :level => level,
187           :source => self,
188           :message => args
189         )
190       else
191 
192         Puppet::Util::Log.create(
193           :level => level,
194           :message => args
195         )
196       end
197     })
198   }
199 end
merge_environment(env_hash, mode = default_env) click to toggle source

@param name [Hash] Environment variables to merge into the existing environment. nil values will remove the variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

    # File lib/puppet/util.rb
105 def merge_environment(env_hash, mode = default_env)
106   case mode
107     when :posix
108       env_hash.each { |name, val| ENV[name.to_s] = val }
109     when :windows
110       env_hash.each do |name, val|
111         Puppet::Util::Windows::Process.set_environment_variable(name.to_s, val)
112       end
113     else
114       raise _("Unable to merge given values into the current environment for mode %{mode}") % { mode: mode }
115   end
116 end
path_to_uri(path) click to toggle source

Convert a path to a file URI

    # File lib/puppet/util.rb
326 def path_to_uri(path)
327   return unless path
328 
329   params = { :scheme => 'file' }
330 
331   if Puppet::Util::Platform.windows?
332     path = path.tr('\\', '/')
333 
334     unc = /^\/\/([^\/]+)(\/.+)/.match(path)
335     if unc
336       params[:host] = unc[1]
337       path = unc[2]
338     elsif path =~ /^[a-z]:\//i
339       path = '/' + path
340     end
341   end
342 
343   # have to split *after* any relevant escaping
344   params[:path], params[:query] = uri_encode(path).split('?')
345   search_for_fragment = params[:query] ? :query : :path
346   if params[search_for_fragment].include?('#')
347     params[search_for_fragment], _, params[:fragment] = params[search_for_fragment].rpartition('#')
348   end
349 
350   begin
351     URI::Generic.build(params)
352   rescue => detail
353     raise Puppet::Error, _("Failed to convert '%{path}' to URI: %{detail}") % { path: path, detail: detail }, detail.backtrace
354   end
355 end
pretty_backtrace(backtrace = caller(1), puppetstack = []) click to toggle source

utility method to get the current call stack and format it to a human-readable string (which some IDEs/editors will recognize as links to the line numbers in the trace)

    # File lib/puppet/util.rb
567 def self.pretty_backtrace(backtrace = caller(1), puppetstack = [])
568   format_backtrace_array(backtrace, puppetstack).join("\n")
569 end
replace_file(file, default_mode, staging_location: nil, validate_callback: nil) { |tempfile| ... } click to toggle source
    # File lib/puppet/util.rb
629 def replace_file(file, default_mode, staging_location: nil, validate_callback: nil, &block)
630   raise Puppet::DevError, _("replace_file requires a block") unless block_given?
631 
632   if default_mode
633     unless valid_symbolic_mode?(default_mode)
634       raise Puppet::DevError, _("replace_file default_mode: %{default_mode} is invalid") % { default_mode: default_mode }
635     end
636 
637     mode = symbolic_mode_to_int(normalize_symbolic_mode(default_mode))
638   else
639     if Puppet::Util::Platform.windows?
640       mode = DEFAULT_WINDOWS_MODE
641     else
642       mode = DEFAULT_POSIX_MODE
643     end
644   end
645 
646   begin
647     file = Puppet::FileSystem.pathname(file)
648 
649     # encoding for Uniquefile is not important here because the caller writes to it as it sees fit
650     if staging_location
651       tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), staging_location)
652     else
653       tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), Puppet::FileSystem.dir_string(file))
654     end
655 
656 
657     effective_mode =
658     if !Puppet::Util::Platform.windows?
659       # Grab the current file mode, and fall back to the defaults.
660 
661       if Puppet::FileSystem.exist?(file)
662         stat = Puppet::FileSystem.lstat(file)
663         tempfile.chown(stat.uid, stat.gid)
664         stat.mode
665       else
666         mode
667       end
668     end
669 
670     # OK, now allow the caller to write the content of the file.
671     yield tempfile
672 
673     if effective_mode
674       # We only care about the bottom four slots, which make the real mode,
675       # and not the rest of the platform stat call fluff and stuff.
676       tempfile.chmod(effective_mode & 07777)
677     end
678 
679     # Now, make sure the data (which includes the mode) is safe on disk.
680     tempfile.flush
681     begin
682       tempfile.fsync
683     rescue NotImplementedError
684       # fsync may not be implemented by Ruby on all platforms, but
685       # there is absolutely no recovery path if we detect that.  So, we just
686       # ignore the return code.
687       #
688       # However, don't be fooled: that is accepting that we are running in
689       # an unsafe fashion.  If you are porting to a new platform don't stub
690       # that out.
691     end
692 
693     tempfile.close
694 
695     if validate_callback
696       validate_callback.call(tempfile.path)
697     end
698 
699     if Puppet::Util::Platform.windows?
700       # Windows ReplaceFile needs a file to exist, so touch handles this
701       if !Puppet::FileSystem.exist?(file)
702         Puppet::FileSystem.touch(file)
703         if mode
704           Puppet::Util::Windows::Security.set_mode(mode, Puppet::FileSystem.path_string(file))
705         end
706       end
707       # Yes, the arguments are reversed compared to the rename in the rest
708       # of the world.
709       Puppet::Util::Windows::File.replace_file(FileSystem.path_string(file), tempfile.path)
710 
711     else
712       # MRI Ruby checks for this and raises an error, while JRuby removes the directory
713       # and replaces it with a file. This makes the our version of replace_file() consistent
714       if Puppet::FileSystem.exist?(file) && Puppet::FileSystem.directory?(file)
715         raise Errno::EISDIR, _("Is a directory: %{directory}") % { directory: file }
716       end
717       File.rename(tempfile.path, Puppet::FileSystem.path_string(file))
718     end
719   ensure
720     # in case an error occurred before we renamed the temp file, make sure it
721     # gets deleted
722     if tempfile
723       tempfile.close!
724     end
725   end
726 
727 
728   # Ideally, we would now fsync the directory as well, but Ruby doesn't
729   # have support for that, and it doesn't matter /that/ much...
730 
731   # Return something true, and possibly useful.
732   file
733 end
resolve_stackframe(frame) click to toggle source
    # File lib/puppet/util.rb
589 def self.resolve_stackframe(frame)
590   _, path, rest = /^(.*):(\d+.*)$/.match(frame).to_a
591   if path
592     path = Pathname(path).realpath rescue path
593     "#{path}:#{rest}"
594   else
595     frame
596   end
597 end
rfc2396_escape(str) click to toggle source
    # File lib/puppet/util.rb
498 def rfc2396_escape(str)
499   str.gsub(UNSAFE) do |match|
500     tmp = ''
501     match.each_byte do |uc|
502       tmp << sprintf('%%%02X', uc)
503     end
504     tmp
505   end.force_encoding(Encoding::US_ASCII)
506 end
safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block) click to toggle source
    # File lib/puppet/util.rb
516 def safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block)
517   child_pid = Kernel.fork do
518     STDIN.reopen(stdin)
519     STDOUT.reopen(stdout)
520     STDERR.reopen(stderr)
521 
522     $stdin = STDIN
523     $stdout = STDOUT
524     $stderr = STDERR
525 
526     begin
527       Dir.foreach('/proc/self/fd') do |f|
528         if f != '.' && f != '..' && f.to_i >= 3
529           IO::new(f.to_i).close rescue nil
530         end
531       end
532     rescue Errno::ENOENT # /proc/self/fd not found
533       3.upto(256){|fd| IO::new(fd).close rescue nil}
534     end
535 
536     block.call if block
537   end
538   child_pid
539 end
set_env(name, value = nil, mode = default_env) click to toggle source

@param name [String] The name of the environment variable to set @param value [String] The value to set the variable to. nil deletes the environment variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

   # File lib/puppet/util.rb
90 def set_env(name, value = nil, mode = default_env)
91   case mode
92     when :posix
93       ENV[name] = value
94     when :windows
95       Puppet::Util::Windows::Process.set_environment_variable(name,value)
96     else
97       raise _("Unable to set the environment variable %{name} for mode %{mode}") % { name: name, mode: mode }
98   end
99 end
skip_external_facts() { || ... } click to toggle source

Executes a block of code, wrapped around Facter.load_external(false) and Facter.load_external(true) which will cause Facter to not evaluate external facts.

    # File lib/puppet/util.rb
774 def skip_external_facts
775   return yield unless Facter.respond_to? :load_external
776   begin
777     Facter.load_external(false)
778     yield
779   ensure
780     Facter.load_external(true)
781   end
782 end
symbolizehash(hash) click to toggle source
    # File lib/puppet/util.rb
542 def symbolizehash(hash)
543   newhash = {}
544   hash.each do |name, val|
545     name = name.intern if name.respond_to? :intern
546     newhash[name] = val
547   end
548   newhash
549 end
thinmark() { || ... } click to toggle source

Just benchmark, with no logging.

    # File lib/puppet/util.rb
553 def thinmark
554   seconds = Benchmark.realtime {
555     yield
556   }
557 
558   seconds
559 end
uri_encode(path, opts = { :allow_fragment => false }) click to toggle source

Percent-encodes a URI string per RFC3986 - tools.ietf.org/html/rfc3986

Properly handles escaping rules for paths, query strings and fragments independently

The output is safe to pass to URI.parse or URI::Generic.build and will correctly round-trip through URI.unescape

@param [String path] A URI string that may be in the form of:

http://foo.com/bar?query
file://tmp/foo bar
//foo.com/bar?query
/bar?query
bar?query
bar
.
C:\Windows\Temp

Note that with no specified scheme, authority or query parameter delimiter
? that a naked string will be treated as a path.

Note that if query parameters need to contain data such as & or =
that this method should not be used, as there is no way to differentiate
query parameter data from query delimiters when multiple parameters
are specified

@param [Hash{Symbol=>String} opts] Options to alter encoding @option opts [Array<Symbol>] :allow_fragment defaults to false. When false

will treat # as part of a path or query and not a fragment delimiter

@return [String] a new string containing appropriate portions of the URI

encoded per the rules of RFC3986.
In particular,
path will not encode +, but will encode space as %20
query will encode + as %2B and space as %20
fragment behaves like query
    # File lib/puppet/util.rb
452 def uri_encode(path, opts = { :allow_fragment => false })
453   raise ArgumentError.new(_('path may not be nil')) if path.nil?
454 
455   # ensure string starts as UTF-8 for the sake of Ruby 1.9.3
456   encoded = ''.encode!(Encoding::UTF_8)
457 
458   # parse uri into named matches, then reassemble properly encoded
459   parts = path.match(RFC_3986_URI_REGEX)
460 
461   encoded += parts[:scheme] unless parts[:scheme].nil?
462   encoded += parts[:authority] unless parts[:authority].nil?
463 
464   # path requires space to be encoded as %20 (NEVER +)
465   # + should be left unencoded
466   # URI::parse and URI::Generic.build don't like paths encoded with CGI.escape
467   # URI.escape does not change / to %2F and : to %3A like CGI.escape
468   #
469   encoded += rfc2396_escape(parts[:path]) unless parts[:path].nil?
470 
471   # each query parameter
472   if !parts[:query].nil?
473     query_string = parts[:query].split('&').map do |pair|
474       # can optionally be separated by an =
475       pair.split('=').map do |v|
476         uri_query_encode(v)
477       end.join('=')
478     end.join('&')
479     encoded += '?' + query_string
480   end
481 
482   encoded += ((opts[:allow_fragment] ? '#' : '%23') + uri_query_encode(parts[:fragment])) unless parts[:fragment].nil?
483 
484   encoded
485 end
uri_query_encode(query_string) click to toggle source

Percent-encodes a URI query parameter per RFC3986 - tools.ietf.org/html/rfc3986

The output will correctly round-trip through URI.unescape

@param [String query_string] A URI query parameter that may contain reserved

characters that must be percent encoded for the key or value to be
properly decoded as part of a larger query string:

query
encodes as : query

query_with_special=chars like&and * and# plus+this
encodes as:
query_with_special%3Dchars%20like%26and%20%2A%20and%23%20plus%2Bthis

Note: Also usable by fragments, but not suitable for paths

@return [String] a new string containing an encoded query string per the

rules of RFC3986.

In particular,
query will encode + as %2B and space as %20
    # File lib/puppet/util.rb
402 def uri_query_encode(query_string)
403   return nil if query_string.nil?
404 
405   # query can encode space to %20 OR +
406   # + MUST be encoded as %2B
407   # in RFC3968 both query and fragment are defined as:
408   # = *( pchar / "/" / "?" )
409   # CGI.escape turns space into + which is the most backward compatible
410   # however it doesn't roundtrip through URI.unescape which prefers %20
411   CGI.escape(query_string).gsub('+', '%20')
412 end
uri_to_path(uri) click to toggle source

Get the path component of a URI

    # File lib/puppet/util.rb
359 def uri_to_path(uri)
360   return unless uri.is_a?(URI)
361 
362   # CGI.unescape doesn't handle space rules properly in uri paths
363   # URI.unescape does, but returns strings in their original encoding
364   path = uri_unescape(uri.path.encode(Encoding::UTF_8))
365 
366   if Puppet::Util::Platform.windows? && uri.scheme == 'file'
367     if uri.host && !uri.host.empty?
368       path = "//#{uri.host}" + path # UNC
369     else
370       path.sub!(/^\//, '')
371     end
372   end
373 
374   path
375 end
uri_unescape(str) click to toggle source
    # File lib/puppet/util.rb
509 def uri_unescape(str)
510   enc = str.encoding
511   enc = Encoding::UTF_8 if enc == Encoding::US_ASCII
512   str.gsub(ESCAPED) { [$&[1, 2]].pack('H2').force_encoding(enc) }
513 end
which(bin) click to toggle source

Resolve a path for an executable to the absolute path. This tries to behave in the same manner as the unix `which` command and uses the `PATH` environment variable.

@api public @param bin [String] the name of the executable to find. @return [String] the absolute path to the found executable.

    # File lib/puppet/util.rb
250 def which(bin)
251   if absolute_path?(bin)
252     return bin if FileTest.file? bin and FileTest.executable? bin
253   else
254     exts = Puppet::Util.get_env('PATHEXT')
255     exts = exts ? exts.split(File::PATH_SEPARATOR) : %w[.COM .EXE .BAT .CMD]
256     Puppet::Util.get_env('PATH').split(File::PATH_SEPARATOR).each do |dir|
257       begin
258         dest = File.expand_path(File.join(dir, bin))
259       rescue ArgumentError => e
260         # if the user's PATH contains a literal tilde (~) character and HOME is not set, we may get
261         # an ArgumentError here.  Let's check to see if that is the case; if not, re-raise whatever error
262         # was thrown.
263         if e.to_s =~ /HOME/ and (Puppet::Util.get_env('HOME').nil? || Puppet::Util.get_env('HOME') == "")
264           # if we get here they have a tilde in their PATH.  We'll issue a single warning about this and then
265           # ignore this path element and carry on with our lives.
266           #TRANSLATORS PATH and HOME are environment variables and should not be translated
267           Puppet::Util::Warnings.warnonce(_("PATH contains a ~ character, and HOME is not set; ignoring PATH element '%{dir}'.") % { dir: dir })
268         elsif e.to_s =~ /doesn't exist|can't find user/
269           # ...otherwise, we just skip the non-existent entry, and do nothing.
270           #TRANSLATORS PATH is an environment variable and should not be translated
271           Puppet::Util::Warnings.warnonce(_("Couldn't expand PATH containing a ~ character; ignoring PATH element '%{dir}'.") % { dir: dir })
272         else
273           raise
274         end
275       else
276         if Puppet::Util::Platform.windows? && File.extname(dest).empty?
277           exts.each do |ext|
278             destext = File.expand_path(dest + ext)
279             return destext if FileTest.file? destext and FileTest.executable? destext
280           end
281         end
282         return dest if FileTest.file? dest and FileTest.executable? dest
283       end
284     end
285   end
286   nil
287 end
withenv(hash, mode = :posix) { || ... } click to toggle source

Run some code with a specific environment. Resets the environment back to what it was at the end of the code. Windows can store Unicode chars in the environment as keys or values, but Ruby's ENV tries to roundtrip them through the local codepage, which can cause encoding problems - underlying helpers use Windows APIs on Windows see bugs.ruby-lang.org/issues/8822

    # File lib/puppet/util.rb
125 def withenv(hash, mode = :posix)
126   saved = get_environment(mode)
127   merge_environment(hash, mode)
128   yield
129 ensure
130   if saved
131     clear_environment(mode)
132     merge_environment(saved, mode)
133   end
134 end
withumask(mask) { || ... } click to toggle source

Execute a given chunk of code with a new umask.

    # File lib/puppet/util.rb
138 def self.withumask(mask)
139   cur = File.umask(mask)
140 
141   begin
142     yield
143   ensure
144     File.umask(cur)
145   end
146 end

Private Instance Methods

absolute_path?(path, platform=nil) click to toggle source
    # File lib/puppet/util.rb
298 def absolute_path?(path, platform=nil)
299   unless path.is_a?(String)
300     Puppet.warning("Cannot check if #{path} is an absolute path because it is a '#{path.class}' and not a String'")
301     return false
302   end
303 
304   # Ruby only sets File::ALT_SEPARATOR on Windows and the Ruby standard
305   # library uses that to test what platform it's on.  Normally in Puppet we
306   # would use Puppet.features.microsoft_windows?, but this method needs to
307   # be called during the initialization of features so it can't depend on
308   # that.
309   #
310   # @deprecated Use ruby's built-in methods to determine if a path is absolute.
311   platform ||= Puppet::Util::Platform.windows? ? :windows : :posix
312   regex = case platform
313           when :windows
314             AbsolutePathWindows
315           when :posix
316             AbsolutePathPosix
317           else
318             raise Puppet::DevError, _("unknown platform %{platform} in absolute_path") % { platform: platform }
319           end
320 
321   !! (path =~ regex)
322 end
benchmark(*args) { || ... } click to toggle source

execute a block of work and based on the logging level provided, log the provided message with the seconds taken The message 'msg' should include string ' in %{seconds} seconds' as part of the message and any content should escape any percent signs '%' so that they are not interpreted as formatting commands

escaped_str = str.gsub(/%/, '%%')

@param msg [String] the message to be formated to assigned the %{seconds} seconds take to execute,

other percent signs '%' need to be escaped

@param level [Symbol] the logging level for this message @param object [Object] The object use for logging the message

    # File lib/puppet/util.rb
210 def benchmark(*args)
211   msg = args.pop
212   level = args.pop
213   object = if args.empty?
214              if respond_to?(level)
215                self
216              else
217                Puppet
218              end
219            else
220              args.pop
221            end
222 
223   #TRANSLATORS 'benchmark' is a method name and should not be translated
224   raise Puppet::DevError, _("Failed to provide level to benchmark") unless level
225 
226   unless level == :none or object.respond_to? level
227     raise Puppet::DevError, _("Benchmarked object does not respond to %{value}") % { value: level }
228   end
229 
230   # Only benchmark if our log level is high enough
231   if level != :none and Puppet::Util::Log.sendlevel?(level)
232     seconds = Benchmark.realtime {
233       yield
234     }
235     object.send(level, msg % { seconds: "%0.2f" % seconds })
236     return seconds
237   else
238     yield
239   end
240 end
clear_environment(mode = default_env) click to toggle source

Removes all environment variables @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

   # File lib/puppet/util.rb
72 def clear_environment(mode = default_env)
73   case mode
74     when :posix
75       ENV.clear
76     when :windows
77       Puppet::Util::Windows::Process.get_environment_strings.each do |key, _|
78         Puppet::Util::Windows::Process.set_environment_variable(key, nil)
79       end
80     else
81       raise _("Unable to clear the environment for mode %{mode}") % { mode: mode }
82   end
83 end
default_env() click to toggle source
   # File lib/puppet/util.rb
29 def default_env
30   Puppet.features.microsoft_windows? ?
31     :windows :
32     :posix
33 end
deterministic_rand(seed,max) click to toggle source
    # File lib/puppet/util.rb
762 def deterministic_rand(seed,max)
763   deterministic_rand_int(seed, max).to_s
764 end
deterministic_rand_int(seed,max) click to toggle source
    # File lib/puppet/util.rb
767 def deterministic_rand_int(seed,max)
768   Random.new(seed).rand(max)
769 end
exit_on_fail(message, code = 1) { || ... } click to toggle source

Executes a block of code, wrapped with some special exception handling. Causes the ruby interpreter to

exit if the block throws an exception.

@api public @param [String] message a message to log if the block fails @param [Integer] code the exit code that the ruby interpreter should return if the block fails @yield

    # File lib/puppet/util.rb
743 def exit_on_fail(message, code = 1)
744   yield
745 # First, we need to check and see if we are catching a SystemExit error.  These will be raised
746 #  when we daemonize/fork, and they do not necessarily indicate a failure case.
747 rescue SystemExit => err
748   raise err
749 
750 # Now we need to catch *any* other kind of exception, because we may be calling third-party
751 #  code (e.g. webrick), and we have no idea what they might throw.
752 rescue Exception => err
753   ## NOTE: when debugging spec failures, these two lines can be very useful
754   #puts err.inspect
755   #puts Puppet::Util.pretty_backtrace(err.backtrace)
756   Puppet.log_exception(err, "#{message}: #{err}")
757   Puppet::Util::Log.force_flushqueue()
758   exit(code)
759 end
get_env(name, mode = default_env) click to toggle source

@param name [String] The name of the environment variable to retrieve @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [String] Value of the specified environment variable. nil if it does not exist @api private

   # File lib/puppet/util.rb
40 def get_env(name, mode = default_env)
41   if mode == :windows
42     Puppet::Util::Windows::Process.get_environment_strings.each do |key, value |
43       if name.casecmp(key) == 0 then
44         return value
45       end
46     end
47     return nil
48   else
49     ENV[name]
50   end
51 end
get_environment(mode = default_env) click to toggle source

@param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @return [Hash] A hashtable of all environment variables @api private

   # File lib/puppet/util.rb
57 def get_environment(mode = default_env)
58   case mode
59     when :posix
60       ENV.to_hash
61     when :windows
62       Puppet::Util::Windows::Process.get_environment_strings
63     else
64       raise _("Unable to retrieve the environment for mode %{mode}") % { mode: mode }
65   end
66 end
merge_environment(env_hash, mode = default_env) click to toggle source

@param name [Hash] Environment variables to merge into the existing environment. nil values will remove the variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

    # File lib/puppet/util.rb
105 def merge_environment(env_hash, mode = default_env)
106   case mode
107     when :posix
108       env_hash.each { |name, val| ENV[name.to_s] = val }
109     when :windows
110       env_hash.each do |name, val|
111         Puppet::Util::Windows::Process.set_environment_variable(name.to_s, val)
112       end
113     else
114       raise _("Unable to merge given values into the current environment for mode %{mode}") % { mode: mode }
115   end
116 end
path_to_uri(path) click to toggle source

Convert a path to a file URI

    # File lib/puppet/util.rb
326 def path_to_uri(path)
327   return unless path
328 
329   params = { :scheme => 'file' }
330 
331   if Puppet::Util::Platform.windows?
332     path = path.tr('\\', '/')
333 
334     unc = /^\/\/([^\/]+)(\/.+)/.match(path)
335     if unc
336       params[:host] = unc[1]
337       path = unc[2]
338     elsif path =~ /^[a-z]:\//i
339       path = '/' + path
340     end
341   end
342 
343   # have to split *after* any relevant escaping
344   params[:path], params[:query] = uri_encode(path).split('?')
345   search_for_fragment = params[:query] ? :query : :path
346   if params[search_for_fragment].include?('#')
347     params[search_for_fragment], _, params[:fragment] = params[search_for_fragment].rpartition('#')
348   end
349 
350   begin
351     URI::Generic.build(params)
352   rescue => detail
353     raise Puppet::Error, _("Failed to convert '%{path}' to URI: %{detail}") % { path: path, detail: detail }, detail.backtrace
354   end
355 end
replace_file(file, default_mode, staging_location: nil, validate_callback: nil) { |tempfile| ... } click to toggle source
    # File lib/puppet/util.rb
629 def replace_file(file, default_mode, staging_location: nil, validate_callback: nil, &block)
630   raise Puppet::DevError, _("replace_file requires a block") unless block_given?
631 
632   if default_mode
633     unless valid_symbolic_mode?(default_mode)
634       raise Puppet::DevError, _("replace_file default_mode: %{default_mode} is invalid") % { default_mode: default_mode }
635     end
636 
637     mode = symbolic_mode_to_int(normalize_symbolic_mode(default_mode))
638   else
639     if Puppet::Util::Platform.windows?
640       mode = DEFAULT_WINDOWS_MODE
641     else
642       mode = DEFAULT_POSIX_MODE
643     end
644   end
645 
646   begin
647     file = Puppet::FileSystem.pathname(file)
648 
649     # encoding for Uniquefile is not important here because the caller writes to it as it sees fit
650     if staging_location
651       tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), staging_location)
652     else
653       tempfile = Puppet::FileSystem::Uniquefile.new(Puppet::FileSystem.basename_string(file), Puppet::FileSystem.dir_string(file))
654     end
655 
656 
657     effective_mode =
658     if !Puppet::Util::Platform.windows?
659       # Grab the current file mode, and fall back to the defaults.
660 
661       if Puppet::FileSystem.exist?(file)
662         stat = Puppet::FileSystem.lstat(file)
663         tempfile.chown(stat.uid, stat.gid)
664         stat.mode
665       else
666         mode
667       end
668     end
669 
670     # OK, now allow the caller to write the content of the file.
671     yield tempfile
672 
673     if effective_mode
674       # We only care about the bottom four slots, which make the real mode,
675       # and not the rest of the platform stat call fluff and stuff.
676       tempfile.chmod(effective_mode & 07777)
677     end
678 
679     # Now, make sure the data (which includes the mode) is safe on disk.
680     tempfile.flush
681     begin
682       tempfile.fsync
683     rescue NotImplementedError
684       # fsync may not be implemented by Ruby on all platforms, but
685       # there is absolutely no recovery path if we detect that.  So, we just
686       # ignore the return code.
687       #
688       # However, don't be fooled: that is accepting that we are running in
689       # an unsafe fashion.  If you are porting to a new platform don't stub
690       # that out.
691     end
692 
693     tempfile.close
694 
695     if validate_callback
696       validate_callback.call(tempfile.path)
697     end
698 
699     if Puppet::Util::Platform.windows?
700       # Windows ReplaceFile needs a file to exist, so touch handles this
701       if !Puppet::FileSystem.exist?(file)
702         Puppet::FileSystem.touch(file)
703         if mode
704           Puppet::Util::Windows::Security.set_mode(mode, Puppet::FileSystem.path_string(file))
705         end
706       end
707       # Yes, the arguments are reversed compared to the rename in the rest
708       # of the world.
709       Puppet::Util::Windows::File.replace_file(FileSystem.path_string(file), tempfile.path)
710 
711     else
712       # MRI Ruby checks for this and raises an error, while JRuby removes the directory
713       # and replaces it with a file. This makes the our version of replace_file() consistent
714       if Puppet::FileSystem.exist?(file) && Puppet::FileSystem.directory?(file)
715         raise Errno::EISDIR, _("Is a directory: %{directory}") % { directory: file }
716       end
717       File.rename(tempfile.path, Puppet::FileSystem.path_string(file))
718     end
719   ensure
720     # in case an error occurred before we renamed the temp file, make sure it
721     # gets deleted
722     if tempfile
723       tempfile.close!
724     end
725   end
726 
727 
728   # Ideally, we would now fsync the directory as well, but Ruby doesn't
729   # have support for that, and it doesn't matter /that/ much...
730 
731   # Return something true, and possibly useful.
732   file
733 end
rfc2396_escape(str) click to toggle source
    # File lib/puppet/util.rb
498 def rfc2396_escape(str)
499   str.gsub(UNSAFE) do |match|
500     tmp = ''
501     match.each_byte do |uc|
502       tmp << sprintf('%%%02X', uc)
503     end
504     tmp
505   end.force_encoding(Encoding::US_ASCII)
506 end
safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block) click to toggle source
    # File lib/puppet/util.rb
516 def safe_posix_fork(stdin=$stdin, stdout=$stdout, stderr=$stderr, &block)
517   child_pid = Kernel.fork do
518     STDIN.reopen(stdin)
519     STDOUT.reopen(stdout)
520     STDERR.reopen(stderr)
521 
522     $stdin = STDIN
523     $stdout = STDOUT
524     $stderr = STDERR
525 
526     begin
527       Dir.foreach('/proc/self/fd') do |f|
528         if f != '.' && f != '..' && f.to_i >= 3
529           IO::new(f.to_i).close rescue nil
530         end
531       end
532     rescue Errno::ENOENT # /proc/self/fd not found
533       3.upto(256){|fd| IO::new(fd).close rescue nil}
534     end
535 
536     block.call if block
537   end
538   child_pid
539 end
set_env(name, value = nil, mode = default_env) click to toggle source

@param name [String] The name of the environment variable to set @param value [String] The value to set the variable to. nil deletes the environment variable @param mode [Symbol] Which operating system mode to use e.g. :posix or :windows. Use nil to autodetect @api private

   # File lib/puppet/util.rb
90 def set_env(name, value = nil, mode = default_env)
91   case mode
92     when :posix
93       ENV[name] = value
94     when :windows
95       Puppet::Util::Windows::Process.set_environment_variable(name,value)
96     else
97       raise _("Unable to set the environment variable %{name} for mode %{mode}") % { name: name, mode: mode }
98   end
99 end
skip_external_facts() { || ... } click to toggle source

Executes a block of code, wrapped around Facter.load_external(false) and Facter.load_external(true) which will cause Facter to not evaluate external facts.

    # File lib/puppet/util.rb
774 def skip_external_facts
775   return yield unless Facter.respond_to? :load_external
776   begin
777     Facter.load_external(false)
778     yield
779   ensure
780     Facter.load_external(true)
781   end
782 end
symbolizehash(hash) click to toggle source
    # File lib/puppet/util.rb
542 def symbolizehash(hash)
543   newhash = {}
544   hash.each do |name, val|
545     name = name.intern if name.respond_to? :intern
546     newhash[name] = val
547   end
548   newhash
549 end
thinmark() { || ... } click to toggle source

Just benchmark, with no logging.

    # File lib/puppet/util.rb
553 def thinmark
554   seconds = Benchmark.realtime {
555     yield
556   }
557 
558   seconds
559 end
uri_encode(path, opts = { :allow_fragment => false }) click to toggle source

Percent-encodes a URI string per RFC3986 - tools.ietf.org/html/rfc3986

Properly handles escaping rules for paths, query strings and fragments independently

The output is safe to pass to URI.parse or URI::Generic.build and will correctly round-trip through URI.unescape

@param [String path] A URI string that may be in the form of:

http://foo.com/bar?query
file://tmp/foo bar
//foo.com/bar?query
/bar?query
bar?query
bar
.
C:\Windows\Temp

Note that with no specified scheme, authority or query parameter delimiter
? that a naked string will be treated as a path.

Note that if query parameters need to contain data such as & or =
that this method should not be used, as there is no way to differentiate
query parameter data from query delimiters when multiple parameters
are specified

@param [Hash{Symbol=>String} opts] Options to alter encoding @option opts [Array<Symbol>] :allow_fragment defaults to false. When false

will treat # as part of a path or query and not a fragment delimiter

@return [String] a new string containing appropriate portions of the URI

encoded per the rules of RFC3986.
In particular,
path will not encode +, but will encode space as %20
query will encode + as %2B and space as %20
fragment behaves like query
    # File lib/puppet/util.rb
452 def uri_encode(path, opts = { :allow_fragment => false })
453   raise ArgumentError.new(_('path may not be nil')) if path.nil?
454 
455   # ensure string starts as UTF-8 for the sake of Ruby 1.9.3
456   encoded = ''.encode!(Encoding::UTF_8)
457 
458   # parse uri into named matches, then reassemble properly encoded
459   parts = path.match(RFC_3986_URI_REGEX)
460 
461   encoded += parts[:scheme] unless parts[:scheme].nil?
462   encoded += parts[:authority] unless parts[:authority].nil?
463 
464   # path requires space to be encoded as %20 (NEVER +)
465   # + should be left unencoded
466   # URI::parse and URI::Generic.build don't like paths encoded with CGI.escape
467   # URI.escape does not change / to %2F and : to %3A like CGI.escape
468   #
469   encoded += rfc2396_escape(parts[:path]) unless parts[:path].nil?
470 
471   # each query parameter
472   if !parts[:query].nil?
473     query_string = parts[:query].split('&').map do |pair|
474       # can optionally be separated by an =
475       pair.split('=').map do |v|
476         uri_query_encode(v)
477       end.join('=')
478     end.join('&')
479     encoded += '?' + query_string
480   end
481 
482   encoded += ((opts[:allow_fragment] ? '#' : '%23') + uri_query_encode(parts[:fragment])) unless parts[:fragment].nil?
483 
484   encoded
485 end
uri_query_encode(query_string) click to toggle source

Percent-encodes a URI query parameter per RFC3986 - tools.ietf.org/html/rfc3986

The output will correctly round-trip through URI.unescape

@param [String query_string] A URI query parameter that may contain reserved

characters that must be percent encoded for the key or value to be
properly decoded as part of a larger query string:

query
encodes as : query

query_with_special=chars like&and * and# plus+this
encodes as:
query_with_special%3Dchars%20like%26and%20%2A%20and%23%20plus%2Bthis

Note: Also usable by fragments, but not suitable for paths

@return [String] a new string containing an encoded query string per the

rules of RFC3986.

In particular,
query will encode + as %2B and space as %20
    # File lib/puppet/util.rb
402 def uri_query_encode(query_string)
403   return nil if query_string.nil?
404 
405   # query can encode space to %20 OR +
406   # + MUST be encoded as %2B
407   # in RFC3968 both query and fragment are defined as:
408   # = *( pchar / "/" / "?" )
409   # CGI.escape turns space into + which is the most backward compatible
410   # however it doesn't roundtrip through URI.unescape which prefers %20
411   CGI.escape(query_string).gsub('+', '%20')
412 end
uri_to_path(uri) click to toggle source

Get the path component of a URI

    # File lib/puppet/util.rb
359 def uri_to_path(uri)
360   return unless uri.is_a?(URI)
361 
362   # CGI.unescape doesn't handle space rules properly in uri paths
363   # URI.unescape does, but returns strings in their original encoding
364   path = uri_unescape(uri.path.encode(Encoding::UTF_8))
365 
366   if Puppet::Util::Platform.windows? && uri.scheme == 'file'
367     if uri.host && !uri.host.empty?
368       path = "//#{uri.host}" + path # UNC
369     else
370       path.sub!(/^\//, '')
371     end
372   end
373 
374   path
375 end
uri_unescape(str) click to toggle source
    # File lib/puppet/util.rb
509 def uri_unescape(str)
510   enc = str.encoding
511   enc = Encoding::UTF_8 if enc == Encoding::US_ASCII
512   str.gsub(ESCAPED) { [$&[1, 2]].pack('H2').force_encoding(enc) }
513 end
which(bin) click to toggle source

Resolve a path for an executable to the absolute path. This tries to behave in the same manner as the unix `which` command and uses the `PATH` environment variable.

@api public @param bin [String] the name of the executable to find. @return [String] the absolute path to the found executable.

    # File lib/puppet/util.rb
250 def which(bin)
251   if absolute_path?(bin)
252     return bin if FileTest.file? bin and FileTest.executable? bin
253   else
254     exts = Puppet::Util.get_env('PATHEXT')
255     exts = exts ? exts.split(File::PATH_SEPARATOR) : %w[.COM .EXE .BAT .CMD]
256     Puppet::Util.get_env('PATH').split(File::PATH_SEPARATOR).each do |dir|
257       begin
258         dest = File.expand_path(File.join(dir, bin))
259       rescue ArgumentError => e
260         # if the user's PATH contains a literal tilde (~) character and HOME is not set, we may get
261         # an ArgumentError here.  Let's check to see if that is the case; if not, re-raise whatever error
262         # was thrown.
263         if e.to_s =~ /HOME/ and (Puppet::Util.get_env('HOME').nil? || Puppet::Util.get_env('HOME') == "")
264           # if we get here they have a tilde in their PATH.  We'll issue a single warning about this and then
265           # ignore this path element and carry on with our lives.
266           #TRANSLATORS PATH and HOME are environment variables and should not be translated
267           Puppet::Util::Warnings.warnonce(_("PATH contains a ~ character, and HOME is not set; ignoring PATH element '%{dir}'.") % { dir: dir })
268         elsif e.to_s =~ /doesn't exist|can't find user/
269           # ...otherwise, we just skip the non-existent entry, and do nothing.
270           #TRANSLATORS PATH is an environment variable and should not be translated
271           Puppet::Util::Warnings.warnonce(_("Couldn't expand PATH containing a ~ character; ignoring PATH element '%{dir}'.") % { dir: dir })
272         else
273           raise
274         end
275       else
276         if Puppet::Util::Platform.windows? && File.extname(dest).empty?
277           exts.each do |ext|
278             destext = File.expand_path(dest + ext)
279             return destext if FileTest.file? destext and FileTest.executable? destext
280           end
281         end
282         return dest if FileTest.file? dest and FileTest.executable? dest
283       end
284     end
285   end
286   nil
287 end
withenv(hash, mode = :posix) { || ... } click to toggle source

Run some code with a specific environment. Resets the environment back to what it was at the end of the code. Windows can store Unicode chars in the environment as keys or values, but Ruby's ENV tries to roundtrip them through the local codepage, which can cause encoding problems - underlying helpers use Windows APIs on Windows see bugs.ruby-lang.org/issues/8822

    # File lib/puppet/util.rb
125 def withenv(hash, mode = :posix)
126   saved = get_environment(mode)
127   merge_environment(hash, mode)
128   yield
129 ensure
130   if saved
131     clear_environment(mode)
132     merge_environment(saved, mode)
133   end
134 end