module Puppet::Util::Execution

This module defines methods for execution of system commands. It is intended for inclusion in classes that needs to execute system commands. @api public

Constants

NoOptionsSpecified

Default empty options for {execute}

Public Class Methods

execpipe(command, failonfail = true) { |pipe| ... } click to toggle source

The command can be a simple string, which is executed as-is, or an Array, which is treated as a set of command arguments to pass through.

In either case, the command is passed directly to the shell, STDOUT and STDERR are connected together, and STDOUT will be streamed to the yielded pipe.

@param command [String, Array<String>] the command to execute as one string,

or as parts in an array. The parts of the array are joined with one
separating space between each entry when converting to the command line
string to execute.

@param failonfail [Boolean] (true) if the execution should fail with

Exception on failure or not.

@yield [pipe] to a block executing a subprocess @yieldparam pipe [IO] the opened pipe @yieldreturn [String] the output to return @raise [Puppet::ExecutionFailure] if the executed child process did not

exit with status == 0 and `failonfail` is `true`.

@return [String] a string with the output from the subprocess executed by

the given block

@see Kernel#open for `mode` values @api public

   # File lib/puppet/util/execution.rb
58 def self.execpipe(command, failonfail = true)
59   # Paste together an array with spaces.  We used to paste directly
60   # together, no spaces, which made for odd invocations; the user had to
61   # include whitespace between arguments.
62   #
63   # Having two spaces is really not a big drama, since this passes to the
64   # shell anyhow, while no spaces makes for a small developer cost every
65   # time this is invoked. --daniel 2012-02-13
66   command_str = command.respond_to?(:join) ? command.join(' ') : command
67 
68   if respond_to? :debug
69     debug "Executing '#{command_str}'"
70   else
71     Puppet.debug { "Executing '#{command_str}'" }
72   end
73 
74   # force the run of the command with
75   # the user/system locale to "C" (via environment variables LANG and LC_*)
76   # it enables to have non localized output for some commands and therefore
77   # a predictable output
78   english_env = ENV.to_hash.merge( {'LANG' => 'C', 'LC_ALL' => 'C'} )
79   output = Puppet::Util.withenv(english_env) do
80     open("| #{command_str} 2>&1") do |pipe|
81       yield pipe
82     end
83   end
84 
85   if failonfail && exitstatus != 0
86     raise Puppet::ExecutionFailure, output.to_s
87   end
88 
89   output
90 end
execute(command, options = NoOptionsSpecified) click to toggle source

Executes the desired command, and return the status and output. def execute(command, options) @param command [Array<String>, String] the command to execute. If it is

an Array the first element should be the executable and the rest of the
elements should be the individual arguments to that executable.

@param options [Hash] a Hash of options @option options [String] :cwd the directory from which to run the command. Raises an error if the directory does not exist.

This option is only available on the agent. It cannot be used on the master, meaning it cannot be used in, for example,
regular functions, hiera backends, or report processors.

@option options [Boolean] :failonfail if this value is set to true, then this method will raise an error if the

command is not executed successfully.

@option options [Integer, String] :uid (nil) the user id of the user that the process should be run as. Will be ignored if the

user id matches the effective user id of the current process.

@option options [Integer, String] :gid (nil) the group id of the group that the process should be run as. Will be ignored if the

group id matches the effective group id of the current process.

@option options [Boolean] :combine sets whether or not to combine stdout/stderr in the output, if false stderr output is discarded @option options [String] :stdinfile (nil) sets a file that can be used for stdin. Passing a string for stdin is not currently

supported.

@option options [Boolean] :squelch (false) if true, ignore stdout / stderr completely. @option options [Boolean] :override_locale (true) by default (and if this option is set to true), we will temporarily override

the user/system locale to "C" (via environment variables LANG and LC_*) while we are executing the command.
This ensures that the output of the command will be formatted consistently, making it predictable for parsing.
Passing in a value of false for this option will allow the command to be executed using the user/system locale.

@option options [Hash<{String => String}>] :custom_environment ({}) a hash of key/value pairs to set as environment variables for the duration

of the command.

@return [Puppet::Util::Execution::ProcessOutput] output as specified by options @raise [Puppet::ExecutionFailure] if the executed chiled process did not exit with status == 0 and `failonfail` is

`true`.

@note Unfortunately, the default behavior for failonfail and combine (since

0.22.4 and 0.24.7, respectively) depend on whether options are specified
or not. If specified, then failonfail and combine default to false (even
when the options specified are neither failonfail nor combine). If no
options are specified, then failonfail and combine default to true.

@comment See commits efe9a833c and d32d7f30 @api public

    # File lib/puppet/util/execution.rb
136 def self.execute(command, options = NoOptionsSpecified)
137   # specifying these here rather than in the method signature to allow callers to pass in a partial
138   # set of overrides without affecting the default values for options that they don't pass in
139   default_options = {
140       :failonfail => NoOptionsSpecified.equal?(options),
141       :uid => nil,
142       :gid => nil,
143       :combine => NoOptionsSpecified.equal?(options),
144       :stdinfile => nil,
145       :squelch => false,
146       :override_locale => true,
147       :custom_environment => {},
148       :sensitive => false,
149       :suppress_window => false,
150   }
151 
152   options = default_options.merge(options)
153 
154   if command.is_a?(Array)
155     command = command.flatten.map(&:to_s)
156     command_str = command.join(" ")
157   elsif command.is_a?(String)
158     command_str = command
159   end
160 
161   # do this after processing 'command' array or string
162   command_str = '[redacted]' if options[:sensitive]
163 
164   user_log_s = ''
165   if options[:uid]
166     user_log_s << " uid=#{options[:uid]}"
167   end
168   if options[:gid]
169     user_log_s << " gid=#{options[:gid]}"
170   end
171   if user_log_s != ''
172     user_log_s.prepend(' with')
173   end
174 
175   if respond_to? :debug
176     debug "Executing#{user_log_s}: '#{command_str}'"
177   else
178     Puppet.debug { "Executing#{user_log_s}: '#{command_str}'" }
179   end
180 
181   null_file = Puppet::Util::Platform.windows? ? 'NUL' : '/dev/null'
182 
183   cwd = options[:cwd]
184   if cwd && ! Puppet::FileSystem.directory?(cwd)
185     raise ArgumentError, _("Working directory %{cwd} does not exist!") % { cwd: cwd }
186   end
187 
188   begin
189     stdin = Puppet::FileSystem.open(options[:stdinfile] || null_file, nil, 'r')
190     # On Windows, continue to use the file-based approach to avoid breaking people's existing
191     # manifests. If they use a script that doesn't background cleanly, such as
192     # `start /b ping 127.0.0.1`, we couldn't handle it with pipes as there's no non-blocking
193     # read available.
194     if options[:squelch]
195       stdout = Puppet::FileSystem.open(null_file, nil, 'w')
196     elsif Puppet.features.posix?
197       reader, stdout = IO.pipe
198     else
199       stdout = Puppet::FileSystem::Uniquefile.new('puppet')
200     end
201     stderr = options[:combine] ? stdout : Puppet::FileSystem.open(null_file, nil, 'w')
202 
203     exec_args = [command, options, stdin, stdout, stderr]
204     output = ''
205 
206     # We close stdin/stdout/stderr immediately after fork/exec as they're no longer needed by
207     # this process. In most cases they could be closed later, but when `stdout` is the "writer"
208     # pipe we must close it or we'll never reach eof on the `reader` pipe.
209     execution_stub = Puppet::Util::ExecutionStub.current_value
210     if execution_stub
211       child_pid = execution_stub.call(*exec_args)
212       [stdin, stdout, stderr].each {|io| io.close rescue nil}
213       return child_pid
214     elsif Puppet.features.posix?
215       child_pid = nil
216       begin
217         child_pid = execute_posix(*exec_args)
218         [stdin, stdout, stderr].each {|io| io.close rescue nil}
219         if options[:squelch]
220           exit_status = Process.waitpid2(child_pid).last.exitstatus
221         else
222           # Use non-blocking read to check for data. After each attempt,
223           # check whether the child is done. This is done in case the child
224           # forks and inherits stdout, as happens in `foo &`.
225           
226           until results = Process.waitpid2(child_pid, Process::WNOHANG) #rubocop:disable Lint/AssignmentInCondition
227 
228             # If not done, wait for data to read with a timeout
229             # This timeout is selected to keep activity low while waiting on
230             # a long process, while not waiting too long for the pathological
231             # case where stdout is never closed.
232             ready = IO.select([reader], [], [], 0.1)
233             begin
234               output << reader.read_nonblock(4096) if ready
235             rescue Errno::EAGAIN
236             rescue EOFError
237             end
238           end
239 
240           # Read any remaining data. Allow for but don't expect EOF.
241           begin
242             loop do
243               output << reader.read_nonblock(4096)
244             end
245           rescue Errno::EAGAIN
246           rescue EOFError
247           end
248 
249           # Force to external encoding to preserve prior behavior when reading a file.
250           # Wait until after reading all data so we don't encounter corruption when
251           # reading part of a multi-byte unicode character if default_external is UTF-8.
252           output.force_encoding(Encoding.default_external)
253           exit_status = results.last.exitstatus
254         end
255         child_pid = nil
256       rescue Timeout::Error => e
257         # NOTE: For Ruby 2.1+, an explicit Timeout::Error class has to be
258         # passed to Timeout.timeout in order for there to be something for
259         # this block to rescue.
260         unless child_pid.nil?
261           Process.kill(:TERM, child_pid)
262           # Spawn a thread to reap the process if it dies.
263           Thread.new { Process.waitpid(child_pid) }
264         end
265 
266         raise e
267       end
268     elsif Puppet::Util::Platform.windows?
269       process_info = execute_windows(*exec_args)
270       begin
271         [stdin, stderr].each {|io| io.close rescue nil}
272         exit_status = Puppet::Util::Windows::Process.wait_process(process_info.process_handle)
273 
274         # read output in if required
275         unless options[:squelch]
276           output = wait_for_output(stdout)
277           Puppet.warning _("Could not get output") unless output
278         end
279       ensure
280         FFI::WIN32.CloseHandle(process_info.process_handle)
281         FFI::WIN32.CloseHandle(process_info.thread_handle)
282       end
283     end
284 
285     if options[:failonfail] and exit_status != 0
286       raise Puppet::ExecutionFailure, _("Execution of '%{str}' returned %{exit_status}: %{output}") % { str: command_str, exit_status: exit_status, output: output.strip }
287     end
288   ensure
289     # Make sure all handles are closed in case an exception was thrown attempting to execute.
290     [stdin, stdout, stderr].each {|io| io.close rescue nil}
291     if !options[:squelch]
292       # if we opened a pipe, we need to clean it up.
293       reader.close if reader
294       stdout.close! if Puppet::Util::Platform.windows?
295     end
296   end
297 
298   Puppet::Util::Execution::ProcessOutput.new(output || '', exit_status)
299 end
Also aliased as: util_execute
ruby_path() click to toggle source

Returns the path to the ruby executable (available via Config object, even if it's not in the PATH… so this is slightly safer than just using Puppet::Util.which) @return [String] the path to the Ruby executable @api private

    # File lib/puppet/util/execution.rb
306 def self.ruby_path()
307   File.join(RbConfig::CONFIG['bindir'],
308             RbConfig::CONFIG['ruby_install_name'] + RbConfig::CONFIG['EXEEXT']).
309     sub(/.*\s.*/m, '"\&"')
310 end
util_execute(command, options = NoOptionsSpecified)
Alias for: execute

Private Class Methods

execute_posix(command, options, stdin, stdout, stderr) click to toggle source

This is private method. @comment see call to private_class_method after method definition @api private

    # File lib/puppet/util/execution.rb
322 def self.execute_posix(command, options, stdin, stdout, stderr)
323   child_pid = Puppet::Util.safe_posix_fork(stdin, stdout, stderr) do
324     # We can't just call Array(command), and rely on it returning
325     # things like ['foo'], when passed ['foo'], because
326     # Array(command) will call command.to_a internally, which when
327     # given a string can end up doing Very Bad Things(TM), such as
328     # turning "/tmp/foo;\r\n /bin/echo" into ["/tmp/foo;\r\n", " /bin/echo"]
329     command = [command].flatten
330     Process.setsid
331     begin
332       # We need to chdir to our cwd before changing privileges as there's a
333       # chance that the user may not have permissions to access the cwd, which
334       # would cause execute_posix to fail.
335       cwd = options[:cwd]
336       Dir.chdir(cwd) if cwd
337 
338       Puppet::Util::SUIDManager.change_privileges(options[:uid], options[:gid], true)
339 
340       # if the caller has requested that we override locale environment variables,
341       if (options[:override_locale]) then
342         # loop over them and clear them
343         Puppet::Util::POSIX::LOCALE_ENV_VARS.each { |name| ENV.delete(name) }
344         # set LANG and LC_ALL to 'C' so that the command will have consistent, predictable output
345         # it's OK to manipulate these directly rather than, e.g., via "withenv", because we are in
346         # a forked process.
347         ENV['LANG'] = 'C'
348         ENV['LC_ALL'] = 'C'
349       end
350 
351       # unset all of the user-related environment variables so that different methods of starting puppet
352       # (automatic start during boot, via 'service', via /etc/init.d, etc.) won't have unexpected side
353       # effects relating to user / home dir environment vars.
354       # it's OK to manipulate these directly rather than, e.g., via "withenv", because we are in
355       # a forked process.
356       Puppet::Util::POSIX::USER_ENV_VARS.each { |name| ENV.delete(name) }
357 
358       options[:custom_environment] ||= {}
359       Puppet::Util.withenv(options[:custom_environment]) do
360         Kernel.exec(*command)
361       end
362     rescue => detail
363       Puppet.log_exception(detail, _("Could not execute posix command: %{detail}") % { detail: detail })
364       exit!(1)
365     end
366   end
367   child_pid
368 end
execute_windows(command, options, stdin, stdout, stderr) click to toggle source

This is private method. @comment see call to private_class_method after method definition @api private

    # File lib/puppet/util/execution.rb
376 def self.execute_windows(command, options, stdin, stdout, stderr)
377   command = command.map do |part|
378     part.include?(' ') ? %Q["#{part.gsub(/"/, '\"')}"] : part
379   end.join(" ") if command.is_a?(Array)
380 
381   options[:custom_environment] ||= {}
382   Puppet::Util.withenv(options[:custom_environment], :windows) do
383     Puppet::Util::Windows::Process.execute(command, options, stdin, stdout, stderr)
384   end
385 end
exitstatus() click to toggle source
   # File lib/puppet/util/execution.rb
92 def self.exitstatus
93   $CHILD_STATUS.exitstatus
94 end
wait_for_output(stdout) click to toggle source

This is private method. @comment see call to private_class_method after method definition @api private

    # File lib/puppet/util/execution.rb
393 def self.wait_for_output(stdout)
394   # Make sure the file's actually been written.  This is basically a race
395   # condition, and is probably a horrible way to handle it, but, well, oh
396   # well.
397   # (If this method were treated as private / inaccessible from outside of this file, we shouldn't have to worry
398   #  about a race condition because all of the places that we call this from are preceded by a call to "waitpid2",
399   #  meaning that the processes responsible for writing the file have completed before we get here.)
400   2.times do |try|
401     if Puppet::FileSystem.exist?(stdout.path)
402       stdout.open
403       begin
404         return stdout.read
405       ensure
406         stdout.close
407         stdout.unlink
408       end
409     else
410       time_to_sleep = try / 2.0
411       Puppet.warning _("Waiting for output; will sleep %{time_to_sleep} seconds") % { time_to_sleep: time_to_sleep }
412       sleep(time_to_sleep)
413     end
414   end
415   nil
416 end