module Puppet::Util::Execution
This module defines methods for execution of system commands. It is intended for inclusion in classes that needs to execute system commands. @api public
Constants
- NoOptionsSpecified
Default empty options for {execute}
Public Class Methods
The command can be a simple string, which is executed as-is, or an Array, which is treated as a set of command arguments to pass through.
In either case, the command is passed directly to the shell, STDOUT and STDERR are connected together, and STDOUT will be streamed to the yielded pipe.
@param command [String, Array<String>] the command to execute as one string,
or as parts in an array. The parts of the array are joined with one separating space between each entry when converting to the command line string to execute.
@param failonfail [Boolean] (true) if the execution should fail with
Exception on failure or not.
@yield [pipe] to a block executing a subprocess @yieldparam pipe [IO] the opened pipe @yieldreturn [String] the output to return @raise [Puppet::ExecutionFailure] if the executed child process did not
exit with status == 0 and `failonfail` is `true`.
@return [String] a string with the output from the subprocess executed by
the given block
@see Kernel#open for `mode` values @api public
# File lib/puppet/util/execution.rb 58 def self.execpipe(command, failonfail = true) 59 # Paste together an array with spaces. We used to paste directly 60 # together, no spaces, which made for odd invocations; the user had to 61 # include whitespace between arguments. 62 # 63 # Having two spaces is really not a big drama, since this passes to the 64 # shell anyhow, while no spaces makes for a small developer cost every 65 # time this is invoked. --daniel 2012-02-13 66 command_str = command.respond_to?(:join) ? command.join(' ') : command 67 68 if respond_to? :debug 69 debug "Executing '#{command_str}'" 70 else 71 Puppet.debug { "Executing '#{command_str}'" } 72 end 73 74 # force the run of the command with 75 # the user/system locale to "C" (via environment variables LANG and LC_*) 76 # it enables to have non localized output for some commands and therefore 77 # a predictable output 78 english_env = ENV.to_hash.merge( {'LANG' => 'C', 'LC_ALL' => 'C'} ) 79 output = Puppet::Util.withenv(english_env) do 80 open("| #{command_str} 2>&1") do |pipe| 81 yield pipe 82 end 83 end 84 85 if failonfail && exitstatus != 0 86 raise Puppet::ExecutionFailure, output.to_s 87 end 88 89 output 90 end
Executes the desired command, and return the status and output. def execute(command, options) @param command [Array<String>, String] the command to execute. If it is
an Array the first element should be the executable and the rest of the elements should be the individual arguments to that executable.
@param options [Hash] a Hash of options @option options [String] :cwd the directory from which to run the command. Raises an error if the directory does not exist.
This option is only available on the agent. It cannot be used on the master, meaning it cannot be used in, for example, regular functions, hiera backends, or report processors.
@option options [Boolean] :failonfail if this value is set to true, then this method will raise an error if the
command is not executed successfully.
@option options [Integer, String] :uid (nil) the user id of the user that the process should be run as. Will be ignored if the
user id matches the effective user id of the current process.
@option options [Integer, String] :gid (nil) the group id of the group that the process should be run as. Will be ignored if the
group id matches the effective group id of the current process.
@option options [Boolean] :combine sets whether or not to combine stdout/stderr in the output, if false stderr output is discarded @option options [String] :stdinfile (nil) sets a file that can be used for stdin. Passing a string for stdin is not currently
supported.
@option options [Boolean] :squelch (false) if true, ignore stdout / stderr completely. @option options [Boolean] :override_locale (true) by default (and if this option is set to true), we will temporarily override
the user/system locale to "C" (via environment variables LANG and LC_*) while we are executing the command. This ensures that the output of the command will be formatted consistently, making it predictable for parsing. Passing in a value of false for this option will allow the command to be executed using the user/system locale.
@option options [Hash<{String => String}>] :custom_environment ({}) a hash of key/value pairs to set as environment variables for the duration
of the command.
@return [Puppet::Util::Execution::ProcessOutput] output as specified by options @raise [Puppet::ExecutionFailure] if the executed chiled process did not exit with status == 0 and `failonfail` is
`true`.
@note Unfortunately, the default behavior for failonfail and combine (since
0.22.4 and 0.24.7, respectively) depend on whether options are specified or not. If specified, then failonfail and combine default to false (even when the options specified are neither failonfail nor combine). If no options are specified, then failonfail and combine default to true.
@comment See commits efe9a833c and d32d7f30 @api public
# File lib/puppet/util/execution.rb 136 def self.execute(command, options = NoOptionsSpecified) 137 # specifying these here rather than in the method signature to allow callers to pass in a partial 138 # set of overrides without affecting the default values for options that they don't pass in 139 default_options = { 140 :failonfail => NoOptionsSpecified.equal?(options), 141 :uid => nil, 142 :gid => nil, 143 :combine => NoOptionsSpecified.equal?(options), 144 :stdinfile => nil, 145 :squelch => false, 146 :override_locale => true, 147 :custom_environment => {}, 148 :sensitive => false, 149 :suppress_window => false, 150 } 151 152 options = default_options.merge(options) 153 154 if command.is_a?(Array) 155 command = command.flatten.map(&:to_s) 156 command_str = command.join(" ") 157 elsif command.is_a?(String) 158 command_str = command 159 end 160 161 # do this after processing 'command' array or string 162 command_str = '[redacted]' if options[:sensitive] 163 164 user_log_s = '' 165 if options[:uid] 166 user_log_s << " uid=#{options[:uid]}" 167 end 168 if options[:gid] 169 user_log_s << " gid=#{options[:gid]}" 170 end 171 if user_log_s != '' 172 user_log_s.prepend(' with') 173 end 174 175 if respond_to? :debug 176 debug "Executing#{user_log_s}: '#{command_str}'" 177 else 178 Puppet.debug { "Executing#{user_log_s}: '#{command_str}'" } 179 end 180 181 null_file = Puppet::Util::Platform.windows? ? 'NUL' : '/dev/null' 182 183 cwd = options[:cwd] 184 if cwd && ! Puppet::FileSystem.directory?(cwd) 185 raise ArgumentError, _("Working directory %{cwd} does not exist!") % { cwd: cwd } 186 end 187 188 begin 189 stdin = Puppet::FileSystem.open(options[:stdinfile] || null_file, nil, 'r') 190 # On Windows, continue to use the file-based approach to avoid breaking people's existing 191 # manifests. If they use a script that doesn't background cleanly, such as 192 # `start /b ping 127.0.0.1`, we couldn't handle it with pipes as there's no non-blocking 193 # read available. 194 if options[:squelch] 195 stdout = Puppet::FileSystem.open(null_file, nil, 'w') 196 elsif Puppet.features.posix? 197 reader, stdout = IO.pipe 198 else 199 stdout = Puppet::FileSystem::Uniquefile.new('puppet') 200 end 201 stderr = options[:combine] ? stdout : Puppet::FileSystem.open(null_file, nil, 'w') 202 203 exec_args = [command, options, stdin, stdout, stderr] 204 output = '' 205 206 # We close stdin/stdout/stderr immediately after fork/exec as they're no longer needed by 207 # this process. In most cases they could be closed later, but when `stdout` is the "writer" 208 # pipe we must close it or we'll never reach eof on the `reader` pipe. 209 execution_stub = Puppet::Util::ExecutionStub.current_value 210 if execution_stub 211 child_pid = execution_stub.call(*exec_args) 212 [stdin, stdout, stderr].each {|io| io.close rescue nil} 213 return child_pid 214 elsif Puppet.features.posix? 215 child_pid = nil 216 begin 217 child_pid = execute_posix(*exec_args) 218 [stdin, stdout, stderr].each {|io| io.close rescue nil} 219 if options[:squelch] 220 exit_status = Process.waitpid2(child_pid).last.exitstatus 221 else 222 # Use non-blocking read to check for data. After each attempt, 223 # check whether the child is done. This is done in case the child 224 # forks and inherits stdout, as happens in `foo &`. 225 226 until results = Process.waitpid2(child_pid, Process::WNOHANG) #rubocop:disable Lint/AssignmentInCondition 227 228 # If not done, wait for data to read with a timeout 229 # This timeout is selected to keep activity low while waiting on 230 # a long process, while not waiting too long for the pathological 231 # case where stdout is never closed. 232 ready = IO.select([reader], [], [], 0.1) 233 begin 234 output << reader.read_nonblock(4096) if ready 235 rescue Errno::EAGAIN 236 rescue EOFError 237 end 238 end 239 240 # Read any remaining data. Allow for but don't expect EOF. 241 begin 242 loop do 243 output << reader.read_nonblock(4096) 244 end 245 rescue Errno::EAGAIN 246 rescue EOFError 247 end 248 249 # Force to external encoding to preserve prior behavior when reading a file. 250 # Wait until after reading all data so we don't encounter corruption when 251 # reading part of a multi-byte unicode character if default_external is UTF-8. 252 output.force_encoding(Encoding.default_external) 253 exit_status = results.last.exitstatus 254 end 255 child_pid = nil 256 rescue Timeout::Error => e 257 # NOTE: For Ruby 2.1+, an explicit Timeout::Error class has to be 258 # passed to Timeout.timeout in order for there to be something for 259 # this block to rescue. 260 unless child_pid.nil? 261 Process.kill(:TERM, child_pid) 262 # Spawn a thread to reap the process if it dies. 263 Thread.new { Process.waitpid(child_pid) } 264 end 265 266 raise e 267 end 268 elsif Puppet::Util::Platform.windows? 269 process_info = execute_windows(*exec_args) 270 begin 271 [stdin, stderr].each {|io| io.close rescue nil} 272 exit_status = Puppet::Util::Windows::Process.wait_process(process_info.process_handle) 273 274 # read output in if required 275 unless options[:squelch] 276 output = wait_for_output(stdout) 277 Puppet.warning _("Could not get output") unless output 278 end 279 ensure 280 FFI::WIN32.CloseHandle(process_info.process_handle) 281 FFI::WIN32.CloseHandle(process_info.thread_handle) 282 end 283 end 284 285 if options[:failonfail] and exit_status != 0 286 raise Puppet::ExecutionFailure, _("Execution of '%{str}' returned %{exit_status}: %{output}") % { str: command_str, exit_status: exit_status, output: output.strip } 287 end 288 ensure 289 # Make sure all handles are closed in case an exception was thrown attempting to execute. 290 [stdin, stdout, stderr].each {|io| io.close rescue nil} 291 if !options[:squelch] 292 # if we opened a pipe, we need to clean it up. 293 reader.close if reader 294 stdout.close! if Puppet::Util::Platform.windows? 295 end 296 end 297 298 Puppet::Util::Execution::ProcessOutput.new(output || '', exit_status) 299 end
Returns the path to the ruby executable (available via Config object, even if it's not in the PATH… so this is slightly safer than just using Puppet::Util.which
) @return [String] the path to the Ruby executable @api private
# File lib/puppet/util/execution.rb 306 def self.ruby_path() 307 File.join(RbConfig::CONFIG['bindir'], 308 RbConfig::CONFIG['ruby_install_name'] + RbConfig::CONFIG['EXEEXT']). 309 sub(/.*\s.*/m, '"\&"') 310 end
Private Class Methods
This is private method. @comment see call to private_class_method after method definition @api private
# File lib/puppet/util/execution.rb 322 def self.execute_posix(command, options, stdin, stdout, stderr) 323 child_pid = Puppet::Util.safe_posix_fork(stdin, stdout, stderr) do 324 # We can't just call Array(command), and rely on it returning 325 # things like ['foo'], when passed ['foo'], because 326 # Array(command) will call command.to_a internally, which when 327 # given a string can end up doing Very Bad Things(TM), such as 328 # turning "/tmp/foo;\r\n /bin/echo" into ["/tmp/foo;\r\n", " /bin/echo"] 329 command = [command].flatten 330 Process.setsid 331 begin 332 # We need to chdir to our cwd before changing privileges as there's a 333 # chance that the user may not have permissions to access the cwd, which 334 # would cause execute_posix to fail. 335 cwd = options[:cwd] 336 Dir.chdir(cwd) if cwd 337 338 Puppet::Util::SUIDManager.change_privileges(options[:uid], options[:gid], true) 339 340 # if the caller has requested that we override locale environment variables, 341 if (options[:override_locale]) then 342 # loop over them and clear them 343 Puppet::Util::POSIX::LOCALE_ENV_VARS.each { |name| ENV.delete(name) } 344 # set LANG and LC_ALL to 'C' so that the command will have consistent, predictable output 345 # it's OK to manipulate these directly rather than, e.g., via "withenv", because we are in 346 # a forked process. 347 ENV['LANG'] = 'C' 348 ENV['LC_ALL'] = 'C' 349 end 350 351 # unset all of the user-related environment variables so that different methods of starting puppet 352 # (automatic start during boot, via 'service', via /etc/init.d, etc.) won't have unexpected side 353 # effects relating to user / home dir environment vars. 354 # it's OK to manipulate these directly rather than, e.g., via "withenv", because we are in 355 # a forked process. 356 Puppet::Util::POSIX::USER_ENV_VARS.each { |name| ENV.delete(name) } 357 358 options[:custom_environment] ||= {} 359 Puppet::Util.withenv(options[:custom_environment]) do 360 Kernel.exec(*command) 361 end 362 rescue => detail 363 Puppet.log_exception(detail, _("Could not execute posix command: %{detail}") % { detail: detail }) 364 exit!(1) 365 end 366 end 367 child_pid 368 end
This is private method. @comment see call to private_class_method after method definition @api private
# File lib/puppet/util/execution.rb 376 def self.execute_windows(command, options, stdin, stdout, stderr) 377 command = command.map do |part| 378 part.include?(' ') ? %Q["#{part.gsub(/"/, '\"')}"] : part 379 end.join(" ") if command.is_a?(Array) 380 381 options[:custom_environment] ||= {} 382 Puppet::Util.withenv(options[:custom_environment], :windows) do 383 Puppet::Util::Windows::Process.execute(command, options, stdin, stdout, stderr) 384 end 385 end
# File lib/puppet/util/execution.rb 92 def self.exitstatus 93 $CHILD_STATUS.exitstatus 94 end
This is private method. @comment see call to private_class_method after method definition @api private
# File lib/puppet/util/execution.rb 393 def self.wait_for_output(stdout) 394 # Make sure the file's actually been written. This is basically a race 395 # condition, and is probably a horrible way to handle it, but, well, oh 396 # well. 397 # (If this method were treated as private / inaccessible from outside of this file, we shouldn't have to worry 398 # about a race condition because all of the places that we call this from are preceded by a call to "waitpid2", 399 # meaning that the processes responsible for writing the file have completed before we get here.) 400 2.times do |try| 401 if Puppet::FileSystem.exist?(stdout.path) 402 stdout.open 403 begin 404 return stdout.read 405 ensure 406 stdout.close 407 stdout.unlink 408 end 409 else 410 time_to_sleep = try / 2.0 411 Puppet.warning _("Waiting for output; will sleep %{time_to_sleep} seconds") % { time_to_sleep: time_to_sleep } 412 sleep(time_to_sleep) 413 end 414 end 415 nil 416 end