module Mixlib::ShellOut::Unix

Constants

ONE_DOT_EIGHT_DOT_SEVEN

“1.8.7” as a frozen string. We use this with a hack that disables GC to avoid segfaults on Ruby 1.8.7, so we need to allocate the fewest objects we possibly can.

Public Instance Methods

all_seconderies() click to toggle source

Helper method for sgids

# File lib/mixlib/shellout/unix.rb, line 41
def all_seconderies
  ret = []
  Etc.endgrent
  while ( g = Etc.getgrent )
    ret << g
  end
  Etc.endgrent
  ret
end
logon_environment() click to toggle source

The environment variables that are deduced from simulating logon Only valid if login is used

# File lib/mixlib/shellout/unix.rb, line 63
def logon_environment
  return {} unless using_login?

  entry = Etc.getpwuid(uid)
  # According to `man su`, the set fields are:
  #  $HOME, $SHELL, $USER, $LOGNAME, $PATH, and $IFS
  # Values are copied from "shadow" package in Ubuntu 14.10
  { "HOME" => entry.dir, "SHELL" => entry.shell, "USER" => entry.name, "LOGNAME" => entry.name, "PATH" => "/sbin:/bin:/usr/sbin:/usr/bin", "IFS" => "\t\n" }
end
process_environment() click to toggle source

Merges the two environments for the process

# File lib/mixlib/shellout/unix.rb, line 74
def process_environment
  logon_environment.merge(environment)
end
run_command() click to toggle source

Run the command, writing the command’s standard out and standard error to stdout and stderr, and saving its exit status object to status

Returns

returns self; stdout, stderr, status, and exitstatus will be populated with results of the command.

Raises

  • Errno::EACCES when you are not privileged to execute the command

  • Errno::ENOENT when the command is not available on the system (or not in the current $PATH)

  • Chef::Exceptions::CommandTimeout when the command does not complete within timeout seconds (default: 600s). When this happens, ShellOut will send a TERM and then KILL to the entire process group to ensure that any grandchild processes are terminated. If the invocation of the child process spawned multiple child processes (which commonly happens if the command is passed as a single string to be interpreted by bin/sh, and bin/sh is not bash), the exit status object may not contain the correct exit code of the process (of course there is no exit code if the command is killed by SIGKILL, also).

# File lib/mixlib/shellout/unix.rb, line 96
def run_command
  @child_pid = fork_subprocess
  @reaped = false

  configure_parent_process_file_descriptors

  # Ruby 1.8.7 and 1.8.6 from mid 2009 try to allocate objects during GC
  # when calling IO.select and IO#read. Disabling GC works around the
  # segfault, but obviously it's a bad workaround. We no longer support
  # 1.8.6 so we only need this hack for 1.8.7.
  GC.disable if RUBY_VERSION == ONE_DOT_EIGHT_DOT_SEVEN

  # CHEF-3390: Marshall.load on Ruby < 1.8.7p369 also has a GC bug related
  # to Marshall.load, so try disabling GC first.
  propagate_pre_exec_failure

  @status = nil
  @result = nil
  @execution_time = 0

  write_to_child_stdin

  until @status
    ready_buffers = attempt_buffer_read
    unless ready_buffers
      @execution_time += READ_WAIT_TIME
      if @execution_time >= timeout && !@result
        # kill the bad proccess
        reap_errant_child
        # read anything it wrote when we killed it
        attempt_buffer_read
        # raise
        raise CommandTimeout, "Command timed out after #{@execution_time.to_i}s:\n#{format_for_exception}"
      end
    end

    attempt_reap
  end

  self
rescue Errno::ENOENT
  # When ENOENT happens, we can be reasonably sure that the child process
  # is going to exit quickly, so we use the blocking variant of waitpid2
  reap
  raise
ensure
  reap_errant_child if should_reap?
  # make one more pass to get the last of the output after the
  # child process dies
  attempt_buffer_read
  # no matter what happens, turn the GC back on, and hope whatever busted
  # version of ruby we're on doesn't allocate some objects during the next
  # GC run.
  GC.enable
  close_all_pipes
end
sgids() click to toggle source

The secondary groups that the subprocess will switch to. Currently valid only if login is used, and is set to the user’s secondary groups

# File lib/mixlib/shellout/unix.rb, line 54
def sgids
  return nil unless using_login?

  user_name = Etc.getpwuid(uid).name
  all_seconderies.select { |g| g.mem.include?(user_name) }.map(&:gid)
end
using_login?() click to toggle source

Whether we’re simulating a login shell

# File lib/mixlib/shellout/unix.rb, line 36
def using_login?
  login && user
end
validate_options(opts) click to toggle source

Option validation that is unix specific

# File lib/mixlib/shellout/unix.rb, line 29
def validate_options(opts)
  if opts[:elevated]
    raise InvalidCommandOption, "Option `elevated` is supported for Powershell commands only"
  end
end

Private Instance Methods

attempt_buffer_read() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 279
def attempt_buffer_read
  ready = IO.select(open_pipes, nil, nil, READ_WAIT_TIME)
  if ready
    read_stdout_to_buffer if ready.first.include?(child_stdout)
    read_stderr_to_buffer if ready.first.include?(child_stderr)
    read_process_status_to_buffer if ready.first.include?(child_process_status)
  end
  ready
end
attempt_reap() click to toggle source

Try to reap the child process but don’t block if it isn’t dead yet.

# File lib/mixlib/shellout/unix.rb, line 408
def attempt_reap
  results = Process.waitpid2(@child_pid, Process::WNOHANG)
  if results
    @reaped = true
    @status = results.last
  else
    nil
  end
end
child_pgid() click to toggle source

Since we call setsid the child_pgid will be the child_pid, set to negative here so it can be directly used in arguments to kill, wait, etc.

# File lib/mixlib/shellout/unix.rb, line 192
def child_pgid
  -@child_pid
end
child_process_status() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 213
def child_process_status
  @process_status_pipe[0]
end
child_stderr() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 209
def child_stderr
  @stderr_pipe[0]
end
child_stdin() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 201
def child_stdin
  @stdin_pipe[1]
end
child_stdout() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 205
def child_stdout
  @stdout_pipe[0]
end
close_all_pipes() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 217
def close_all_pipes
  child_stdin.close   unless child_stdin.closed?
  child_stdout.close  unless child_stdout.closed?
  child_stderr.close  unless child_stderr.closed?
  child_process_status.close unless child_process_status.closed?
end
configure_parent_process_file_descriptors() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 250
def configure_parent_process_file_descriptors
  # Close the sides of the pipes we don't care about
  stdin_pipe.first.close
  stdin_pipe.last.close unless input
  stdout_pipe.last.close
  stderr_pipe.last.close
  process_status_pipe.last.close
  # Get output as it happens rather than buffered
  child_stdin.sync = true if input
  child_stdout.sync = true
  child_stderr.sync = true

  true
end
configure_subprocess_file_descriptors() click to toggle source

Replace stdout, and stderr with pipes to the parent, and close the reader side of the error marshaling side channel.

If there is no input, close STDIN so when we exec, the new program will know it’s never getting input ever.

# File lib/mixlib/shellout/unix.rb, line 229
def configure_subprocess_file_descriptors
  process_status_pipe.first.close

  # HACK: for some reason, just STDIN.close isn't good enough when running
  # under ruby 1.9.2, so make it good enough:
  stdin_pipe.last.close
  STDIN.reopen stdin_pipe.first
  stdin_pipe.first.close unless input

  stdout_pipe.first.close
  STDOUT.reopen stdout_pipe.last
  stdout_pipe.last.close

  stderr_pipe.first.close
  STDERR.reopen stderr_pipe.last
  stderr_pipe.last.close

  STDOUT.sync = STDERR.sync = true
  STDIN.sync = true if input
end
fork_subprocess() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 318
def fork_subprocess
  initialize_ipc

  fork do
    # Child processes may themselves fork off children. A common case
    # is when the command is given as a single string (instead of
    # command name plus Array of arguments) and /bin/sh does not
    # support the "ONESHOT" optimization (where sh -c does exec without
    # forking). To support cleaning up all the children, we need to
    # ensure they're in a unique process group.
    #
    # We use setsid here to abandon our controlling tty and get a new session
    # and process group that are set to the pid of the child process.
    Process.setsid

    configure_subprocess_file_descriptors

    set_secondarygroups
    set_group
    set_user
    set_environment
    set_umask
    set_cwd

    begin
      command.is_a?(Array) ? exec(*command, close_others: true) : exec(command, close_others: true)

      raise "forty-two" # Should never get here
    rescue Exception => e
      Marshal.dump(e, process_status_pipe.last)
      process_status_pipe.last.flush
    end
    process_status_pipe.last.close unless process_status_pipe.last.closed?
    exit!
  end
end
initialize_ipc() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 196
def initialize_ipc
  @stdin_pipe, @stdout_pipe, @stderr_pipe, @process_status_pipe = IO.pipe, IO.pipe, IO.pipe, IO.pipe
  @process_status_pipe.last.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
end
open_pipes() click to toggle source

Some patch levels of ruby in wide use (in particular the ruby 1.8.6 on OSX) segfault when you IO.select a pipe that’s reached eof. Weak sauce.

# File lib/mixlib/shellout/unix.rb, line 267
def open_pipes
  @open_pipes ||= [child_stdout, child_stderr, child_process_status]
end
propagate_pre_exec_failure() click to toggle source

Attempt to get a Marshaled error from the side-channel. If it’s there, un-marshal it and raise. If it’s not there, assume everything went well.

# File lib/mixlib/shellout/unix.rb, line 358
def propagate_pre_exec_failure
  attempt_buffer_read until child_process_status.eof?
  e = Marshal.load(@process_status)
  raise(Exception === e ? e : "unknown failure: #{e.inspect}")
rescue ArgumentError # If we get an ArgumentError error, then the exec was successful
  true
ensure
  child_process_status.close
  open_pipes.delete(child_process_status)
end
read_process_status_to_buffer() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 309
def read_process_status_to_buffer
  while ( chunk = child_process_status.read_nonblock(READ_SIZE) )
    @process_status << chunk
  end
rescue Errno::EAGAIN
rescue EOFError
  open_pipes.delete(child_process_status)
end
read_stderr_to_buffer() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 299
def read_stderr_to_buffer
  while ( chunk = child_stderr.read_nonblock(READ_SIZE) )
    @stderr << chunk
    @live_stderr << chunk if @live_stderr
  end
rescue Errno::EAGAIN
rescue EOFError
  open_pipes.delete(child_stderr)
end
read_stdout_to_buffer() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 289
def read_stdout_to_buffer
  while ( chunk = child_stdout.read_nonblock(READ_SIZE) )
    @stdout << chunk
    @live_stdout << chunk if @live_stdout
  end
rescue Errno::EAGAIN
rescue EOFError
  open_pipes.delete(child_stdout)
end
reap() click to toggle source

Unconditionally reap the child process. This is used in scenarios where we can be confident the child will exit quickly, and has not spawned and grandchild processes.

# File lib/mixlib/shellout/unix.rb, line 395
def reap
  results = Process.waitpid2(@child_pid)
  @reaped = true
  @status = results.last
rescue Errno::ECHILD
  # When cleaning up timed-out processes, we might send SIGKILL to the
  # whole process group after we've cleaned up the direct child. In that
  # case the grandchildren will have been adopted by init so we can't
  # reap them even if we wanted to (we don't).
  nil
end
reap_errant_child() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 369
def reap_errant_child
  return if attempt_reap

  @terminate_reason = "Command exceeded allowed execution time, process terminated"
  logger&.error("Command exceeded allowed execution time, sending TERM")
  Process.kill(:TERM, child_pgid)
  sleep 3
  attempt_reap
  logger&.error("Command exceeded allowed execution time, sending KILL")
  Process.kill(:KILL, child_pgid)
  reap

  # Should not hit this but it's possible if something is calling waitall
  # in a separate thread.
rescue Errno::ESRCH
  nil
end
set_cwd() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 186
def set_cwd
  Dir.chdir(cwd) if cwd
end
set_environment() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 175
def set_environment
  # user-set variables should override the login ones
  process_environment.each do |env_var, value|
    ENV[env_var] = value
  end
end
set_group() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 162
def set_group
  if group
    Process.egid = gid
    Process.gid = gid
  end
end
set_secondarygroups() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 169
def set_secondarygroups
  if sgids
    Process.groups = sgids
  end
end
set_umask() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 182
def set_umask
  File.umask(umask) if umask
end
set_user() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 155
def set_user
  if user
    Process.uid = uid
    Process.euid = uid
  end
end
should_reap?() click to toggle source
# File lib/mixlib/shellout/unix.rb, line 387
def should_reap?
  # if we fail to fork, no child pid so nothing to reap
  @child_pid && !@reaped
end
write_to_child_stdin() click to toggle source

Keep this unbuffered for now

# File lib/mixlib/shellout/unix.rb, line 272
def write_to_child_stdin
  return unless input

  child_stdin << input
  child_stdin.close # Kick things off
end