datalad.cmd¶
Wrapper for command and function calls, allowing for dry runs and output handling
- class datalad.cmd.BatchedCommand(cmd, path=None, output_proc=None)[source]¶
Bases:
datalad.cmd.SafeDelCloseMixin
Container for a process which would allow for persistent communication
- close(return_stderr=False)[source]¶
Close communication and wait for process to terminate
- Returns
stderr output if return_stderr and stderr file was there. None otherwise
- Return type
str
- class datalad.cmd.GitRunnerBase[source]¶
Bases:
object
Mix-in class for Runners to be used to run git and git annex commands
Overloads the runner class to check & update GIT_DIR and GIT_WORK_TREE environment variables set to the absolute path if is defined and is relative path
- class datalad.cmd.GitWitlessRunner(*args, **kwargs)[source]¶
Bases:
datalad.cmd.WitlessRunner
,datalad.cmd.GitRunnerBase
A WitlessRunner for git and git-annex commands.
See GitRunnerBase it mixes in for more details
- cwd¶
- env¶
- run_on_filelist_chunks(cmd, files, protocol=None, cwd=None, env=None, **kwargs)[source]¶
Run a git-style command multiple times if files is too long
- Parameters
cmd (list) – Sequence of program arguments.
files (list) – List of files.
protocol (WitlessProtocol, optional) – Protocol class handling interaction with the running process (e.g. output capture). A number of pre-crafted classes are provided (e.g KillOutput, NoCapture, GitProgress).
cwd (path-like, optional) – If given, commands are executed with this path as PWD, the PWD of the parent process is used otherwise. Overrides any cwd given to the constructor.
env (dict, optional) – Environment to be used for command execution. If cwd was given, ‘PWD’ in the environment is set to its value. This must be a complete environment definition, no values from the current environment will be inherited. Overrides any env given to the constructor.
kwargs – Passed to the Protocol class constructor.
- Returns
At minimum there will be keys ‘stdout’, ‘stderr’ with unicode strings of the cumulative standard output and error of the process as values.
- Return type
dict
- Raises
CommandError – On execution failure (non-zero exit code) this exception is raised which provides the command (cmd), stdout, stderr, exit code (status), and a message identifying the failed command, as properties.
FileNotFoundError – When a given executable does not exist.
- class datalad.cmd.KillOutput(done_future, encoding=None)[source]¶
Bases:
datalad.cmd.WitlessProtocol
WitlessProtocol that swallows stdout/stderr of a subprocess
- pipe_data_received(fd, data)[source]¶
Called when the subprocess writes data into stdout/stderr pipe.
fd is int file descriptor. data is bytes object.
- proc_err = True¶
- proc_out = True¶
- class datalad.cmd.NoCapture(done_future, encoding=None)[source]¶
Bases:
datalad.cmd.WitlessProtocol
WitlessProtocol that captures no subprocess output
As this is identical with the behavior of the WitlessProtocol base class, this class is merely a more readable convenience alias.
- class datalad.cmd.SafeDelCloseMixin[source]¶
Bases:
object
A helper class to use where __del__ would call .close() which might fail if “too late in GC game”
- class datalad.cmd.StdErrCapture(done_future, encoding=None)[source]¶
Bases:
datalad.cmd.WitlessProtocol
WitlessProtocol that only captures and returns stderr of a subprocess
- proc_err = True¶
- class datalad.cmd.StdOutCapture(done_future, encoding=None)[source]¶
Bases:
datalad.cmd.WitlessProtocol
WitlessProtocol that only captures and returns stdout of a subprocess
- proc_out = True¶
- class datalad.cmd.StdOutErrCapture(done_future, encoding=None)[source]¶
Bases:
datalad.cmd.WitlessProtocol
WitlessProtocol that captures and returns stdout/stderr of a subprocess
- proc_err = True¶
- proc_out = True¶
- class datalad.cmd.WitlessProtocol(done_future, encoding=None)[source]¶
Bases:
asyncio.protocols.SubprocessProtocol
Subprocess communication protocol base class for run_async_cmd
This class implements basic subprocess output handling. Derived classes like StdOutCapture should be used for subprocess communication that need to capture and return output. In particular, the pipe_data_received() method can be overwritten to implement “online” processing of process output.
This class defines a default return value setup that causes run_async_cmd() to return a 2-tuple with the subprocess’s exit code and a list with bytestrings of all captured output streams.
- FD_NAMES = ['stdin', 'stdout', 'stderr']¶
- connection_made(transport)[source]¶
Called when a connection is made.
The argument is the transport representing the pipe connection. To receive data, wait for data_received() calls. When the connection is closed, connection_lost() is called.
- pipe_data_received(fd, data)[source]¶
Called when the subprocess writes data into stdout/stderr pipe.
fd is int file descriptor. data is bytes object.
- proc_err = None¶
- proc_out = None¶
- class datalad.cmd.WitlessRunner(cwd=None, env=None)[source]¶
Bases:
object
Minimal Runner with support for online command output processing
It aims to be as simple as possible, providing only essential functionality.
- cwd¶
- env¶
- run(cmd, protocol=None, stdin=None, cwd=None, env=None, **kwargs)[source]¶
Execute a command and communicate with it.
- Parameters
cmd (list or str) – Sequence of program arguments. Passing a single string causes execution via the platform shell.
protocol (WitlessProtocol, optional) – Protocol class handling interaction with the running process (e.g. output capture). A number of pre-crafted classes are provided (e.g KillOutput, NoCapture, GitProgress).
stdin (byte stream, optional) – File descriptor like, used as stdin for the process. Passed verbatim to subprocess.Popen().
cwd (path-like, optional) – If given, commands are executed with this path as PWD, the PWD of the parent process is used otherwise. Overrides any cwd given to the constructor.
env (dict, optional) – Environment to be used for command execution. If cwd was given, ‘PWD’ in the environment is set to its value. This must be a complete environment definition, no values from the current environment will be inherited. Overrides any env given to the constructor.
kwargs – Passed to the Protocol class constructor.
- Returns
At minimum there will be keys ‘stdout’, ‘stderr’ with unicode strings of the cumulative standard output and error of the process as values.
- Return type
dict
- Raises
CommandError – On execution failure (non-zero exit code) this exception is raised which provides the command (cmd), stdout, stderr, exit code (status), and a message identifying the failed command, as properties.
FileNotFoundError – When a given executable does not exist.
- async datalad.cmd.run_async_cmd(loop, cmd, protocol, stdin, protocol_kwargs=None, **kwargs)[source]¶
Run a command in a subprocess managed by asyncio
This implementation has been inspired by https://pymotw.com/3/asyncio/subprocesses.html
- Parameters
loop (asyncio.AbstractEventLoop) – asyncio event loop instance. Must support subprocesses on the target platform.
cmd (list or str) – Command to be executed, passed to subprocess_exec (list), or subprocess_shell (str).
protocol (WitlessProtocol) – Protocol class to be instantiated for managing communication with the subprocess.
stdin (file-like or None) – Passed to the subprocess as its standard input.
protocol_kwargs (dict, optional) – Passed to the Protocol class constructor.
kwargs (Pass to subprocess_exec, will typically be parameters) – supported by subprocess.Popen.
- Returns
The nature of the return value is determined by the given protocol class.
- Return type
undefined