Multiprocess Helpers

run(command, input=None, stdout=-1, stderr=-1, timeout=None, copy_local_env=False, **kwargs)

Cross platform compatible subprocess with CompletedProcess return.

No formatting or encoding is performed on the output of subprocess, so it’s output will appear the same on each version / interpreter as before.

reusables.run('echo "hello world!', shell=True)
# CPython 3.6
# CompletedProcess(args='echo "hello world!', returncode=0,
#                  stdout=b'"hello world!\r\n', stderr=b'')
#
# PyPy 5.4 (Python 2.7.10)
# CompletedProcess(args='echo "hello world!', returncode=0L,
# stdout='"hello world!\r\n')

Timeout is only usable in Python 3.X, as it was not implemented before then, a NotImplementedError will be raised if specified on 2.x version of Python.

Parameters:
  • command – command to run, str if shell=True otherwise must be list
  • input – send something communicate
  • stdout – PIPE or None
  • stderr – PIPE or None
  • timeout – max time to wait for command to complete
  • copy_local_env – Use all current ENV vars in the subprocess as well
  • kwargs – additional arguments to pass to Popen
Returns:

CompletedProcess class
run_in_pool(target, iterable, threaded=True, processes=4, asynchronous=False, target_kwargs=None)

Run a set of iterables to a function in a Threaded or MP Pool.

Parameters:
  • target – function to run
  • iterable – positional arg to pass to function
  • threaded – Threaded if True multiprocessed if False
  • processes – Number of workers
  • asynchronous – will do map_async if True
  • target_kwargs – Keyword arguments to set on the function as a partial
Returns:

pool results