Tasker¶
-
class
Tasker
(tasks=(), max_tasks=4, task_timeout=None, task_queue=None, result_queue=None, command_queue=None, run_until=None, logger='reusables', **task_kwargs)¶ An advanced multiprocessing pool, designed to run in the background, with ability to change number of workers or pause the service all together.
Simply subclass Tasker, overwrite perform_task and run!
It has in and out queues (task_queue, result_queue) which can be provided or will automatically be created as `multiprocessing.Queue()`s.
Warning
Use multiprocessing.Queue not queue.Queue
Warning
Do not use on Windows at this time
Parameters: - tasks – list of tasks to pre-populate the queue with
- max_tasks – the max number of parallel workers
- task_timeout – how long can each task take
- task_queue – option to specify an existing queue of tasks
- result_queue – option to specify an existing queue for results
- run_until – datetime to run until
-
change_task_size
(size)¶ Blocking request to change number of running tasks
-
get
(timeout=None)¶ Retrieve next result from the queue
-
get_state
()¶ Get general information about the state of the class
-
hook_post_command
()¶
-
hook_post_task
()¶
-
hook_pre_command
()¶
-
hook_pre_task
()¶
-
main_loop
(stop_at_empty=False)¶ Blocking function that can be run directly, if so would probably want to specify ‘stop_at_empty’ to true, or have a separate process adding items to the queue.
-
pause
()¶ Stop any more tasks from being run
-
static
perform_task
(task, result_queue, **kwargs)¶ Function to be overwritten that performs the tasks from the list
-
put
(task)¶ Add a task to be processed to the queue
-
stop
()¶ Hard stop the server and sub process
-
unpuase
()¶ Allows tasks to be run again