Tasker

class Tasker(tasks=(), max_tasks=4, task_timeout=None, task_queue=None, result_queue=None, command_queue=None, run_until=None, logger='reusables', **task_kwargs)

An advanced multiprocessing pool, designed to run in the background, with ability to change number of workers or pause the service all together.

Simply subclass Tasker, overwrite perform_task and run!

It has in and out queues (task_queue, result_queue) which can be provided or will automatically be created as `multiprocessing.Queue()`s.

Warning

Use multiprocessing.Queue not queue.Queue

Warning

Do not use on Windows at this time

Parameters:

• tasks – list of tasks to pre-populate the queue with
• max_tasks – the max number of parallel workers
• task_timeout – how long can each task take
• task_queue – option to specify an existing queue of tasks
• result_queue – option to specify an existing queue for results
• run_until – datetime to run until

change_task_size(size)

Blocking request to change number of running tasks

get(timeout=None)

Retrieve next result from the queue

get_state()

Get general information about the state of the class

hook_post_command()

hook_post_task()

hook_pre_command()

hook_pre_task()

main_loop(stop_at_empty=False)

Blocking function that can be run directly, if so would probably want to specify ‘stop_at_empty’ to true, or have a separate process adding items to the queue.

pause()

Stop any more tasks from being run

static perform_task(task, result_queue, **kwargs)

Function to be overwritten that performs the tasks from the list

put(task)

Add a task to be processed to the queue

run()

Start the main loop as a background process. *nix only

stop()

Hard stop the server and sub process

unpuase()

Allows tasks to be run again