Wrapper around the multiprocessing pool used for generating products Since we want to avoid processing old jobs before newer jobs we have to prioritize the jobs and make sure that we are not draining computer resources on irrelevant jobs. More...

Inheritance diagram for Lib.algorithm_runner.algorithm_runner:

Public Member Functions
	__init__ (self, nrprocesses)

	add (self, func, jobid, algorithm, files, arguments, jobdone=None)
	Adds one job to be processed.

	async_callback (self, arg)
	Invoked by the async pool on answer.

	join (self)
	Shutsdown and waits for pool to terminate.

	terminate (self)
	Terminates the pool.

Public Attributes
	lock = threading.Lock()

	queue = queue.PriorityQueue()

	pool = RavePool(nrprocesses, initializer=worker_initfunction)

	nrprocesses = nrprocesses

int	running_jobs = 0

Static Public Attributes
	Process = NonDaemonProcess

Protected Member Functions
	_handle_queue (self)
	Runs one job from the queue.

Detailed Description

Wrapper around the multiprocessing pool used for generating products Since we want to avoid processing old jobs before newer jobs we have to prioritize the jobs and make sure that we are not draining computer resources on irrelevant jobs.

Also, if a job with same algorithm_id, date and time is added when another job is in the queue, the queued job will be replaced with the new jobs jobid, arguments and files.

Member Function Documentation

◆ _handle_queue()

Lib.algorithm_runner.algorithm_runner._handle_queue ( self )

protected

Runs one job from the queue.

Synchronization must be performed before entering this method. Will increase running jobs with 1 if job added async to pool

◆ add()

Lib.algorithm_runner.algorithm_runner.add	(	self,
		func,
		jobid,
		algorithm,
		files,
		arguments,
		jobdone = None )

Adds one job to be processed.

If arguments contains an algorithm_id, the priority of this job will be increased. Also, if the arguments contains date & time, this will also increase priority. If queue already contains an algorithm that is matching a previously added algorithm, its contant will be replaced (files and arguments).

◆ async_callback()

Lib.algorithm_runner.algorithm_runner.async_callback	(		self,
			arg )

Invoked by the async pool on answer.

◆ join()

Lib.algorithm_runner.algorithm_runner.join ( self )

Shutsdown and waits for pool to terminate.

◆ terminate()

Lib.algorithm_runner.algorithm_runner.terminate ( self )

Terminates the pool.

The documentation for this class was generated from the following file:

/github/workspace/Lib/algorithm_runner.py

Public Member Functions

Public Attributes

Static Public Attributes