bexchange.processor package

Submodules

bexchange.processor.processors module

class bexchange.processor.processors.example_processor(backend, name, active, extra_arguments)

Bases: processor

process(path, meta)

The process function. Must be non-blocking. Otherwise this will lock up threads. This means that the path/metadata should be put on a queue or in some other way passed on to the actual processing-part without locking up the resources.

class bexchange.processor.processors.processor(backend, name, active, extra_arguments=None)

Bases: object

Base class to be used by all processor implementations

active()

Returns:

if this processor is active or not

backend()

Returns:

the backend instance

name()

Returns:

the name of this processor

process(path, metadata)

The process function. Must be non-blocking. Otherwise this will lock up threads. This means that the path/metadata should be put on a queue or in some other way passed on to the actual processing-part without locking up the resources.

start()

Starts this processor. Typically by starting a thread or consumer pool

stop()

Stops this processor. Typically by joining a number of threads

class bexchange.processor.processors.processor_manager

Bases: object

The processor manager. Will ensure that files are passed on to the processors.

add_processor(processor)

Adds a processor to the manager :param processor: The processor that should be added

classmethod create_processor(name, clz, backend, active, extra_arguments)

Creates an instance of clz with specified arguments :param clz: class name specified as <module>.<classname> :param arguments: a list of arguments that should be used to initialize the class

classmethod from_conf(config, backend)

Creates a processor instance from provided json config :param config: The json config as a dictionary :param backend: The backend this processor should have access to :return: the processor instance

process(file, meta)

Passes on the file to all registered processors. It is up to the processor to determine how the provided file should be handled.

remove_processor(name)

Removes a processor from the manager :param name: The name of the processor that should be removed

Module contents