dazu.components package



dazu.components.component module

class dazu.components.component.Component(component_config: Optional[Dict[str, Any]] = None)

Bases: dazu.typing.module.Module

A component is a message processing unit in a pipeline. Components are collected sequentially in a pipeline. Each component is called one after another. This holds for initialization, training, persisting and loading the components. If a component comes first in a pipeline, its methods will be called first. E.g. to process an incoming message, the process method of each component will be called. During the processing (as well as the training, persisting and initialization) components can pass information to other components. The information is passed to other components by providing attributes to the so called pipeline context. The pipeline context contains all the information of the previous components a component can use to do its own processing. For example, a featurizer component can provide features that are used by another component down the pipeline to do intent classification.

classmethod cache_key(component_meta: Dict[str, Any], model_metadata: dazu.typing.model.Metadata) → Optional[str]

This key is used to cache components. If a component is unique to a model it should return None. Otherwise, an instantiation of the component will be reused for all models where the metadata creates the same key.

classmethod can_handle_language(language: Hashable) → bool

Check if component supports a specific language. This method can be overwritten when needed. (e.g. dynamically determine which language is supported.)

classmethod create(component_config: Dict[str, Any], config: dazu.config.DazuConfig) → dazu.components.component.Component

Creates this component (e.g. before a training is started). Method can access all configuration parameters.

defaults = {}
language_list = None
classmethod load(component_config: Dict[str, Any], model_dir: Optional[str] = None, model_metadata: Optional[Metadata] = None, cached_component: Optional[Component] = None, **kwargs: Any) → dazu.components.component.Component

Load this component from file. After a component has been trained, it will be persisted by calling persist. When the pipeline gets loaded again, this component needs to be able to restore itself. Components can rely on any context attributes that are created by components.Component.create() calls to components previous to this one.

partially_process(message: dazu.typing.message.Message) → dazu.typing.message.Message

Allows the component to process messages during training (e.g. external training data). The passed message will be processed by all components previous to this one in the pipeline.

persist(file_name: str, model_dir: str) → Optional[Dict[str, Any]]

Persist this component to disk for future loading.

prepare_partial_processing(pipeline: List[Component], context: Dict[str, Any]) → None

Sets the pipeline and context used for partial processing. The pipeline should be a list of components that are previous to this one in the pipeline and have already finished their training (and can therefore be safely used to process messages).

process(message: dazu.typing.message.Message, **kwargs: Any) → None

Process an incoming message. This is the components chance to process an incoming message. The component can rely on any context attribute to be present, that gets created by a call to rasa.nlu.components.Component.create() of ANY component and on any context attributes created by a call to rasa.nlu.components.Component.process() of components previous to this one.

provide_context() → Optional[Dict[str, Any]]

Initialize this component for a new pipeline This function will be called before the training is started and before the first message is processed using the interpreter. The component gets the opportunity to add information to the context that is passed through the pipeline during training and message parsing. Most components do not need to implement this method. It’s mostly used to initialize framework environments like MITIE and spacy (e.g. loading word vectors for the pipeline).

provides = []
classmethod required_packages() → List[str]

Specify which python packages need to be installed to use this component, e.g. ["spacy"]. More specifically, these should be importable python package names e.g. sklearn and not package names in the dependencies sense e.g. scikit-learn This list of requirements allows us to fail early during training if a required package is not installed.

requires = []
train(training_data: dazu.typing.training_data.TrainingData, cfg: dazu.config.DazuConfig, **kwargs: Any) → None

Train this component. This is the components chance to train itself provided with the training data. The component can rely on any context attribute to be present, that gets created by a call to rasa.nlu.components.Component.create() of ANY component and on any context attributes created by a call to rasa.nlu.components.Component.train() of components previous to this one.

exception dazu.components.component.UnsupportedLanguageError(component: str, language: str)

Bases: Exception

Raised when a component is created but the language is not supported.

  • component – component name

  • language – language that component doesn’t support

dazu.components.engine module

class dazu.components.engine.Engine(config: dazu.config.DazuConfig)

Bases: object

components: List[dazu.components.component.Component] = []
pipeline: List[Type[dazu.components.component.Component]] = []
respond(message, context={})
dazu.components.engine.build_model_filename(idx, componentCls: Type[dazu.components.component.Component])

Module contents