eogrow.pipelines.batch_to_eopatch

Pipeline for conversion of batch results to EOPatches.

pydantic model eogrow.pipelines.batch_to_eopatch.FeatureMappingSchema[source]

Bases: Schema

Defines a mapping between 1 or more batch outputs into an EOPatch feature

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Fields:
field batch_files: List[str] [Required]

A list of files that will be converted into an EOPatch feature. If you specify multiple tiff files they will be concatenated together over the bands dimension in the specified order.

field dtype: dtype | None = None

Dtype of the output feature.

Validated by:

  • optional_parse_dtype

field feature: Tuple[FeatureType, str] [Required]
field multiply_factor: float = 1

Factor used to multiply feature values with.

class eogrow.pipelines.batch_to_eopatch.BatchToEOPatchPipeline(*args, **kwargs)[source]

Bases: Pipeline

Additionally sets some basic parameters calculated from config parameters

Parameters:

  • args (Any) –

  • kwargs (Any) –

pydantic model Schema[source]

Bases: Schema

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Fields:

  • input_folder_key (str)

  • mapping (List[eogrow.pipelines.batch_to_eopatch.FeatureMappingSchema])

  • output_folder_key (str)

  • remove_batch_data (bool)

  • userdata_feature_name (str | None)

  • userdata_timestamp_reader (str | None)

field input_folder_key: str [Required]

Storage manager key pointing to the path with Batch results

Validated by:

  • validate_storage_key

field mapping: List[FeatureMappingSchema] [Required]

A list of mapping from batch files into EOPatch features.

Validated by:

  • check_nonempty_input

field output_folder_key: str [Required]

Storage manager key pointing to where the EOPatches are saved

Validated by:

  • validate_storage_key

field remove_batch_data: bool = True

Remove the raw batch data after the conversion is complete

field userdata_feature_name: str | None = None

A name of META_INFO feature in which userdata.json would be stored.

field userdata_timestamp_reader: str | None = None

Either an import path to a utility function or a Python code describing how to read dates from userdata dictionary.

config: Schema
filter_patch_list(patch_list)[source]

EOPatches are filtered according to existence of specified output features

Parameters:

patch_list (List[Tuple[str, BBox]]) –

Return type:

List[Tuple[str, BBox]]

build_workflow()[source]

Builds the workflow

Return type:

EOWorkflow

get_processing_node(previous_node)[source]

This method can be overwritten to add more tasks that process loaded data before saving it.

Parameters:

previous_node (EONode) –

Return type:

EONode

get_execution_arguments(workflow, patch_list)[source]

Prepare execution arguments per each EOPatch

Parameters:

  • workflow (EOWorkflow) –

  • patch_list (List[Tuple[str, BBox]]) –

Return type:

Dict[str, Dict[EONode, Dict[str, object]]]