eogrow.pipelines.export_maps

Implements a pipeline for exporting data to TIFF files, can be used to prepare BYOC tiles.

class eogrow.pipelines.export_maps.ExportMapsPipeline(config, raw_config=None)[source]

Bases: Pipeline

Pipeline to export a feature into a tiff map

Parameters:

  • config (Schema) – A dictionary with configuration parameters

  • raw_config (RawConfig | None) – The configuration parameters pre-validation, for logging purposes only

pydantic model Schema[source]

Bases: Schema

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Fields:

  • band_indices (List[int] | None)

  • cogification_resampling (Literal[None, 'NEAREST', 'MODE', 'AVERAGE', 'BILINEAR', 'CUBIC', 'CUBICSPLINE', 'LANCZOS'])

  • cogify (bool)

  • feature (Tuple[eolearn.core.constants.FeatureType, str])

  • force_local_copies (bool)

  • input_folder_key (str)

  • interim_results_suffix (str)

  • map_dtype (Literal['int8', 'int16', 'uint8', 'uint16', 'float32'])

  • map_name (str | None)

  • merge_workers (int | None)

  • no_data_value (float | None)

  • output_folder_key (str)

  • scale_factor (float | None)

  • skip_existing (Literal[False])

  • split_per_timestamp (bool)

  • warp_resampling (Literal[None, 'near', 'bilinear', 'cubic', 'cubicspline', 'lanczos', 'average', 'rms', 'mode', 'max', 'min', 'med', 'q1', 'q3', 'sum'])

field band_indices: List[int] | None = None

A list of band indices to be exported for the export feature. Default is all bands

field cogification_resampling: CogifyResamplingOptions = None

Which resampling to use in the cogification process for creating overviews.

field cogify: bool = False

Whether exported GeoTIFFs will be converted into Cloud Optimized GeoTIFFs (COG)

field feature: Feature [Required]
field force_local_copies: bool = False

By default copying to local temporary folder will happen only if an AWS S3 path is configured. With this parameter you force to always make copies.

field input_folder_key: str [Required]

The storage manager key pointing to the input folder for the export maps pipeline.

Validated by:

  • validate_storage_key

field interim_results_suffix: str = ''

Has no effect on end results. Adds a suffix to names of temporary files in order to avoid clashes from pipelines working in parallel on same maps (e.g. exporting same map for different timestamps).

field map_dtype: Literal['int8', 'int16', 'uint8', 'uint16', 'float32'] [Required]
field map_name: str | None = None
Constraints:

  • pattern = .+.tiff?b

field merge_workers: int | None = None

How many workers are used to parallelize merging of TIFFs. Uses all cores (of head node) by default.Decreasing this should help with memory and disk-space issues.

field no_data_value: float | None = None

No data value to be passed to GeoTIFFs

field output_folder_key: str [Required]

The storage manager key pointing to the output folder for the maps in the export maps pipeline.

Validated by:

  • validate_storage_key

field scale_factor: float | None = None

Feature will be multiplied by this value at export

field skip_existing: Literal[False] = False
field split_per_timestamp: bool = True

Temporal features are by default exported as multiple TIFF files in a per-timestamp manner. Disabling this parameter results in a single TIFF with the same band order as the one of ExportToTiffTask.

field warp_resampling: WarpResamplingOptions = None

The resampling method used when warping, useful for pixel misalignment

config: Schema
MERGED_MAP_NAME = 'merged'
run_procedure()[source]

Extracts and merges the data from EOPatches into a TIFF file.

  1. Extracts data from EOPatches via workflow into per-EOPatch tiffs.

  2. For each UTM zone:

    • Prepares tiffs for merging (transfers to local if needed).

    • Performs temporal split of tiffs if needed (assumption that all eopatches share the same timestamps)

    • Merges the tiffs

    • Cogification is done if requested.

    • The output files are finalized (renamed/transferred) and per-EOPatch tiffs are cleaned.

Return type:

tuple[list[str], list[str]]

build_workflow()[source]
Return type:

EOWorkflow

get_execution_arguments(workflow, patch_list)[source]

Prepares execution arguments for each eopatch from a list of patches.

The output should be a dictionary of form {execution_name: {node: node_kwargs}}. Execution names are usually names of EOPatches, but can be anything.

Parameters:

  • workflow (EOWorkflow) – A workflow for which arguments will be prepared

  • patch_list (List[Tuple[str, BBox]]) –

Return type:

Dict[str, Dict[EONode, Dict[str, object]]]

class eogrow.pipelines.export_maps.SplitTiffsJob(input_path, bands, output_path)[source]

Bases: object

Describes which bands of input path to extract to the output path.

Parameters:

  • input_path (str) –

  • bands (Iterable[int]) –

  • output_path (str) –

input_path: str
bands: Iterable[int]
output_path: str
class eogrow.pipelines.export_maps.CombineTiffsJob(input_paths, output_path, time)[source]

Bases: object

Describes which tiffs to merge, what the output path is, and the time the merged tiff represents.

The time is relevant in order to correctly place the finalized tiff. If left out then the tiffs are either timeless or not split temporally.

Parameters:

  • input_paths (Iterable[str]) –

  • output_path (str) –

  • time (datetime | None) –

input_paths: Iterable[str]
output_path: str
time: datetime | None
eogrow.pipelines.export_maps.get_tiff_name(map_name, name, crs=None, time=None, suffix='')[source]

Creates a name of a geotiff image

Parameters:

  • map_name (str) –

  • name (str) –

  • crs (CRS | None) –

  • time (datetime | None) –

  • suffix (str) –

Return type:

str