eogrow.pipelines.byoc

Defines pipelines for ingesting and modifying BYOC collections.

class eogrow.pipelines.byoc.IngestByocTilesPipeline(config, raw_config=None)[source]

Bases: Pipeline

Ingests .tiff files to a BYOC collection. Based on ExportMapsPipeline output for timeless features.

Parameters:

  • config (Schema) – A dictionary with configuration parameters

  • raw_config (RawConfig | None) – The configuration parameters pre-validation, for logging purposes only

pydantic model Schema[source]

Bases: Schema

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Fields:

  • byoc_tile_folder_key (str)

  • cover_geometry (str | None)

  • cover_geometry_folder_key (str | None)

  • existing_collection (sentinelhub.data_collections.DataCollection | None)

  • file_glob_pattern (str)

  • is_temporal (bool)

  • new_collection_name (str | None)

  • reingest_existing (bool)

  • sensing_time (datetime.datetime | None)

field byoc_tile_folder_key: str [Required]
Validated by:

  • validate_storage_key

field cover_geometry: str | None = None

Specifies a geometry file describing the cover geometry.

Validated by:

  • must_be_used_with_cover_geometry_folder_key

field cover_geometry_folder_key: str | None = None

Folder for supplying a custom cover geometry.

Validated by:

  • validate_storage_key

field existing_collection: DataCollection | None = None

Used when updating and reingesting.

Validated by:

  • cannot_be_used_with_new_collection_name

  • optional_parse_data_collection

field file_glob_pattern: str = '**/*.tiff'

Pattern used to obtain the TIFF files to use

field is_temporal: bool = False

If the BYOC is marked as temporal the pipeline will assume that the direct parent folder of a TIFF contains the sensing time, i.e. filesystem structure follows that used by ExportMapsPipeline. Example of such a path is UTM_32638/2019-01-04T07-48-37/BANDS_S2_L1C.tiff.

field new_collection_name: str | None = None

Used for defining a new BYOC collection.

field reingest_existing: bool = False

Reingest or skip already ingested tiles.

field sensing_time: datetime.datetime | None = None

Sensing time (ISO format) added to BYOC tiles. Only used for timeless collections.

Validated by:

  • _parse_sensing_time

config: Schema
get_byoc_collection(byoc_client)[source]

Obtains information about the existing collection or creates a new one.

Parameters:

byoc_client (SentinelHubBYOC) –

Return type:

Dict[str, Any]

get_tile_paths()[source]

Collects the folders and filenames of .tiff files to be ingested.

Paths are relative to bucket, not project.

Return type:

dict[str, list[str]]

run_procedure()[source]

Runs the procedure.

  1. Creates or loads the collection,

  2. Checks existing tiles,

  3. Creates new tiles and skips/reingests existing ones.

Return type:

tuple[list[str], list[str]]