Skip to content

DARTS components

Components

The idea behind the Architecture of darts-nextgen is to provide Components to the user. Users should pick their components and put them together in their custom pipeline, utilizing their own parallelization framework. Pipeline v2 shows how this could look like for a simple sequential pipeline - hence without parallelization framework.

There are many different components implemented, all within the different darts packages. Here is an overview over the currently implemented components and their hardware requirements / bounds.

Note: the following table was generated from the public re-exports in each package's __init__.py. I inferred resource requirements conservatively from each component's role (data acquisition -> network/disk; preprocessing/postprocessing/metrics -> CPU/disk; training/ensembling -> heavy compute and often GPU). Where unclear I made reasonable assumptions and marked them in the table. If you want tighter/verified bounds I can inspect individual implementations and tests to refine the entries.

Component Stateful? and why? Network 1 Disk 2 Compute 3 GPU 4
Acquisition
darts_acquisition.download_arcticdem Stateless ✅ ✅
darts_acquisition.load_arcticdem Stateless
darts_acquisition.load_planet_masks Stateless ✅
darts_acquisition.load_planet_scene Stateless ✅ ✅
darts_acquisition.download_cdse_s2_sr_scene Stateless ✅ ✅
darts_acquisition.load_cdse_s2_sr_scene Stateless ✅ ✅
darts_acquisition.download_gee_s2_sr_scene Stateless ✅ ✅
darts_acquisition.load_gee_s2_sr_scene Stateless ✅ ✅
darts_acquisition.download_tcvis Stateless ✅ ✅
darts_acquisition.load_tcvis Stateless
Preprocessing
darts_preprocessing.calculate_aspect Stateless ✅ ✅
darts_preprocessing.calculate_curvature Stateless ✅ ✅
darts_preprocessing.calculate_dissection_index Stateless ✅ ✅
darts_preprocessing.calculate_hillshade Stateless ✅ ✅
darts_preprocessing.calculate_slope Stateless ✅ ✅
darts_preprocessing.calculate_terrain_ruggedness_index Stateless ✅ ✅
darts_preprocessing.calculate_topographic_position_index Stateless ✅ ✅
darts_preprocessing.calculate_vector_ruggedness_measure Stateless ✅ ✅
darts_preprocessing.calculate_ctvi Stateless ✅ ✅
darts_preprocessing.calculate_evi Stateless ✅ ✅
darts_preprocessing.calculate_exg Stateless ✅ ✅
darts_preprocessing.calculate_gli Stateless ✅ ✅
darts_preprocessing.calculate_gndvi Stateless ✅ ✅
darts_preprocessing.calculate_grvi Stateless ✅ ✅
darts_preprocessing.calculate_ndvi Stateless ✅ ✅
darts_preprocessing.calculate_nrvi Stateless ✅ ✅
darts_preprocessing.calculate_rvi Stateless ✅ ✅
darts_preprocessing.calculate_savi Stateless ✅ ✅
darts_preprocessing.calculate_tgi Stateless ✅ ✅
darts_preprocessing.calculate_ttvi Stateless ✅ ✅
darts_preprocessing.calculate_tvi Stateless ✅ ✅
darts_preprocessing.calculate_vari Stateless ✅ ✅
darts_preprocessing.calculate_vdvi Stateless ✅ ✅
darts_preprocessing.calculate_vigreen Stateless ✅ ✅
darts_preprocessing.calculate_spyndex Stateless ✅ ✅
darts_preprocessing.preprocess_legacy_fast 5 Stateless ✅ ✅
darts_preprocessing.preprocess_v2 5 Stateless ✅ ✅
Segmentation
darts_segmentation.segment.SMPSegmenter Stateful — holds model state (weights/config) ✅ ✅
Ensemble
darts_ensemble.EnsembleV1 Stateful — holds ensemble state (weights/config) ✅ ✅
Postprocessing
darts_postprocessing.binarize Stateless ✅ ✅
darts_postprocessing.erode_mask Stateless ✅ ✅
darts_postprocessing.prepare_export 6 Stateless ✅ ✅
Export
darts_export.export_tile Stateless ✅

Next to the components, there exist several helper functions for e.g. searching Sentinel-2 scenes or metrics for the training. Have a look at the Reference for a list of all functions and a describtion of what they do.

  1. Network: Requires network access if ↩

  2. Disk: Reads from or writes to disk ↩

  3. Compute: Does some heavy compute, utilizing the CPU ↩

  4. GPU: Supports offloading compute to the GPU ↩

  5. Wrapper for multiple preprocessing steps, handling compatibility with each other ↩↩

  6. Wrapper for multiple postprocessing steps ↩