Skip to content

ECMWF ENS from Dynamical saved as Parquet using int8 & H3 spatial index #33

@JackKelly

Description

@JackKelly
  • Create a Marimo notebook to grab data from Dynamical, change to H3 spatial index, re-scale to [0, 255], cast to uint8, and save to Delta Lake. Marimo notebook to get ECMWF ENS data from Dynamical #34
  • Turn this notebook into a Dagster pipeline
  • Convert wind_u and wind_v into wind speed & wind direction whilst still in float32 space, and save as uint8.
  • Partition by init day

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions