We have an ingested_forecast_length coordinate that is currently empty. It should report the maximum lead time present in the dataset at each init_time. This allows users to quickly determine which initialization times are complete enough for their needs.
This can be done in each dataset's RegionJob subclass's update_template_with_results. The existing coordinate values in the array will need to be read and then updated. A shared utility might help future integrations be bug free.
Add unit tests as well as test in the dynamical_dataset_integration_test.py for each dataset as part of the testing of a dataset update.
Backfill handling: backfills are parallelized and they can't/shouldn't all attempt to write to the same coordinate array. One option I see is a script that we can run after a backfill that reads data at a single point and calculates and writes this array. This can be a follow on. Perhaps there are better ways that are also simple.
We have an
ingested_forecast_lengthcoordinate that is currently empty. It should report the maximum lead time present in the dataset at eachinit_time. This allows users to quickly determine which initialization times are complete enough for their needs.This can be done in each dataset's
RegionJobsubclass'supdate_template_with_results. The existing coordinate values in the array will need to be read and then updated. A shared utility might help future integrations be bug free.Add unit tests as well as test in the dynamical_dataset_integration_test.py for each dataset as part of the testing of a dataset
update.Backfill handling: backfills are parallelized and they can't/shouldn't all attempt to write to the same coordinate array. One option I see is a script that we can run after a backfill that reads data at a single point and calculates and writes this array. This can be a follow on. Perhaps there are better ways that are also simple.