Draft
Conversation
andreas-grafberger
force-pushed
the
feature/hdate_reanalysis_data
branch
from
43a8c27 to
d25fb01
Compare
| } | ||
|
|
||
| assert "Forecast date" not in cov["mars:metadata"] | ||
| assert cov["mars:metadata"] == EXPECTED_REANALYSIS_METADATA |
There was a problem hiding this comment.
@awarde96 In this scenario of test_hdate_multiple_times, is it just outside of the current design to return a single coverage containing both timestamps? Or is that something that should generally be possible?
Since support for our hdate-based reanalysis data does seem to be a rare (and soon outdated) use case, this changes the tests and implementation to only focus on reforecast data in general. Users for old efas reanalysis data now just will not profit from automatic merging of hdate + time + step axis into a single dimension, as this could easily break reforecast retrieval patterns. Each hdate is an independent forecast initialisation, so one coverage per (point × hdate) is the correct reforecast semantic. Fix test expectations to match (coverage counts, per-coverage Forecast date), rename helpers for clarity, and add test_multiple_hdates_and_steps.
- Add from_polytope_reforecast to Encoder ABC - Remove inline hdate hacks from from_polytope in all 8 encoders that had them - Implement from_polytope_reforecast on all 9 non-TimeSeries encoders by delegating to from_polytope(result, date_key="hdate") - Add from_polytope regression tests for all 10 encoder types - Add from_polytope_reforecast tests for all 10 encoder types - Extract shared test helpers into tests/conftest.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fundamental Questions
These are some of the open questions/issues we found after starting the work on the TimeSeries encoder. They partly also apply to other encoders but we'll focus on them first and hopefully solve this for other encoders at the same time.
"Forecast date"and"hdate"with"date"(the model version date). If merging multiple hdates into one coverage, either drop"Forecast date"entirely or make it a list of the hdate+time values.hdatekey in themars:metadatasection (which we likely don't want anyway) is simply a copy of the lasthdate. So for the first few coverages, this key is wrong.mars:metadatasection does not contain thedatekey associated with the model version / reforecast reference date. This is not a big issue but if there's a quick fix, that would be nice.Encoders
Decoders
At the minimum, the existing decoders should know how to deal with the coveragejsons generated by the updated encoders and return correct results. Any further work on compacting multiple coverages e.g. into
MultiPointSeriesor more efficient representations should be left to #109 as a follow-up.Contributor Declaration
By opening this pull request, I affirm the following: