Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 26 additions & 8 deletions .claude/skills/create-data-source/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1679,7 +1679,7 @@ Zarr file and mock `fetch_array` or the filesystem layer.
- **At least one mock test** per source (see 12d) — no network,
no timeout, no xfail
- **Target 90%+ line coverage** on the source module when running
with `--slow` (see Step 13b). Use `--cov-report=term-missing`
with `--slow` (see Step 13c). Use `--cov-report=term-missing`
to identify uncovered lines.
- Run via: `make pytest TOX_ENV=test-data` or
`pytest test/data/test_<filename>.py -v`
Expand Down Expand Up @@ -1720,24 +1720,42 @@ Present:

## Step 13 — Run Tests

### 13a. Run the new test file
### 13a. Run non-slow tests first

Run only the non-slow (offline / mock) tests to verify they all pass:

```bash
uv run python -m pytest test/data/test_<filename>.py -v
```

All non-slow tests must pass. Fix failures and re-run until green.

### 13b. Run slow tests without timeout

Run the slow (network) tests **without** a `--timeout` CLI flag to
ensure they actually complete successfully without being killed by
a global timeout. The per-test `@pytest.mark.timeout()` decorators
remain on the tests as a safeguard, but the CLI must not impose an
additional cap:

```bash
uv run python -m pytest test/data/test_<filename>.py -v --timeout=60
uv run python -m pytest test/data/test_<filename>.py -v --slow
```

All tests must pass (or `xfail` for network tests). Fix failures
and re-run until green.
All slow tests should either pass or `xfail`. If any test times out
via its `@pytest.mark.timeout()` decorator, investigate whether the
timeout value is too low or the fetch is genuinely hanging, and fix
accordingly.

### 13b. Run coverage report with `--slow` tests
### 13c. Run coverage report with `--slow` tests

Run the new test file **with coverage** and the `--slow` flag to
include network tests. The new data source file must achieve
**at least 90% line coverage**:

```bash
uv run python -m pytest test/data/test_<filename>.py -v \
--slow --timeout=300 \
--slow \
--cov=earth2studio/data/<filename> \
--cov-report=term-missing \
--cov-fail-under=90
Expand All @@ -1759,7 +1777,7 @@ cover the missing lines. Common gaps:

Re-run until coverage is at or above 90%.

### 13c. Run the full data test suite (optional but recommended)
### 13d. Run the full data test suite (optional but recommended)

```bash
make pytest TOX_ENV=test-data
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added AMSU-A (channels 1–14) and AVHRR channel variables to `E2STUDIO_VOCAB`
- Added JPSS ATMS Level 1 BUFR brightness-temperature data source (`JPSS_ATMS`)
- Added `JPSSATMSLexicon` for ATMS variable mappings
- Added JPSS CrIS FSR Level 1 spectral radiance data source (`JPSS_CRIS`)
- Added `JPSSCrISLexicon` for CrIS FSR variable mappings

### Changed

Expand Down
1 change: 1 addition & 0 deletions docs/modules/datasources_dataframe.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Data sources that provide tabular data as DataFrames.

data.ISD
data.JPSS_ATMS
data.JPSS_CRIS
data.MetOpAMSUA
data.MetOpAVHRR
data.MetOpMHS
Expand Down
1 change: 1 addition & 0 deletions earth2studio/data/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
from .isd import ISD
from .jpss import JPSS
from .jpss_atms import JPSS_ATMS
from .jpss_cris import JPSS_CRIS
from .metop_amsua import MetOpAMSUA
from .metop_avhrr import MetOpAVHRR
from .metop_mhs import MetOpMHS
Expand Down
Loading