AEMO migrated FCAS Providers data out of the registration XLS; current handler returns nothing

[nick-gorman]

Summary

NEMOSIS's static_table(\"FCAS Providers\", ...) reads the Ancillary Services sheet of AEMO's NEM Registration and Exemption List.xlsx. AEMO has since emptied that sheet entirely and migrated the data to a separate weekly-archive endpoint on nemweb. The current NEMOSIS handler downloads the (still-existing) workbook and parses an empty sheet, then chokes on the primary-key dedup step. Effective result: the table cannot be retrieved via NEMOSIS at all.

Evidence

The `Ancillary Services` sheet, as served by AEMO today, contains exactly two non-empty cells:

| Row 1 | `Ancillary services report can now be located here:` |
| Row 2 | `www.nemweb.com.au - /REPORTS/CURRENT/ANCILLARY_SERVICES_REPORTS/` |

The new location serves weekly snapshots like:

```
https://www.nemweb.com.au/REPORTS/CURRENT/ANCILLARY_SERVICES_REPORTS/PUBLIC_ANCILLARY_SERVICES_20260519.zip
https://www.nemweb.com.au/REPORTS/CURRENT/ANCILLARY_SERVICES_REPORTS/PUBLIC_ANCILLARY_SERVICES_20260512.zip
...
```

(~12 weekly snapshots, ~3 months of history at the time of writing.)

Failure mode on current master

```python

from nemosis import static_table
static_table("FCAS Providers", "/tmp/cache") # cold cache
NoDataToReturn: Compiling data for table FCAS Providers failed.

static_table("Generators and Scheduled Loads", "/tmp/cache") # prime workaround for #69
static_table("FCAS Providers", "/tmp/cache")
KeyError: Index(['Bid Type', 'DUID'], dtype='str')
```

The KeyError comes from `drop_duplicates(primary_keys)` where `defaults.table_primary_keys["FCAS Providers"] = ["DUID", "Bid Type"]` — columns that no longer exist anywhere in the workbook. `select_columns='all'` doesn't avoid the dedup step, so there's no user-side workaround.

Relationship to #69

Issue #69 was filed when `FCAS Providers` had no entry in `static_downloader_map`, causing `NoDataToReturn` on a cold cache. The workaround was to prime the cache by fetching `Generators and Scheduled Loads` first (same workbook). That workaround no longer helps — once the file is on disk, the dedup step fails because the sheet's data has been removed at the AEMO source. #69 is effectively obsoleted by this finding.

Recommendation

Two options, in increasing order of effort:

1. Deprecate cleanly (short term): have `static_table("FCAS Providers", ...)` raise immediately with a clear message pointing users to the new AEMO endpoint and this issue. Hide `FCAS Providers` from `defaults.static_tables`. Submitting a PR for this now.
2. Rewrite against the new endpoint (longer term): implement a new handler that fetches the latest `PUBLIC_ANCILLARY_SERVICES_YYYYMMDD.zip`, extracts the CSV inside, and exposes it under the same `FCAS Providers` name. Schema discovery is required first — column names, types, and primary keys all need to be re-derived from a recent zip.

The deprecation PR will go up shortly. The long-term rewrite is parked pending interest / a contributor.

How surfaced

Found while building an end-to-end smoke harness against the live AEMO server. The smoke for FCAS Providers consistently failed; tracing showed the underlying sheet was empty. AEMO's announcement of this migration (if any) wasn't located — found by direct inspection of the workbook.