Skip to content

Burn down the example-notebook parameter-coverage backlog (~94 methods / 257 params) #345

Description

@breimanntools

Problem

Example notebooks must demonstrate every public parameter of the method they document
(.claude/rules/notebooks.md). This was never enforced, so a large backlog accumulated: when
the new CI gate tests/unit/api_tests/test_notebook_param_coverage.py was added, it found
~94 methods with ~257 undemonstrated public parameters across the non-prediction notebooks.
Those are currently recorded in tests/unit/api_tests/notebook_param_coverage_baseline.txt so
the gate is shippable (prediction notebooks were fixed to zero gaps; the rest are baselined).

A baselined param is a documented, public parameter that its example notebook never shows — a
silent hole in the executable docs.

Goal

Burn the baseline down to empty: every public parameter of every method with an example
notebook is demonstrated (as an explicit keyword, grouped sensibly), and the baseline file is
deleted.

Scope / requirements

  • Work through notebook_param_coverage_baseline.txt subpackage by subpackage (each row is
    Symbol::method::param). For each: add the param to its notebook as an explicit kwarg in a
    "further parameters" cell, re-execute with saved outputs (aa.display_df for tables,
    plt.tight_layout() + plt.show() for plots), and remove the corresponding baseline
    row(s)
    in the same change (the staleness guard enforces this).
  • The gate may only shrink the baseline, never grow it. New notebooks/params must ship with
    zero gaps (already enforced).
  • A handful of genuinely non-demonstrable params (visual-only passthroughs) may move to the
    ALLOWLIST in the test with a one-line reason instead of the baseline — keep this tiny.

Non-goals

  • No change to the gate's mechanism (already in place).
  • No behavioural code changes — this is notebook + docs work only.

Measurable acceptance (KPIs)

  • notebook_param_coverage_baseline.txt reaches 0 rows and is removed; ALLOWLIST holds
    only visual-only passthroughs, each with a reason.
  • test_notebook_param_coverage.py passes with the baseline empty.
  • Every touched notebook executes cleanly under pytest --nbmake with saved outputs.
  • Suggested increments: one PR per subpackage (e.g. sf_*, stp_*, sm_*, plot_*, …) so
    each is reviewable, each dropping its rows from the baseline.

Context

Follow-up to the AAPredPlot eval / notebook-coverage PR, which fixed the 14 prediction
notebooks and introduced the gate + baseline. The recurring root cause was that the notebook
param-coverage checker existed only as a skill script, never wired into CI; it is now a real
gate, and this issue tracks paying down the debt it surfaced.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions