Skip to content

Compositional baseline featurizers (scale, AAC, ACC, …) + baseline comparison in AAPred #335

Description

@breimanntools

Problem

CPP produces positional Part–Split–Scale features. To show that CPP's positional
features actually beat simpler encodings, users need baseline featurizers — fixed-length,
non-positional sequence descriptors — to compare against (the use case's
"scale 61/60/75 vs CPP 71/74/94" barplot). Today:

  • Only the scale-average baseline exists (SequenceFeature.scale_composition, née
    scale_mean). Other standard compositional/statistical descriptors are missing.
  • Getting a baseline number is a manual in-between step: df_seq → X_baseline → model → score. The baseline is not a first-class capability of the class that does
    prediction/evaluation (AAPred), so every comparison re-implements the loop.

Goal

  1. A small, consistent family of sequence-level baseline featurizers on
    SequenceFeature — each df_seq → X with no CPP positional split — covering the
    common descriptors used as ML baselines for protein sequences.
  2. Baseline comparison built into the prediction class: AAPred.eval(..., baseline=...) computes the chosen baseline X internally from df_seq and reports
    its score as an extra row of df_eval, next to the CPP-features score — so the whole
    "CPP vs baseline" comparison (and aap.plot_eval barplot) comes from one call, with
    no manual featurization.

Scope / requirements

Baseline featurizers (each: df_seq + list_parts in, X (n_seq, n_dim) out;
return_df option; honest signatures; fully vectorized — no per-sequence Python loop;
non-canonical/gap handling documented; ≥30 tests incl. a byte-identical reference-value
test on a fixture):

  • scale-basedscale_composition (mean scale value per scale over the span).
    Already shipped (feat(sf): SequenceFeature.scale_composition — scale-based baseline featurizer (#307) #319); this issue only tracks it as the first family member.
  • amino-acid composition (AAC) — fraction of each canonical amino acid (20-dim).
  • auto-cross-covariance (ACC) — lag-based auto/cross-covariance of scale values
    (captures short-range sequence order the pure composition drops); n_lag parameter.
  • (candidates, decide during design) dipeptide composition (DPC), CTD, pseudo-AAC (PAAC).

Prediction-class integration:

  • AAPred.eval(df_seq=..., baseline=<kind|True|list>) builds the baseline X
    internally and appends one df_eval row per baseline kind, using the same model/CV as
    the CPP evaluation (apples-to-apples). baseline=Truescale_composition (the
    default baseline).
  • No change to the non-baseline df_eval shape/values when baseline is unset
    (additive).
  • aap.plot_eval renders CPP vs baseline(s) as the comparison barplot.

Consistency: one naming scheme for the family (e.g. <basis>_composition +
acc); shared backend helpers; example notebook per method; docstrings that state the
application (which baseline, when to reach for it) up front.

Non-goals

  • Not a general feature-engineering framework — a small, curated baseline set only.
  • No new heavy dependencies (pure numpy/pandas; reuse the existing scales).
  • Positional/CPP features stay the job of CPP / feature_matrix.

Measurable acceptance (KPIs)

  • A single AAPred.eval(df_seq=..., baseline=True) reproduces the use-case baseline row
    (scale-average vs CPP across the reported metrics) — value-identical to the current
    manual scale_composition → model → score path.
  • Each new descriptor: ≥30 unit tests, vectorized (assert no per-sequence Python loop in
    the hot path), and byte-identical to a documented reference implementation on a fixed
    input.
  • AAPred.eval with baseline unset is byte-identical to today's df_eval; with
    baseline=[k1,k2] adds exactly len([k1,k2]) rows and nothing else.
  • Each featurizer + the AAPred.baseline path is demonstrated in an executed example
    notebook.

Context

Follow-up to the epic #305 "collapse notebook boilerplate" work and the AAPred
prediction layer (#332). Grew out of the observation that scale_mean (now
scale_composition, #319) exists only to feed a baseline model, so the baseline belongs
where prediction/evaluation lives rather than as a manual featurization step.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions