Rule provenance + query API for interactive editors (glyph <-> rule, inheritance)

[typedev]

Motivation

Interactive editors (not just batch apply) need to map glyph ↔ rule both ways and to introspect inheritance:

• select a glyph → highlight the rule(s) that affect it;
• cursor on a rule → highlight the glyphs it matches in a grid;
• with !extends, a glyph is often hit by several rules across base + user layers — show all of them and/or the resolved per-glyph result.

None of this is possible today because (a) rules carry no source provenance (Document.rules are bare (selector, op, items) tuples — parse_dsl knows the line number n but discards it, and !extends merge concatenates base.rules + child.rules with no record of origin), and (b) the matcher (_matches) and per-glyph queries are private/absent.

Proposal

1. Rule provenance (foundational)

@dataclass(frozen=True)
class RuleSource:
    ref: str    # preset name ("default") or file path
    line: int   # 1-based line within that source

Attach a source to every rule; preserve it through parse_dsl, _load, and _merge (stamp base rules with the base ref, do not renumber). This lets a UI map a rule to its editor location and tell user-layer rules from inherited (read-only) ones.

2. Matching primitives (public)

def selector_matches(selector, name: str, unicodes: Sequence[int]) -> bool   # promote _matches
def glyphs_for_selector(selector, glyphs: Iterable[tuple[str, Sequence[int]]]) -> list[str]

Cursor → grid: parse the cursor line to a selector with the existing DSL, then list matches. Pure over (name, unicodes) — no fontParts dependency ({category} selectors need unicodes).

3. Per-glyph queries

def rules_for_glyph(doc, name, unicodes) -> list[Rule]          # matching rules in order, with source + op
def explain_glyph(doc, name, unicodes, *, font=None) -> Explanation

rules_for_glyph: grid → editor highlighting. explain_glyph: an ordered trace — for each matching rule its op and the accumulator after it, plus the final specs (resolved coords if font given) — powering a "resolved rules for this glyph" view that makes deep !extends chains legible.

Implementation notes

• parse_dsl must keep the line number it already computes; _merge must preserve each rule's source (do not flatten/renumber).
• accumulate can stay as-is; rules_for_glyph / explain_glyph share its scan logic.

Acceptance

• A user rule round-trips to (ref, line) that points at its editor line.
• explain_glyph(...) final set equals accumulate(...).
• glyphs_for_selector agrees with accumulate membership.

Relation to #2

Independent of #2 (compute_document), but both move the library toward a functional core + a thin imperative shell.

Context

Driving use case: a GTK editor that previews rule-driven anchors and wants bidirectional highlighting (grid selection ↔ rule in the .af editor) and a per-glyph "effective rules" panel. With !extends inheritance, a glyph's anchors come from base + override rules; without provenance and query helpers the editor can't point at the right rule or explain the result.

[typedev]

Maintainer design notes — implementation plan & open decisions

Walked the code paths this touches. The proposal is sound; here's the concrete plan, the non-obvious gotchas, and four decisions I'd like the integration side to weigh in on before I cut it.

Consumers of doc.rules that the format change touches

rules is list[tuple[Selector, Op, list]] today; 3-tuple unpacking lives in:

• apply.py — validate_document, accumulate
• convert.py:40 — render_document
• convert.py:59 — roundtrip.rules != legacy.rules, the lossless round-trip check in the converter
• runner.py:32 — _merge (plain concatenation)

Line numbers are already in scope at build time (dsl.py enumerate, parser.py raw_rows carry n). The ref is known only in runner._load (as ident — preset:<name> or an abspath).

Part 1 — provenance

Rule dataclass, not a 4th tuple element. The gotcha is convert.py:59: if source participates in equality, the converter round-trip breaks (a rule parsed from a legacy file vs. re-parsed from rendered DSL would carry different ref/line). Fix it structurally by excluding source from __eq__:

@dataclass(frozen=True)
class RuleSource:
    ref: str | None   # canonical identity: "preset:default" or an abspath; None = unknown
    line: int         # 1-based

@dataclass(frozen=True)
class Rule:
    selector: Selector
    op: Op
    items: list
    source: RuleSource | None = field(default=None, compare=False)

compare=False makes provenance metadata that doesn't affect semantic identity — convert.py:59 keeps comparing only (selector, op, items) and stays green for free.

Where ref is stamped. The parser shouldn't know the preset:/path convention — that's a runner concern. So:

1. parse_dsl / parse_document set source=RuleSource(ref=None, line=n).
2. _load re-stamps ref=ident over the doc's rules immediately after parse (via dataclasses.replace, since frozen).
3. _merge is untouched — each rule already carries a complete source, so concatenation preserves it. That is exactly the "do not renumber" requirement, with zero logic in merge.

Part 2 — matching primitives

Promote _matches (apply.py:53) to public selector_matches(selector, name, unicodes); add a thin glyphs_for_selector. To avoid an import cycle, keep selector_matches in apply.py (next to accumulate, its only current user) and put the introspection surface (glyphs_for_selector, rules_for_glyph, explain_glyph) in a new query.py that imports one-way from apply. apply never imports query.

Part 3 — per-glyph queries

rules_for_glyph is trivial over selector_matches. For explain_glyph, the important bit is to not duplicate the accumulate scan (the same anti-drift principle as #2). Factor the scan into a private generator:

def _trace(doc, name, unicodes):     # yields (rule, accumulator_snapshot_after)
    ...
def accumulate(doc, name, unicodes):  # = last snapshot, or []
    ...

Then explain_glyph builds on _trace, and the acceptance explain_glyph(...).final == accumulate(...) holds by construction rather than by a postcondition test.

Four decisions for the integration side

1. Rule.__iter__ back-compat shim? I can add __iter__ so existing for sel, op, items in doc.rules keeps working — but it silently drops source. I lean no (we're pre-1.0; update the ~3 internal sites explicitly). If your editor already unpacks 3-tuples from doc.rules, say so and I'll add the shim.
2. ref format. Store the canonical ident (preset:default / abspath) so a UI can detect the preset: prefix to mark inherited/read-only rules — vs. the bare "default" in the issue sketch. I lean ident-with-prefix (unambiguous, and it answers "is this inherited?" directly). OK?
3. explain_glyph suffix depth. Explain the base glyph's specs (op + per-rule snapshots + final), and defer real per-suffix coordinates to compute_document (Add compute-only entry point compute_document() and export resolve() #2)? Or should explain_glyph itself expand suffixes and resolve coords on each target? I lean the former — less overlap with Add compute-only entry point compute_document() and export resolve() #2, cleaner separation.
4. Split delivery. Part 1 (provenance) as one commit, Parts 2+3 (query API) as a second — provenance is self-contained and easier to review in isolation. Or land it all together?

Once these are settled I'll implement. Note ComputeDiagnostic.rule in #4 depends on Part 1 here (it stays None until provenance lands).