docs: pond retrieval ergonomics field test (external session)

[tenequm]

What

A standalone research doc — docs/researches/pond-retrieval-ergonomics-fieldtest-2026-06-16.md — capturing empirical findings from an external Claude Code session (project x402-services, model Opus) that used pond heavily to recover a fact from a long, evolving stored session.

It is deliberately self-contained and does not reference or build on any other doc in this repo, so it can be compared against existing design notes independently rather than merged into them.

Why it might be worth keeping

pond measured worse than raw grep over the session's JSONL for a common task ("find what we concluded about X in a long session"): +22% tokens, +77% tool calls in a controlled A/B. The doc records the measurements and the root causes so they don't have to be rediscovered.

Headline finding: the supersession blind spot

Three agents given the identical question returned three different answers — because each anchored on a different temporal slice of one 3.5-hour session whose conclusion was revised partway through. pond_search ranks by relevance, which surfaces an early, since-overturned conclusion and the agent reports it as current. The only agent that got the latest truth did so solely because it read pond_get(session_from="end"). Verified against the raw transcript by grep — the divergence is real temporal slicing, not model confabulation.

Recommendations (no new tools — reshape the existing three)

• Recency/supersession: pond_search order: relevance|recent; pond_get footer signalling newer messages exist.
• Bounded responses: per-cell truncation in pond_sql_query (not whole-response); match-centered windows in pond_get.
• Conversation-first by construction: clean conversational default; tool outputs reachable by id only; do not index tool outputs into search.
• Exact vs semantic discoverability: pond_search mode: hybrid|fts, with the words-vs-characters rule in the description.
• Bake the happy-path flow into tool descriptions (docs-first runs were ~40% leaner).

Caveats (in the doc, not hidden)

Early rounds used a contaminated prompt; the corpus grew during the experiment; model capability varied; single task/corpus. The clean round-4 run is the trustworthy comparison. Numbers are direction, not precision.

Author note: written by a pond consumer from another project, not a maintainer — treat as outside-in field feedback.

[tenequm]

Absorbed - closing without merge. Every actionable finding here has now landed or was verified already fixed:

• Finding 1 (recency/supersession, the core one): sort_by: relevance|recency already shipped earlier; the supersession guidance is now in PR feat(mcp)!: rename pond_sql_query to pond_sql and make the tool surface route #102 - server instructions + SKILL.md steer "what did we decide / latest state" to session_from="end" / sort_by="recency", and the later-messages page marker now says "conclusions may have been revised".
• Finding 2 (bounded responses): per-cell clipping landed in feat(mcp)!: rename pond_sql_query to pond_sql and make the tool surface route #102 - inline cells clip at 1000 chars with a full-value marker (measured: the fat-cell query dropped 214KB -> 5.5KB); exports stay unclipped.
• Finding 3 (conversation-first): already codified; re-verified live - the "200 empty system carriers" artifact is fixed (a session with 15,500 system messages renders page 1 as pure user/assistant text). Tool outputs stay un-indexed by design.
• Finding 4 (exact vs semantic): mode: vector|fts shipped; the words-vs-characters rule lives in the tool descriptions and schema://pond-sql.
• Finding 5 (subagents + docs-first): include_subagents is gone from search, and feat(mcp)!: rename pond_sql_query to pond_sql and make the tool surface route #102 bakes the happy-path flow into the server instructions and the bundled skill (which pond init now installs).

Keeping the field test itself out of the repo - its numbers were partly prompt-contaminated (rounds 1-3, as the doc itself flags), and the durable lessons now live in the shipped surfaces instead of a research note.