Skip to content

Wiki + librarian workers: emit operator-facing reasoning to the log drawer#34

Merged
sysread merged 1 commit into
mainfrom
claude/add-worker-explanations-Sx3pV
May 12, 2026
Merged

Wiki + librarian workers: emit operator-facing reasoning to the log drawer#34
sysread merged 1 commit into
mainfrom
claude/add-worker-explanations-Sx3pV

Conversation

@sysread
Copy link
Copy Markdown
Owner

@sysread sysread commented May 12, 2026

SYNOPSIS

The wiki per-conversation worker and the wiki librarian now end each cycle with a one-or-two-sentence operator summary of WHY they edited / merged / skipped, surfaced as reasoning="..." on the log-drawer line. The journal worker already did this; this PR lifts the other two to parity.

PURPOSE

Currently the wiki and librarian cycle logs show only a tool-call count - "finished thread X (3 tool calls over 25 messages)", "librarian finished (5 tool calls over 87 articles)". A user skimming the log drawer cannot tell:

  • whether a no-op cycle was correct (generic Q&A, no user-centric subject) or a silent regression,
  • which articles the librarian considered for merge but deliberately left separate, and why,
  • whether the per-conversation agent linked out to Wikipedia instead of creating a sterile article.

The journal worker already solves this via its structured {worthy, reasoning, entry} output, which the loop inlines as reasoning="...". The wiki and librarian are tool-driven (no JSON output), so historically their prompts closed with "reply with a single word; the word is discarded" purely to terminate the tool loop. The final-text channel was free; the prompt just wasn't asking for anything useful on it.

DESCRIPTION

Layer 1 - how existing code behaves

  • Journal worker parses {worthy, reasoning, entry} from a JSON completion. JournalOutput.reasoning is set from that field; the loop logs (wrote=true/false, ..., reasoning="...") at info tier.
  • Wiki per-conversation worker runs runHeadlessToolLoop and returns WikiOutput.finalText - the model's final assistant turn after the tool loop terminates. The prompt's closing block says "reply with a single word; the word is discarded". The loop logs (N tool calls over M messages) with no reasoning= field; finalText is captured into the output struct but never surfaced.
  • Wiki librarian is the same shape: runHeadlessToolLoop, WikiLibrarianOutput.finalText, "reply with a single word", loop logs (N tool calls over M articles).

Layer 2 - what this PR changes

  • Wiki autonomous prompt (src/lib/agents/wiki/prompt.ts) - replaces the "single word" closing block with a "Final reply: one or two sentences explaining your choices" block. Three worked examples cover update, no-op-because-generic-Q&A, and no-op-because-not-user-centric. Asks for plain text, no Markdown, under two sentences, and explicitly tells the model the text surfaces in the user's log drawer.
  • Wiki cycle driver (src/lib/agents/wiki/loop.ts) - whitespace-normalises runResult.output.finalText (collapses \s+ to a single space, trims), falls back to (none) on empty, and inlines it as reasoning="..." on the existing finished thread ... info log. Matches the journal worker's format exactly so grep / scrollback search lines up across all three.
  • Librarian prompt (src/lib/agents/wiki-librarian/prompt.ts) - same change. Examples deliberately call out the "considered for merge but left separate" case, which is the user's stated motivating example - that decision is currently invisible in the logs.
  • Librarian cycle driver (src/lib/agents/wiki-librarian/loop.ts) - same reasoning="..." surface on the librarian finished ... line.
  • Type docs (src/lib/agents/wiki/types.ts, src/lib/agents/wiki-librarian/types.ts) - finalText doc comments no longer call the field "discarded by production callers"; they describe the operator-facing role and reference the prompt's "Final reply" block.
  • Agent file headers (src/lib/agents/wiki/agent.ts, src/lib/agents/wiki-librarian/agent.ts) - updated to match.
  • Dev doc (docs/dev/wiki.md) - the "Entry points" section's WikiAgent.run paragraph now describes the operator-summary contract instead of calling final-text discarded. A new Gotcha entry notes that the prompt's "Final reply" instruction and the loop's reasoning="..." surface move together - reverting one without the other silently drops a user-facing diagnostic.

The journal worker is not touched - it already does this.

Notes for AI reviewers:

  • The \s+ -> ' ' whitespace normalisation in both loops is deliberate. Models occasionally emit a multi-line final turn even when asked for one or two sentences; the single-line log convention is load-bearing and a stray newline would break the (field=value, field=value) "title" shape the journal worker established.
  • The || '(none)' sentinel is also deliberate. An empty finalText shouldn't happen in production (the prompt now mandates a sentence), but a dangling reasoning="" reads as a logger bug; the sentinel makes a missing summary self-explanatory.
  • The new "Final reply" prompt blocks are additive - they sit after the existing scope / preserve-facts / be-conservative discipline blocks. The "single word" closing instruction is removed in the same edit; leaving both would tell the model conflicting things.

Test plan

  • mise run check green: 1069 tests passed, build clean (the pre-existing 500kB chunk warning is unchanged).
  • No tests reference WikiOutput.finalText or WikiLibrarianOutput.finalText directly - the field is unchanged in shape, just newly load-bearing for log output.
  • Manual smoke: after merge, watch the log drawer during the next wiki cycle and the next librarian cycle (12h cadence by default). Look for reasoning="..." on the finished thread / librarian finished lines. A short, decision-focused sentence is the expected shape; if the model emits filler ("Great work!") or stays under one sentence, the prompt may need tightening.

Generated by Claude Code

…rawer

Background. Of the three background "what should I do with this
conversation" agents, only the journal worker has historically
surfaced WHY it made the call it did. Its prompt's structured
output carries a `reasoning` field that the cycle driver inlines
on the finished-thread log line ("wrote=false, reasoning=..."),
so a user skimming the log drawer can tell apart "the model
correctly identified this as factual lookup" from "the model
silently dropped a legitimate journal-worthy session". The
per-conversation wiki agent and the wiki librarian, by contrast,
ended each cycle with the prompt instruction "reply with a
single word; the word is discarded". The cycle drivers then
logged only the tool-call count, so the log drawer could not
distinguish "the agent updated the right article" from "the
agent considered three near-duplicate articles, decided to
merge two and leave the third because it covers a different
subject, then did nothing else this cycle". The user-facing
log was missing the explanation half of the decision.

Change. Both wiki prompts now ask the model for a brief one-
or-two-sentence operator summary as the final reply, with
worked examples that cover the user's stated cases (why an
update happened, why a no-op happened, why two similar
articles were considered for merge but left separate). The
cycle drivers normalise that text (collapse whitespace, fall
back to "(none)" on empty) and inline it as `reasoning="..."`
on the cycle's info-tier log line, matching the journal
worker's existing format. The relevant doc comments on
`WikiOutput.finalText` and `WikiLibrarianOutput.finalText`
now describe their operator-facing role rather than calling
the field "discarded", and `docs/dev/wiki.md` gains a Gotcha
entry explaining that "reply with a single word" is load-
bearing only if the loop-side reasoning surface is also
dropped - the two now move together.

The journal worker already had this and is unchanged.
@sysread sysread merged commit 13a8ffd into main May 12, 2026
1 check passed
@sysread sysread deleted the claude/add-worker-explanations-Sx3pV branch May 12, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants