Wiki + librarian workers: emit operator-facing reasoning to the log drawer#34
Merged
Merged
Conversation
…rawer
Background. Of the three background "what should I do with this
conversation" agents, only the journal worker has historically
surfaced WHY it made the call it did. Its prompt's structured
output carries a `reasoning` field that the cycle driver inlines
on the finished-thread log line ("wrote=false, reasoning=..."),
so a user skimming the log drawer can tell apart "the model
correctly identified this as factual lookup" from "the model
silently dropped a legitimate journal-worthy session". The
per-conversation wiki agent and the wiki librarian, by contrast,
ended each cycle with the prompt instruction "reply with a
single word; the word is discarded". The cycle drivers then
logged only the tool-call count, so the log drawer could not
distinguish "the agent updated the right article" from "the
agent considered three near-duplicate articles, decided to
merge two and leave the third because it covers a different
subject, then did nothing else this cycle". The user-facing
log was missing the explanation half of the decision.
Change. Both wiki prompts now ask the model for a brief one-
or-two-sentence operator summary as the final reply, with
worked examples that cover the user's stated cases (why an
update happened, why a no-op happened, why two similar
articles were considered for merge but left separate). The
cycle drivers normalise that text (collapse whitespace, fall
back to "(none)" on empty) and inline it as `reasoning="..."`
on the cycle's info-tier log line, matching the journal
worker's existing format. The relevant doc comments on
`WikiOutput.finalText` and `WikiLibrarianOutput.finalText`
now describe their operator-facing role rather than calling
the field "discarded", and `docs/dev/wiki.md` gains a Gotcha
entry explaining that "reply with a single word" is load-
bearing only if the loop-side reasoning surface is also
dropped - the two now move together.
The journal worker already had this and is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SYNOPSIS
The wiki per-conversation worker and the wiki librarian now end each cycle with a one-or-two-sentence operator summary of WHY they edited / merged / skipped, surfaced as
reasoning="..."on the log-drawer line. The journal worker already did this; this PR lifts the other two to parity.PURPOSE
Currently the wiki and librarian cycle logs show only a tool-call count - "finished thread X (3 tool calls over 25 messages)", "librarian finished (5 tool calls over 87 articles)". A user skimming the log drawer cannot tell:
The journal worker already solves this via its structured
{worthy, reasoning, entry}output, which the loop inlines asreasoning="...". The wiki and librarian are tool-driven (no JSON output), so historically their prompts closed with "reply with a single word; the word is discarded" purely to terminate the tool loop. The final-text channel was free; the prompt just wasn't asking for anything useful on it.DESCRIPTION
Layer 1 - how existing code behaves
{worthy, reasoning, entry}from a JSON completion.JournalOutput.reasoningis set from that field; the loop logs(wrote=true/false, ..., reasoning="...")at info tier.runHeadlessToolLoopand returnsWikiOutput.finalText- the model's final assistant turn after the tool loop terminates. The prompt's closing block says "reply with a single word; the word is discarded". The loop logs(N tool calls over M messages)with noreasoning=field;finalTextis captured into the output struct but never surfaced.runHeadlessToolLoop,WikiLibrarianOutput.finalText, "reply with a single word", loop logs(N tool calls over M articles).Layer 2 - what this PR changes
src/lib/agents/wiki/prompt.ts) - replaces the "single word" closing block with a "Final reply: one or two sentences explaining your choices" block. Three worked examples cover update, no-op-because-generic-Q&A, and no-op-because-not-user-centric. Asks for plain text, no Markdown, under two sentences, and explicitly tells the model the text surfaces in the user's log drawer.src/lib/agents/wiki/loop.ts) - whitespace-normalisesrunResult.output.finalText(collapses\s+to a single space, trims), falls back to(none)on empty, and inlines it asreasoning="..."on the existingfinished thread ...info log. Matches the journal worker's format exactly so grep / scrollback search lines up across all three.src/lib/agents/wiki-librarian/prompt.ts) - same change. Examples deliberately call out the "considered for merge but left separate" case, which is the user's stated motivating example - that decision is currently invisible in the logs.src/lib/agents/wiki-librarian/loop.ts) - samereasoning="..."surface on thelibrarian finished ...line.src/lib/agents/wiki/types.ts,src/lib/agents/wiki-librarian/types.ts) -finalTextdoc comments no longer call the field "discarded by production callers"; they describe the operator-facing role and reference the prompt's "Final reply" block.src/lib/agents/wiki/agent.ts,src/lib/agents/wiki-librarian/agent.ts) - updated to match.docs/dev/wiki.md) - the "Entry points" section'sWikiAgent.runparagraph now describes the operator-summary contract instead of calling final-text discarded. A new Gotcha entry notes that the prompt's "Final reply" instruction and the loop'sreasoning="..."surface move together - reverting one without the other silently drops a user-facing diagnostic.The journal worker is not touched - it already does this.
Notes for AI reviewers:
\s+ -> ' 'whitespace normalisation in both loops is deliberate. Models occasionally emit a multi-line final turn even when asked for one or two sentences; the single-line log convention is load-bearing and a stray newline would break the(field=value, field=value) "title"shape the journal worker established.|| '(none)'sentinel is also deliberate. An emptyfinalTextshouldn't happen in production (the prompt now mandates a sentence), but a danglingreasoning=""reads as a logger bug; the sentinel makes a missing summary self-explanatory.Test plan
mise run checkgreen: 1069 tests passed, build clean (the pre-existing 500kB chunk warning is unchanged).WikiOutput.finalTextorWikiLibrarianOutput.finalTextdirectly - the field is unchanged in shape, just newly load-bearing for log output.reasoning="..."on thefinished thread/librarian finishedlines. A short, decision-focused sentence is the expected shape; if the model emits filler ("Great work!") or stays under one sentence, the prompt may need tightening.Generated by Claude Code