Wiki: add librarian, strengthen prompt, fix delete button

docs/dev/wiki.md

@@ -2,8 +2,20 @@
 
 Flat encyclopedic articles about the user. A fourth peer to chats,
 memories, and journal entries. The user authors articles directly
-through the Wiki drawer tab; an autonomous background agent maintains
-articles by reading conversations a day after they settle.
+through the Wiki drawer tab; two distinct background agents keep them
+healthy:
+
+- The **per-conversation wiki agent** reads settled threads a day
+  after the newest message and updates / creates articles based on
+  topics that came up.
+- The **wiki librarian** runs every 12 hours, reads the wiki as a
+  whole, and consolidates duplicates / fact-checks claims against
+  conversation history. It cannot create new articles; only update
+  and delete. Cross-device coordination via an atomic claim RPC
+  (`claim_wiki_librarian_run`).
+
+Both agents share the encyclopedic-third-person voice and the
+"preserve facts unless explicitly contradicted" discipline.
 
 ## Role
 
@@ -36,6 +48,10 @@ Schema:
   `claim_next_pending_wiki_article`,
   `save_wiki_article_embedding_if_claimed`,
   `search_wiki_articles_by_embedding`.
+  Plus, for the librarian: a new `profiles.wiki_librarian_last_run_at`
+  column and an atomic-claim RPC `claim_wiki_librarian_run(int)`
+  that returns true at most once per `min_interval_seconds` across
+  all devices.
 
 Data layer (main thread + workers):
 
@@ -64,9 +80,13 @@ Tools:
 - `src/lib/tools/wiki_create.{schema.,}ts`,
   `wiki_update.{schema.,}ts`, `wiki_delete.{schema.,}ts` - the
   agent-only write tools.
-- `src/lib/tools/wiki_toolbox.ts` - the agent toolbox that bundles
-  the four tools above with lazy-loaded schemas, parallel to
-  `memory_toolbox.ts`.
+- `src/lib/tools/wiki_toolbox.ts` - the per-conversation agent's
+  toolbox; bundles search + create + update + delete with lazy-
+  loaded schemas. Parallel to `memory_toolbox.ts`.
+- `src/lib/tools/wiki_librarian_toolbox.ts` - the librarian's
+  toolbox; bundles wiki_search + wiki_update + wiki_delete +
+  conversation_search. **No wiki_create** - the librarian
+  consolidates rather than invents.
 
 Embeddings:
 
@@ -75,11 +95,16 @@ Embeddings:
   picks it up alongside memories, threads, samskara substrate, and
   journal entries.
 
-Autonomous agent:
+Per-conversation autonomous agent:
 
 - `src/lib/agents/wiki/types.ts` - `WikiInput`, `WikiOutput`.
 - `src/lib/agents/wiki/prompt.ts` - `WIKI_AUTONOMOUS_PROMPT` and
-  `WIKI_MANUAL_PROMPT`.
+  `WIKI_MANUAL_PROMPT`. The autonomous prompt biases hard toward
+  "update over create" - the historical failure mode was one new
+  article per conversation; the prompt opens with an explicit
+  rule that update is the default and create is rare, and
+  workflow step 1 mandates at least two different
+  wiki_search angles before considering wiki_create.
 - `src/lib/agents/wiki/agent.ts` - the `WikiAgent` class. Two
   entry points: `run()` for the worker path and `updateOne()` for
   the main-thread per-article manual flow.
@@ -92,19 +117,52 @@ Autonomous agent:
   `wiki-worker`. Bubbles `progress: 'processed'` to
   `emitWikiChange()`.
 
+Librarian:
+
+- `src/lib/agents/wiki-librarian/types.ts` -
+  `WikiLibrarianInput` (a snapshot of all articles - id, title,
+  excerpt), `WikiLibrarianOutput`, plus tunables
+  `LIBRARIAN_EXCERPT_CHARS = 400` and `LIBRARIAN_MIN_ARTICLES = 3`.
+- `src/lib/agents/wiki-librarian/prompt.ts` - `buildWikiLibrarianPrompt`
+  takes the rendered article list and embeds it into a system
+  prompt that frames the agent as a librarian (consolidate,
+  fact-check, tighten boundaries; no wiki_create access).
+- `src/lib/agents/wiki-librarian/agent.ts` - the
+  `WikiLibrarianAgent` class. `run()` reads the snapshot from
+  input, builds the prompt, runs `runHeadlessToolLoop` against
+  `wikiLibrarianToolbox`. No per-thread context; threadId in the
+  ToolContext is set to the empty string (the wiki tools and
+  conversation_search both ignore it - they're scoped by RLS on
+  user_id).
+- `src/lib/agents/wiki-librarian/loop.ts` - different cycle shape
+  from the per-conversation loop. No claim of a thread; instead
+  acquires the lease, then calls `claimWikiLibrarianRun` to gate
+  on the cross-device interval, then snapshots
+  `listWikiArticles({limit: 500})` and either runs the agent or
+  bails on `too-soon` / `too-small`.
+- `src/lib/agents/wiki-librarian/worker.ts` - Web Worker entry
+  point. Lease partition `'wiki-librarian'`.
+- `src/lib/agents/wiki-librarian/manager.ts` - `BaseWorkerManager`
+  subclass. Lock name `nak:wiki-librarian-worker`, logger source
+  `wiki-librarian-worker`. Defaults to 12h min-interval and a 1h
+  idle nap; bubbles `progress: 'reviewed'` to `emitWikiChange()`.
+
 Model registry:
 
-- `src/lib/models/index.ts` - `AgentRole` adds `'wiki'`,
-  `AGENT_MODELS.wiki = 'deepseek-v4-flash'` (same model as journal
-  and reflection; rationale documented inline above the table).
+- `src/lib/models/index.ts` - `AgentRole` adds `'wiki'` and
+  `'wikiLibrarian'`; `AGENT_MODELS.wiki` and
+  `AGENT_MODELS.wikiLibrarian` both pinned to
+  `deepseek-v4-flash` (same family as journal/reflection;
+  rationale documented inline above the table).
 
 Main-thread plumbing:
 
-- `src/lib/state.svelte.ts` - lazy-imports the manager,
-  `app.wikiAutomaticEnabled`, `setWikiAutomaticEnabled`,
-  `persistWikiAutomaticEnabled`, hooks into
-  `applyServerSettings`, `startBackgroundWorkers`,
-  `setJournalTimezone` (live tz update), and `lock()`.
+- `src/lib/state.svelte.ts` - lazy-imports both managers,
+  `app.wikiAutomaticEnabled` + `app.wikiLibrarianEnabled`,
+  `setWikiAutomaticEnabled` / `persistWikiAutomaticEnabled` /
+  `setWikiLibrarianEnabled` / `persistWikiLibrarianEnabled`. Both
+  toggles are independent in `applyServerSettings`,
+  `startBackgroundWorkers`, and `lock()`.
 - `src/lib/routing.svelte.ts` - extends `DrawerTab` with
   `'wiki'` and `Route` with `wiki_article_id`.
 - `src/lib/chat-prompt.ts` - `WIKI_BLOCK` after `JOURNAL_BLOCK`

docs/user/wiki.md

@@ -133,6 +133,42 @@ You can disable the autonomous agent in **Settings -> Wiki**. Manual
 edits and the per-article "Ask agent to update" flow keep working
 when it's off.
 
+## The librarian
+
+A second background agent - the wiki **librarian** - runs every 12
+hours or so. Its job is different from the per-conversation agent:
+instead of reading a single conversation and adding to the wiki, the
+librarian looks at the wiki as a whole and tries to make it more
+coherent. It can:
+
+- **Consolidate near-duplicates.** When two articles cover the same
+  subject under slightly different titles, the librarian merges the
+  unique facts from one into the other and deletes the redundant
+  one.
+- **Fact-check against conversation history.** When an article makes
+  a specific claim that might be stale - a job title, a relationship
+  status, a project status - the librarian searches your past
+  conversations for evidence and updates the article when it finds a
+  clear contradiction.
+- **Tighten subject boundaries.** When two articles bleed into each
+  other (a "Maya" article and a "household" article both covering
+  the same person), the librarian rewrites both so the split is
+  cleaner.
+
+The librarian is intentionally constrained: it cannot create new
+articles, only consolidate or update existing ones. New articles
+flow from the per-conversation agent or from your direct edits. And
+it's conservative - if it isn't confident two articles overlap
+enough to merge, it leaves them alone.
+
+The 12-hour minimum interval is enforced atomically across devices
+(via a Postgres claim); only one run happens per cycle even if you
+have the app open on multiple devices.
+
+You can disable the librarian independently from the per-conversation
+agent in **Settings -> Wiki**. The two toggles are independent: you
+can run one without the other.
+
 ## How the assistant uses the wiki
 
 Articles are **never** auto-injected into the chat. The assistant
@@ -156,6 +192,12 @@ topic (or the title) directly - that's the cue for `wiki_search`.
 
 ## Settings controls
 
-The **Settings -> Wiki** pane has one toggle: whether the autonomous
-wiki agent runs in the background. The wiki uses the same day boundary
-timezone you set on the Journal pane.
+The **Settings -> Wiki** pane has two independent toggles:
+
+- **Automatic articles** - whether the per-conversation wiki agent
+  runs in the background after threads settle.
+- **Librarian** - whether the periodic librarian agent runs in the
+  background to consolidate and fact-check.
+
+Both are on by default. The wiki uses the same day boundary timezone
+you set on the Journal pane.

src/lib/agents/wiki-librarian/agent.ts

@@ -0,0 +1,139 @@
+/**
+ * Wiki librarian agent. Reads the full alphabetical list of the
+ * user's wiki articles, builds a compact prompt with title + excerpt
+ * for each, and runs `runHeadlessToolLoop` against the
+ * `wikiLibrarianToolbox` (wiki_search + wiki_update + wiki_delete +
+ * conversation_search). Side effects from those tool calls ARE the
+ * output; the model's final text is discarded.
+ *
+ * No per-thread claim, no entry_date, no terminal-message slicing -
+ * the librarian operates on the wiki as a whole, on a separate
+ * cadence from the per-conversation wiki agent. Cross-device
+ * coordination comes from the lease coordinator + the atomic
+ * `claim_wiki_librarian_run` RPC the loop checks before instantiating
+ * the agent.
+ *
+ * Pure logic - no leases, no claims, no lifecycle. Those live in
+ * `./loop.ts` and `./worker.ts`. Same separation as the per-
+ * conversation wiki agent.
+ */
+import type { Agent, AgentRunRequest, AgentRunResult } from '../types';
+import type { SupabaseService } from '../../supabase';
+import type { VeniceClient, VeniceMessage } from '../../venice';
+import { wikiLibrarianToolbox } from '../../tools/wiki_librarian_toolbox';
+import { runHeadlessToolLoop } from '../../tools/run';
+import { agentModel } from '../../models';
+import { createLogger } from '../../logger.svelte';
+import { buildWikiLibrarianPrompt } from './prompt';
+import {
+  LIBRARIAN_EXCERPT_CHARS,
+  type WikiLibrarianInput,
+  type WikiLibrarianOutput,
+} from './types';
+
+const log = createLogger('wiki-librarian-worker');
+
+/**
+ * Render the librarian's snapshot of articles into the bullet list
+ * the prompt embeds. Each row is "Title - excerpt" so the model can
+ * scan vertically. Title fences with backticks so a title containing
+ * stray punctuation reads cleanly.
+ */
+function renderArticleList(
+  articles: WikiLibrarianInput['articles']
+): string {
+  if (articles.length === 0) return '(the wiki is currently empty)';
+  return articles
+    .map((a) => {
+      const excerpt = a.excerpt
+        .replace(/\s+/g, ' ')
+        .trim()
+        .slice(0, LIBRARIAN_EXCERPT_CHARS);
+      return `- \`${a.title}\` - ${excerpt || '(empty body)'}`;
+    })
+    .join('\n');
+}
+
+export class WikiLibrarianAgent
+  implements Agent<WikiLibrarianInput, WikiLibrarianOutput>
+{
+  readonly name = 'wiki-librarian';
+  readonly model: string;
+  readonly toolbox = wikiLibrarianToolbox;
+
+  constructor(
+    private venice: VeniceClient,
+    private supabase: SupabaseService,
+    /**
+     * Optional model override. Defaults to the registry's
+     * `wikiLibrarian` slot (currently deepseek-v4-flash). Useful
+     * for tests.
+     */
+    modelId?: string
+  ) {
+    this.model = modelId ?? agentModel('wikiLibrarian').id;
+  }
+
+  async run(
+    req: AgentRunRequest<WikiLibrarianInput>
+  ): Promise<AgentRunResult<WikiLibrarianOutput>> {
+    const signal = req.signal ?? new AbortController().signal;
+    const articles = req.input.articles;
+
+    if (signal.aborted) {
+      return {
+        output: { finalText: '', articleCount: 0 },
+        toolCalls: 0,
+        stoppedReason: 'aborted',
+      };
+    }
+
+    try {
+      const articleList = renderArticleList(articles);
+      const promptText = buildWikiLibrarianPrompt({ articleList });
+
+      log.info(
+        `librarian reviewing ${articles.length} article(s)`
+      );
+
+      const messages: VeniceMessage[] = [
+        { role: 'system', content: promptText },
+      ];
+
+      const result = await runHeadlessToolLoop({
+        venice: this.venice,
+        model: this.model,
+        messages,
+        toolbox: this.toolbox,
+        toolCtx: {
+          supabase: this.supabase,
+          venice: this.venice,
+          userId: req.userId,
+          // The librarian is not thread-scoped. We pass the empty
+          // string here to satisfy the ToolContext shape; the wiki
+          // tools and conversation_search both ignore threadId
+          // (they're scoped by RLS on user_id, not by thread).
+          threadId: '',
+        },
+        signal,
+        reasoningEffort: 'low',
+      });
+
+      return {
+        output: {
+          finalText: result.finalText,
+          articleCount: articles.length,
+        },
+        toolCalls: result.toolCalls,
+        stoppedReason: signal.aborted ? 'aborted' : 'done',
+      };
+    } catch (err) {
+      return {
+        output: { finalText: '', articleCount: articles.length },
+        toolCalls: 0,
+        stoppedReason: 'error',
+        error: err instanceof Error ? err.message : String(err),
+      };
+    }
+  }
+}