From 8a61ac1ca388bf72e794a15c4b9f96b2059c85f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 9 May 2026 20:17:23 +0000 Subject: [PATCH] Wiki: add librarian agent, strengthen autonomous prompt, fix delete button Three changes addressing observations from production traffic. **Delete button visibility**: Wiki.svelte's local `.danger` CSS reset the button's `color` to red on top of the global red background fill, painting the button text invisible. Drop the local override and trust the global `button.danger` rule (matches the inline note in Memories.svelte's stylesheet section). **Per-conversation wiki agent prompt**: production traffic showed the agent generating one new article per conversation - the per-thread shape biased it hard toward "this conversation is its own topic, write a new article". The prompt now opens with an explicit rule ("UPDATE is the default, CREATE is rare") and workflow step 1 mandates at least two different `wiki_search` query angles before considering `wiki_create`. Most conversations should produce zero edits; chitchat / tactical Q&A should produce nothing at all. Bumps the rationale comment block accordingly. **Wiki librarian agent**: a second background agent whose job is to make the wiki more coherent than it found it. Reads the full alphabetical article list, fact-checks specific claims against conversation history via `conversation_search`, consolidates duplicates via `wiki_update` + `wiki_delete`. Critically, the librarian has NO access to `wiki_create` - new articles flow from the per-conversation agent or the user. The librarian only consolidates what exists. - schema: `profiles.wiki_librarian_last_run_at` plus `claim_wiki_librarian_run(int)` RPC. UPDATE-with-WHERE atomically returns true at most once per min-interval across all of a user's devices. - timing: 12h minimum interval; 1h idle nap so a too-soon device rechecks the claim hourly without spamming. - lease partition `'wiki-librarian'` and lock `nak:wiki-librarian-worker`, distinct from the per- conversation wiki worker so both run alongside each other. - new toolbox `wikiLibrarianToolbox` (search/update/delete + conversation_search) parallel to the existing `wikiToolbox`. - new `AgentRole 'wikiLibrarian'` pinned to deepseek-v4-flash; rationale documented next to the existing wiki entry. - new `wikiLibrarianEnabled` setting, independent toggle from `wikiAutomaticEnabled` so the user can run one without the other. Default-on. Settings -> Wiki has both toggles. User and dev docs updated to describe the librarian, the dual toggles, and the `update-is-default` discipline. https://claude.ai/code/session_015XcR7xzLdij66ZbYERUdLH --- docs/dev/wiki.md | 88 ++++++++-- docs/user/wiki.md | 48 +++++- src/lib/agents/wiki-librarian/agent.ts | 139 +++++++++++++++ src/lib/agents/wiki-librarian/loop.ts | 164 ++++++++++++++++++ src/lib/agents/wiki-librarian/manager.ts | 98 +++++++++++ src/lib/agents/wiki-librarian/prompt.ts | 108 ++++++++++++ src/lib/agents/wiki-librarian/types.ts | 47 +++++ src/lib/agents/wiki-librarian/worker.ts | 211 +++++++++++++++++++++++ src/lib/agents/wiki/prompt.ts | 94 ++++++---- src/lib/models/index.ts | 13 ++ src/lib/state.svelte.ts | 52 ++++++ src/lib/supabase.ts | 40 +++++ src/lib/tools/index.ts | 1 + src/lib/tools/wiki_librarian_toolbox.ts | 51 ++++++ src/screens/Settings.svelte | 44 ++++- src/screens/Wiki.svelte | 8 +- supabase/schema.sql | 50 ++++++ 17 files changed, 1200 insertions(+), 56 deletions(-) create mode 100644 src/lib/agents/wiki-librarian/agent.ts create mode 100644 src/lib/agents/wiki-librarian/loop.ts create mode 100644 src/lib/agents/wiki-librarian/manager.ts create mode 100644 src/lib/agents/wiki-librarian/prompt.ts create mode 100644 src/lib/agents/wiki-librarian/types.ts create mode 100644 src/lib/agents/wiki-librarian/worker.ts create mode 100644 src/lib/tools/wiki_librarian_toolbox.ts diff --git a/docs/dev/wiki.md b/docs/dev/wiki.md index f0f7d39..3c96f90 100644 --- a/docs/dev/wiki.md +++ b/docs/dev/wiki.md @@ -2,8 +2,20 @@ Flat encyclopedic articles about the user. A fourth peer to chats, memories, and journal entries. The user authors articles directly -through the Wiki drawer tab; an autonomous background agent maintains -articles by reading conversations a day after they settle. +through the Wiki drawer tab; two distinct background agents keep them +healthy: + +- The **per-conversation wiki agent** reads settled threads a day + after the newest message and updates / creates articles based on + topics that came up. +- The **wiki librarian** runs every 12 hours, reads the wiki as a + whole, and consolidates duplicates / fact-checks claims against + conversation history. It cannot create new articles; only update + and delete. Cross-device coordination via an atomic claim RPC + (`claim_wiki_librarian_run`). + +Both agents share the encyclopedic-third-person voice and the +"preserve facts unless explicitly contradicted" discipline. ## Role @@ -36,6 +48,10 @@ Schema: `claim_next_pending_wiki_article`, `save_wiki_article_embedding_if_claimed`, `search_wiki_articles_by_embedding`. + Plus, for the librarian: a new `profiles.wiki_librarian_last_run_at` + column and an atomic-claim RPC `claim_wiki_librarian_run(int)` + that returns true at most once per `min_interval_seconds` across + all devices. Data layer (main thread + workers): @@ -64,9 +80,13 @@ Tools: - `src/lib/tools/wiki_create.{schema.,}ts`, `wiki_update.{schema.,}ts`, `wiki_delete.{schema.,}ts` - the agent-only write tools. -- `src/lib/tools/wiki_toolbox.ts` - the agent toolbox that bundles - the four tools above with lazy-loaded schemas, parallel to - `memory_toolbox.ts`. +- `src/lib/tools/wiki_toolbox.ts` - the per-conversation agent's + toolbox; bundles search + create + update + delete with lazy- + loaded schemas. Parallel to `memory_toolbox.ts`. +- `src/lib/tools/wiki_librarian_toolbox.ts` - the librarian's + toolbox; bundles wiki_search + wiki_update + wiki_delete + + conversation_search. **No wiki_create** - the librarian + consolidates rather than invents. Embeddings: @@ -75,11 +95,16 @@ Embeddings: picks it up alongside memories, threads, samskara substrate, and journal entries. -Autonomous agent: +Per-conversation autonomous agent: - `src/lib/agents/wiki/types.ts` - `WikiInput`, `WikiOutput`. - `src/lib/agents/wiki/prompt.ts` - `WIKI_AUTONOMOUS_PROMPT` and - `WIKI_MANUAL_PROMPT`. + `WIKI_MANUAL_PROMPT`. The autonomous prompt biases hard toward + "update over create" - the historical failure mode was one new + article per conversation; the prompt opens with an explicit + rule that update is the default and create is rare, and + workflow step 1 mandates at least two different + wiki_search angles before considering wiki_create. - `src/lib/agents/wiki/agent.ts` - the `WikiAgent` class. Two entry points: `run()` for the worker path and `updateOne()` for the main-thread per-article manual flow. @@ -92,19 +117,52 @@ Autonomous agent: `wiki-worker`. Bubbles `progress: 'processed'` to `emitWikiChange()`. +Librarian: + +- `src/lib/agents/wiki-librarian/types.ts` - + `WikiLibrarianInput` (a snapshot of all articles - id, title, + excerpt), `WikiLibrarianOutput`, plus tunables + `LIBRARIAN_EXCERPT_CHARS = 400` and `LIBRARIAN_MIN_ARTICLES = 3`. +- `src/lib/agents/wiki-librarian/prompt.ts` - `buildWikiLibrarianPrompt` + takes the rendered article list and embeds it into a system + prompt that frames the agent as a librarian (consolidate, + fact-check, tighten boundaries; no wiki_create access). +- `src/lib/agents/wiki-librarian/agent.ts` - the + `WikiLibrarianAgent` class. `run()` reads the snapshot from + input, builds the prompt, runs `runHeadlessToolLoop` against + `wikiLibrarianToolbox`. No per-thread context; threadId in the + ToolContext is set to the empty string (the wiki tools and + conversation_search both ignore it - they're scoped by RLS on + user_id). +- `src/lib/agents/wiki-librarian/loop.ts` - different cycle shape + from the per-conversation loop. No claim of a thread; instead + acquires the lease, then calls `claimWikiLibrarianRun` to gate + on the cross-device interval, then snapshots + `listWikiArticles({limit: 500})` and either runs the agent or + bails on `too-soon` / `too-small`. +- `src/lib/agents/wiki-librarian/worker.ts` - Web Worker entry + point. Lease partition `'wiki-librarian'`. +- `src/lib/agents/wiki-librarian/manager.ts` - `BaseWorkerManager` + subclass. Lock name `nak:wiki-librarian-worker`, logger source + `wiki-librarian-worker`. Defaults to 12h min-interval and a 1h + idle nap; bubbles `progress: 'reviewed'` to `emitWikiChange()`. + Model registry: -- `src/lib/models/index.ts` - `AgentRole` adds `'wiki'`, - `AGENT_MODELS.wiki = 'deepseek-v4-flash'` (same model as journal - and reflection; rationale documented inline above the table). +- `src/lib/models/index.ts` - `AgentRole` adds `'wiki'` and + `'wikiLibrarian'`; `AGENT_MODELS.wiki` and + `AGENT_MODELS.wikiLibrarian` both pinned to + `deepseek-v4-flash` (same family as journal/reflection; + rationale documented inline above the table). Main-thread plumbing: -- `src/lib/state.svelte.ts` - lazy-imports the manager, - `app.wikiAutomaticEnabled`, `setWikiAutomaticEnabled`, - `persistWikiAutomaticEnabled`, hooks into - `applyServerSettings`, `startBackgroundWorkers`, - `setJournalTimezone` (live tz update), and `lock()`. +- `src/lib/state.svelte.ts` - lazy-imports both managers, + `app.wikiAutomaticEnabled` + `app.wikiLibrarianEnabled`, + `setWikiAutomaticEnabled` / `persistWikiAutomaticEnabled` / + `setWikiLibrarianEnabled` / `persistWikiLibrarianEnabled`. Both + toggles are independent in `applyServerSettings`, + `startBackgroundWorkers`, and `lock()`. - `src/lib/routing.svelte.ts` - extends `DrawerTab` with `'wiki'` and `Route` with `wiki_article_id`. - `src/lib/chat-prompt.ts` - `WIKI_BLOCK` after `JOURNAL_BLOCK` diff --git a/docs/user/wiki.md b/docs/user/wiki.md index 466c1a2..3deff0c 100644 --- a/docs/user/wiki.md +++ b/docs/user/wiki.md @@ -133,6 +133,42 @@ You can disable the autonomous agent in **Settings -> Wiki**. Manual edits and the per-article "Ask agent to update" flow keep working when it's off. +## The librarian + +A second background agent - the wiki **librarian** - runs every 12 +hours or so. Its job is different from the per-conversation agent: +instead of reading a single conversation and adding to the wiki, the +librarian looks at the wiki as a whole and tries to make it more +coherent. It can: + +- **Consolidate near-duplicates.** When two articles cover the same + subject under slightly different titles, the librarian merges the + unique facts from one into the other and deletes the redundant + one. +- **Fact-check against conversation history.** When an article makes + a specific claim that might be stale - a job title, a relationship + status, a project status - the librarian searches your past + conversations for evidence and updates the article when it finds a + clear contradiction. +- **Tighten subject boundaries.** When two articles bleed into each + other (a "Maya" article and a "household" article both covering + the same person), the librarian rewrites both so the split is + cleaner. + +The librarian is intentionally constrained: it cannot create new +articles, only consolidate or update existing ones. New articles +flow from the per-conversation agent or from your direct edits. And +it's conservative - if it isn't confident two articles overlap +enough to merge, it leaves them alone. + +The 12-hour minimum interval is enforced atomically across devices +(via a Postgres claim); only one run happens per cycle even if you +have the app open on multiple devices. + +You can disable the librarian independently from the per-conversation +agent in **Settings -> Wiki**. The two toggles are independent: you +can run one without the other. + ## How the assistant uses the wiki Articles are **never** auto-injected into the chat. The assistant @@ -156,6 +192,12 @@ topic (or the title) directly - that's the cue for `wiki_search`. ## Settings controls -The **Settings -> Wiki** pane has one toggle: whether the autonomous -wiki agent runs in the background. The wiki uses the same day boundary -timezone you set on the Journal pane. +The **Settings -> Wiki** pane has two independent toggles: + +- **Automatic articles** - whether the per-conversation wiki agent + runs in the background after threads settle. +- **Librarian** - whether the periodic librarian agent runs in the + background to consolidate and fact-check. + +Both are on by default. The wiki uses the same day boundary timezone +you set on the Journal pane. diff --git a/src/lib/agents/wiki-librarian/agent.ts b/src/lib/agents/wiki-librarian/agent.ts new file mode 100644 index 0000000..7a6c224 --- /dev/null +++ b/src/lib/agents/wiki-librarian/agent.ts @@ -0,0 +1,139 @@ +/** + * Wiki librarian agent. Reads the full alphabetical list of the + * user's wiki articles, builds a compact prompt with title + excerpt + * for each, and runs `runHeadlessToolLoop` against the + * `wikiLibrarianToolbox` (wiki_search + wiki_update + wiki_delete + + * conversation_search). Side effects from those tool calls ARE the + * output; the model's final text is discarded. + * + * No per-thread claim, no entry_date, no terminal-message slicing - + * the librarian operates on the wiki as a whole, on a separate + * cadence from the per-conversation wiki agent. Cross-device + * coordination comes from the lease coordinator + the atomic + * `claim_wiki_librarian_run` RPC the loop checks before instantiating + * the agent. + * + * Pure logic - no leases, no claims, no lifecycle. Those live in + * `./loop.ts` and `./worker.ts`. Same separation as the per- + * conversation wiki agent. + */ +import type { Agent, AgentRunRequest, AgentRunResult } from '../types'; +import type { SupabaseService } from '../../supabase'; +import type { VeniceClient, VeniceMessage } from '../../venice'; +import { wikiLibrarianToolbox } from '../../tools/wiki_librarian_toolbox'; +import { runHeadlessToolLoop } from '../../tools/run'; +import { agentModel } from '../../models'; +import { createLogger } from '../../logger.svelte'; +import { buildWikiLibrarianPrompt } from './prompt'; +import { + LIBRARIAN_EXCERPT_CHARS, + type WikiLibrarianInput, + type WikiLibrarianOutput, +} from './types'; + +const log = createLogger('wiki-librarian-worker'); + +/** + * Render the librarian's snapshot of articles into the bullet list + * the prompt embeds. Each row is "Title - excerpt" so the model can + * scan vertically. Title fences with backticks so a title containing + * stray punctuation reads cleanly. + */ +function renderArticleList( + articles: WikiLibrarianInput['articles'] +): string { + if (articles.length === 0) return '(the wiki is currently empty)'; + return articles + .map((a) => { + const excerpt = a.excerpt + .replace(/\s+/g, ' ') + .trim() + .slice(0, LIBRARIAN_EXCERPT_CHARS); + return `- \`${a.title}\` - ${excerpt || '(empty body)'}`; + }) + .join('\n'); +} + +export class WikiLibrarianAgent + implements Agent +{ + readonly name = 'wiki-librarian'; + readonly model: string; + readonly toolbox = wikiLibrarianToolbox; + + constructor( + private venice: VeniceClient, + private supabase: SupabaseService, + /** + * Optional model override. Defaults to the registry's + * `wikiLibrarian` slot (currently deepseek-v4-flash). Useful + * for tests. + */ + modelId?: string + ) { + this.model = modelId ?? agentModel('wikiLibrarian').id; + } + + async run( + req: AgentRunRequest + ): Promise> { + const signal = req.signal ?? new AbortController().signal; + const articles = req.input.articles; + + if (signal.aborted) { + return { + output: { finalText: '', articleCount: 0 }, + toolCalls: 0, + stoppedReason: 'aborted', + }; + } + + try { + const articleList = renderArticleList(articles); + const promptText = buildWikiLibrarianPrompt({ articleList }); + + log.info( + `librarian reviewing ${articles.length} article(s)` + ); + + const messages: VeniceMessage[] = [ + { role: 'system', content: promptText }, + ]; + + const result = await runHeadlessToolLoop({ + venice: this.venice, + model: this.model, + messages, + toolbox: this.toolbox, + toolCtx: { + supabase: this.supabase, + venice: this.venice, + userId: req.userId, + // The librarian is not thread-scoped. We pass the empty + // string here to satisfy the ToolContext shape; the wiki + // tools and conversation_search both ignore threadId + // (they're scoped by RLS on user_id, not by thread). + threadId: '', + }, + signal, + reasoningEffort: 'low', + }); + + return { + output: { + finalText: result.finalText, + articleCount: articles.length, + }, + toolCalls: result.toolCalls, + stoppedReason: signal.aborted ? 'aborted' : 'done', + }; + } catch (err) { + return { + output: { finalText: '', articleCount: articles.length }, + toolCalls: 0, + stoppedReason: 'error', + error: err instanceof Error ? err.message : String(err), + }; + } + } +} diff --git a/src/lib/agents/wiki-librarian/loop.ts b/src/lib/agents/wiki-librarian/loop.ts new file mode 100644 index 0000000..5063c26 --- /dev/null +++ b/src/lib/agents/wiki-librarian/loop.ts @@ -0,0 +1,164 @@ +/** + * Single-cycle driver for the wiki librarian worker. Different shape + * from the per-conversation wiki / journal / reflection loops: + * + * - No per-thread claim. The librarian operates on the wiki as a + * whole, not on a queue of threads. + * - Cross-device run-coordination via `claim_wiki_librarian_run` - + * the RPC's UPDATE-with-WHERE atomically returns true on at most + * one device when the configured minimum interval has elapsed + * since the last run. + * - Long idle interval. Default min-interval between runs is 12 + * hours; the loop's idle-nap config matches so we don't poll + * more often than we'd consider running. + * - Skips trivially-small wikis (< LIBRARIAN_MIN_ARTICLES) without + * spending Venice tokens. + */ +import type { Agent } from '../types'; +import type { SupabaseService } from '../../supabase'; +import type { LeaseCoordinator } from '../../embeddings/lease'; +import type { + WikiLibrarianInput, + WikiLibrarianOutput, +} from './types'; +import { + LIBRARIAN_EXCERPT_CHARS, + LIBRARIAN_MIN_ARTICLES, +} from './types'; +import { createLogger } from '../../logger.svelte'; + +const log = createLogger('wiki-librarian-worker'); + +export type CycleResult = + | 'acquired-lease' + | 'polling' + | 'too-soon' + | 'too-small' + | 'reviewed' + | 'error'; + +export interface CycleContext { + agent: Agent; + supabase: SupabaseService; + coordinator: LeaseCoordinator; + holderId: string; + userId: string; + /** + * Minimum seconds between successive librarian runs (across all + * devices). The atomic claim RPC enforces this server-side; the + * loop just passes it through. Default 12h via the worker + * defaults. + */ + minIntervalSeconds: number; + signal: AbortSignal; + onLeaseLost: () => void; +} + +export async function runOneCycle(ctx: CycleContext): Promise { + if (ctx.signal.aborted) return 'too-soon'; + + if (!ctx.coordinator.isHolding) { + const acquired = await ctx.coordinator.acquire(); + if (!acquired) return 'polling'; + ctx.coordinator.startHeartbeat(ctx.onLeaseLost); + return 'acquired-lease'; + } + + // Atomic claim. Returns false if another device's run was within + // the interval. The UPDATE-with-WHERE shape means at most one + // device's call ever sees the row update. + let claimed: boolean; + try { + claimed = await ctx.supabase.claimWikiLibrarianRun( + ctx.minIntervalSeconds + ); + } catch { + return 'error'; + } + if (!claimed) return 'too-soon'; + + // Snapshot of every article. Same listing the drawer uses, but + // we cap at 500 (matching listWikiArticles' default) - a librarian + // run on a wiki of more than 500 articles would be unusual and the + // prompt would also overflow. + let articles; + try { + articles = await ctx.supabase.listWikiArticles({ limit: 500 }); + } catch (err) { + log.debug( + 'failed to list wiki articles for librarian', + err instanceof Error ? err.message : String(err) + ); + return 'error'; + } + + if (articles.length < LIBRARIAN_MIN_ARTICLES) { + log.info( + `wiki has ${articles.length} article(s); below LIBRARIAN_MIN_ARTICLES, skipping` + ); + return 'too-small'; + } + + const projection = articles.map((a) => ({ + id: a.id, + title: a.title, + excerpt: a.content.slice(0, LIBRARIAN_EXCERPT_CHARS), + })); + + let runResult; + try { + runResult = await ctx.agent.run({ + input: { articles: projection }, + userId: ctx.userId, + signal: ctx.signal, + }); + } catch (err) { + log.debug( + 'librarian agent threw unexpectedly', + err instanceof Error ? err.message : String(err) + ); + return 'error'; + } + + if (runResult.stoppedReason === 'error') { + log.info( + `librarian reported error: ${runResult.error ?? '(no message)'}` + ); + return 'error'; + } + + log.info( + `librarian finished (${runResult.toolCalls} tool calls over ` + + `${runResult.output.articleCount} articles)` + ); + return 'reviewed'; +} + +export interface NapConfig { + /** Sleep when we don't hold the lease. */ + leasePollMs: number; + /** Sleep after `too-soon` / `too-small` / `reviewed`. */ + idleIntervalMs: number; + errorBackoffMs: number; +} + +/** + * Map cycle outcomes to sleep durations. Most outcomes nap for the + * idle interval - the librarian is meant to run rarely, and waking + * up frequently after a no-op cycle would just re-check the claim + * RPC and discover it's still too soon. + */ +export function napForResult(result: CycleResult, config: NapConfig): number { + switch (result) { + case 'acquired-lease': + return 0; + case 'polling': + return config.leasePollMs; + case 'too-soon': + case 'too-small': + case 'reviewed': + return config.idleIntervalMs; + case 'error': + return config.errorBackoffMs; + } +} diff --git a/src/lib/agents/wiki-librarian/manager.ts b/src/lib/agents/wiki-librarian/manager.ts new file mode 100644 index 0000000..5d670fe --- /dev/null +++ b/src/lib/agents/wiki-librarian/manager.ts @@ -0,0 +1,98 @@ +/** + * Main-thread supervisor for the wiki librarian Web Worker. Same + * shape as the per-conversation wiki manager but uses a different + * cross-tab lock (`nak:wiki-librarian-worker`) and ships a different + * StartOpts / payload (no timezone, but a `minIntervalSeconds` knob + * that gates the librarian's run cadence). + * + * Activation is gated on `app.wikiLibrarianEnabled`; state.svelte.ts + * skips the `start()` call when false and starts/stops the worker + * when the user toggles it. + */ +import type { Session } from '@supabase/supabase-js'; +import { agentModel } from '../../models'; +import { emitWikiChange } from '../../wiki-events'; +import { BaseWorkerManager, type BaseStartOpts } from '../base-manager'; + +export interface WikiLibrarianStartOpts extends BaseStartOpts { + // No live-mutable knobs at the moment - the librarian uses fixed + // timing defaults defined below. If/when we expose a user-tunable + // cadence, this becomes a field plus a setMinInterval() method. + // Marker so this interface stays distinct from BaseStartOpts at + // the type level. + readonly _wikiLibrarian?: never; +} + +/** + * Worker timing defaults. + * + * - leaseTtlSeconds 45 / leaseHeartbeatMs 20_000: same as the other + * agent workers. + * - minIntervalSeconds 12 * 3600 = 43_200: minimum 12h between + * successive runs (across all devices). Enforced atomically by + * `claim_wiki_librarian_run`. + * - leasePollMs 60_000: while we don't hold the lease, check once + * per minute. Cheaper than the per-conversation wiki worker's + * 20s because the librarian rarely has urgent work. + * - idleIntervalMs 3_600_000: after a too-soon / too-small / + * reviewed cycle, sleep an hour before the next claim attempt. + * We wake up roughly every 12 cycles to re-check the claim. + * - errorBackoffMs 30_000: longer than the per-conversation + * workers' 10s; transient failures here are not urgent. + */ +const WORKER_DEFAULTS = { + leaseTtlSeconds: 45, + leaseHeartbeatMs: 20_000, + minIntervalSeconds: 12 * 3600, + leasePollMs: 60_000, + idleIntervalMs: 60 * 60_000, + errorBackoffMs: 30_000, +}; + +class WikiLibrarianManager extends BaseWorkerManager { + protected readonly lockName = 'nak:wiki-librarian-worker'; + protected readonly loggerSource = 'wiki-librarian-worker'; + + protected createWorker(): Worker { + return new Worker(new URL('./worker.ts', import.meta.url), { + type: 'module', + name: 'nak-wiki-librarian', + }); + } + + protected buildStartPayload( + opts: WikiLibrarianStartOpts, + session: Session + ): Record { + return { + supabaseUrl: opts.config.supabaseUrl, + supabaseAnonKey: opts.config.supabaseAnonKey, + accessToken: session.access_token, + refreshToken: session.refresh_token, + userId: session.user.id, + veniceApiKey: opts.config.veniceApiKey, + wikiLibrarianModel: agentModel('wikiLibrarian').id, + ...WORKER_DEFAULTS, + }; + } + + /** + * Bubble `progress: 'reviewed'` to the wiki change-event bus so an + * open Wiki drawer / panel refetches whenever the librarian + * actually moves articles around. Other progress states + * (`too-soon` / `too-small` / `polling`) are silent. + */ + protected onWorkerMessage(data: Record): boolean { + if (data.type === 'progress' && data.result === 'reviewed') { + emitWikiChange(); + return true; + } + return false; + } +} + +/** + * Single app-wide instance. Imported by state.svelte.ts and nowhere + * else. + */ +export const wikiLibrarianManager = new WikiLibrarianManager(); diff --git a/src/lib/agents/wiki-librarian/prompt.ts b/src/lib/agents/wiki-librarian/prompt.ts new file mode 100644 index 0000000..36a8fa4 --- /dev/null +++ b/src/lib/agents/wiki-librarian/prompt.ts @@ -0,0 +1,108 @@ +/** + * System prompt for the wiki librarian. Different from the per- + * conversation wiki agent's prompt in three structural ways: + * + * - Input shape. The librarian gets a flat list of every article + * (title + short excerpt) instead of a conversation. The opening + * paragraph reflects that. + * - Goal. Reorganise, fact-check, consolidate. Not "react to a + * conversation". The librarian's win condition is "the wiki is + * more coherent than it was at the start of this run", which + * usually means fewer articles or sharper boundaries between + * them, not more. + * - Tools. Has wiki_search + wiki_update + wiki_delete + + * conversation_search. NO wiki_create - the librarian does not + * invent new articles. + * + * Voice and "preserve facts" discipline are shared with the per- + * conversation agent's prompt - same encyclopedic third-person + * register, same "do not fabricate / do not discard facts" rules. + */ + +export function buildWikiLibrarianPrompt(opts: { + articleList: string; +}): string { + const { articleList } = opts; + return [ + "You are reviewing the user's personal wiki as the librarian. The", + 'list below is every article in the wiki right now, by title, with', + 'a short excerpt of each. Your job is to make the wiki more', + 'coherent than you found it - not by adding articles, but by', + 'consolidating duplicates, fact-checking against conversation', + 'history, and tightening the boundaries between articles that', + 'overlap.', + '', + 'Articles in the wiki:', + '', + articleList, + '', + '**Tools you can use**:', + '', + '- `wiki_search` - read the full body of any article (search by', + ' title, topic, or natural query).', + '- `conversation_search` - read across the user\'s past', + ' conversations to verify a claim or find context. Use this', + ' when an article makes a specific factual assertion that you', + ' want to corroborate, or when you suspect two articles cover', + ' the same conversation thread under different titles.', + '- `wiki_update` - rewrite an article in place. Preserve facts', + ' that are still accurate; integrate facts from a duplicate', + ' article you intend to delete; correct stale information you', + ' verified is contradicted by recent conversations.', + '- `wiki_delete` - hard-delete an article. ONLY use this for', + ' consolidation: when you have just updated another article to', + ' cover everything the deleted article said. Never delete an', + ' article whose content has not been merged elsewhere.', + '', + '**You DO NOT have wiki_create.** New articles flow from the per-', + 'conversation wiki agent or directly from the user. Your job is', + 'to organise what exists; if you think a topic deserves an', + 'article that is not currently there, leave it alone - the per-', + 'conversation agent will land it the next time the topic comes', + 'up.', + '', + '**Workflow**:', + '', + '1. **Scan the list above for duplicates and near-duplicates.**', + ' Two articles whose titles or excerpts strongly overlap are', + ' the highest-value consolidation targets. Use wiki_search to', + ' read full bodies before deciding. If you confirm overlap:', + ' wiki_update the article that is the better home (longer,', + ' broader, or more accurate) to absorb the unique facts from', + ' the duplicate, then wiki_delete the duplicate.', + '2. **Check for stale facts.** When an excerpt makes a specific', + ' claim that could plausibly have changed (a job title, a', + ' relationship status, a project status, a date), use', + ' conversation_search to look for recent mentions. If you find', + ' a clear contradiction, wiki_update the article. If you find', + ' nothing or only ambiguous evidence, leave it alone.', + '3. **Tighten subject boundaries.** When two articles cover', + ' adjacent topics that confusingly bleed into each other (a', + ' "Maya" article and a "household" article that both cover', + ' the same person), decide which article is the right home', + ' for which facts and wiki_update both to clarify the split.', + ' Do not delete in this case - both articles still have a', + ' reason to exist; you just made the boundary cleaner.', + '', + '**Discipline**:', + '', + '- Be conservative. If you are not sure two articles overlap', + ' enough to merge, leave them alone. False merges destroy', + ' information; missed merges just leave a small redundancy.', + '- Preserve facts. When you wiki_update an article to absorb', + ' another, every concrete fact from the absorbed article must', + ' appear in the merged result unless you are confident it is', + ' wrong (and conversation_search corroborates the contradiction).', + '- Do not fabricate. Only assert facts that appear in the', + ' existing articles, in conversations you searched, or in the', + ' excerpts above. Do not import outside knowledge.', + '- Same voice and tone the wiki uses already: encyclopedic,', + ' third-person, present tense, neutral. Refer to subjects', + " directly (a first name, the project name) rather than \"the", + ' user".', + '', + 'When you have nothing more to do, reply with a single word. The', + 'word is discarded - only the tool calls matter. Zero edits is a', + 'normal outcome on a small or already-coherent wiki.', + ].join('\n'); +} diff --git a/src/lib/agents/wiki-librarian/types.ts b/src/lib/agents/wiki-librarian/types.ts new file mode 100644 index 0000000..355373b --- /dev/null +++ b/src/lib/agents/wiki-librarian/types.ts @@ -0,0 +1,47 @@ +/** + * Request/response shapes for the wiki librarian. Kept in their own + * file so the loop and worker can import the types without reaching + * into the agent class (which pulls in `runHeadlessToolLoop` and the + * `wikiLibrarianToolbox`). + */ + +export interface WikiLibrarianInput { + /** + * Snapshot of every article in the user's wiki at the time the + * librarian was claimed. The agent uses this to plan its review - + * which titles to inspect, which look like duplicates, which + * subjects appear under multiple titles. Each item is a compact + * projection (title + a head-of-content excerpt) rather than the + * full body, so the prompt doesn't blow past the model's window + * when the user has dozens of articles. The agent fetches full + * bodies on demand via wiki_search. + */ + articles: ReadonlyArray<{ + id: string; + title: string; + /** First N chars of content; full body via wiki_search by id. */ + excerpt: string; + }>; +} + +export interface WikiLibrarianOutput { + /** Final post-tool-loop text. Discarded by production callers. */ + finalText: string; + /** Number of articles in the snapshot. Surface for observability. */ + articleCount: number; +} + +/** + * Hard cap on the per-article excerpt the prompt carries. 400 chars + * is roughly enough to convey "what's this article about" without + * blowing past the model's window when the user has a hundred + * articles. Full bodies are still reachable via wiki_search. + */ +export const LIBRARIAN_EXCERPT_CHARS = 400; + +/** + * Skip a librarian run when the user has fewer than this many + * articles. There's nothing to consolidate when the wiki is small, + * and the run would just spend Venice tokens to confirm that. + */ +export const LIBRARIAN_MIN_ARTICLES = 3; diff --git a/src/lib/agents/wiki-librarian/worker.ts b/src/lib/agents/wiki-librarian/worker.ts new file mode 100644 index 0000000..3e88f89 --- /dev/null +++ b/src/lib/agents/wiki-librarian/worker.ts @@ -0,0 +1,211 @@ +/** + * Wiki librarian Web Worker entry point. Same shape as the per- + * conversation wiki worker but runs on a much longer cadence and + * has no per-thread queue. + * + * - Lease partition is `'wiki-librarian'` (distinct from `'wiki'`, + * `'journal'`, etc.) so the librarian can run alongside the + * other workers without contention. + * - The loop's `minIntervalSeconds` defaults to 12h; the idle + * nap defaults to 1h so a device that woke up too-soon checks + * again hourly without spamming the claim RPC. + * - No timezone parameter: the librarian's eligibility predicate + * is "has it been long enough since the last run", not "what + * calendar day are we in". + */ +import { createClient, type SupabaseClient } from '@supabase/supabase-js'; +import { VeniceClient } from '../../venice'; +import { SupabaseService } from '../../supabase'; +import { LeaseCoordinator } from '../../embeddings/lease'; +import { WikiLibrarianAgent } from './agent'; +import { + runOneCycle, + napForResult, + type CycleContext, + type NapConfig, +} from './loop'; + +interface StartMessage { + type: 'start'; + supabaseUrl: string; + supabaseAnonKey: string; + accessToken: string; + refreshToken: string; + userId: string; + veniceApiKey: string; + veniceBaseUrl?: string; + wikiLibrarianModel: string; + holderId: string; + /** Min seconds between successive librarian runs (across devices). */ + minIntervalSeconds: number; + leaseTtlSeconds: number; + leaseHeartbeatMs: number; + leasePollMs: number; + idleIntervalMs: number; + errorBackoffMs: number; +} + +interface StopMessage { + type: 'stop'; +} + +interface SessionMessage { + type: 'session'; + accessToken: string; + refreshToken: string; +} + +type InboundMessage = StartMessage | StopMessage | SessionMessage; + +interface LogOutbound { + type: 'log'; + level: 'info' | 'warn' | 'error'; + message: string; +} + +interface ProgressOutbound { + type: 'progress'; + result: string; +} + +const workerGlobal = self as unknown as DedicatedWorkerGlobalScope; + +function post(msg: LogOutbound | ProgressOutbound): void { + workerGlobal.postMessage(msg); +} + +let currentClient: SupabaseClient | null = null; + +function sleep(ms: number, signal: AbortSignal): Promise { + if (ms <= 0) return Promise.resolve(); + return new Promise((resolve) => { + const t = setTimeout(resolve, ms); + signal.addEventListener( + 'abort', + () => { + clearTimeout(t); + resolve(); + }, + { once: true } + ); + }); +} + +async function runWorker(msg: StartMessage, signal: AbortSignal): Promise { + const client: SupabaseClient = createClient(msg.supabaseUrl, msg.supabaseAnonKey, { + auth: { + persistSession: false, + autoRefreshToken: false, + detectSessionInUrl: false, + }, + }); + const { error: sessionError } = await client.auth.setSession({ + access_token: msg.accessToken, + refresh_token: msg.refreshToken, + }); + if (sessionError) { + post({ + type: 'log', + level: 'error', + message: `wiki librarian setSession failed: ${sessionError.message}`, + }); + return; + } + currentClient = client; + + const supabase = new SupabaseService( + { supabaseUrl: msg.supabaseUrl, supabaseAnonKey: msg.supabaseAnonKey }, + { client } + ); + const venice = new VeniceClient({ + apiKey: msg.veniceApiKey, + baseUrl: msg.veniceBaseUrl, + }); + const coordinator = new LeaseCoordinator( + supabase, + 'wiki-librarian', + msg.holderId, + { + ttlSeconds: msg.leaseTtlSeconds, + heartbeatMs: msg.leaseHeartbeatMs, + } + ); + + const agent = new WikiLibrarianAgent( + venice, + supabase, + msg.wikiLibrarianModel + ); + + const napConfig: NapConfig = { + leasePollMs: msg.leasePollMs, + idleIntervalMs: msg.idleIntervalMs, + errorBackoffMs: msg.errorBackoffMs, + }; + + try { + while (!signal.aborted) { + const ctx: CycleContext = { + agent, + supabase, + coordinator, + holderId: msg.holderId, + userId: msg.userId, + minIntervalSeconds: msg.minIntervalSeconds, + signal, + onLeaseLost: () => { + post({ + type: 'log', + level: 'warn', + message: 'wiki librarian lease lost - re-entering polling', + }); + }, + }; + const result = await runOneCycle(ctx); + post({ type: 'progress', result }); + const nap = napForResult(result, napConfig); + if (nap > 0) await sleep(nap, signal); + } + } finally { + currentClient = null; + await coordinator.release(); + } +} + +const controller = new AbortController(); + +workerGlobal.addEventListener('message', (evt: MessageEvent) => { + const msg = evt.data; + if (msg.type === 'start') { + runWorker(msg, controller.signal) + .catch((err: Error) => { + post({ + type: 'log', + level: 'error', + message: `wiki librarian loop crashed: ${err.message}`, + }); + }) + .finally(() => { + workerGlobal.close(); + }); + } else if (msg.type === 'stop') { + controller.abort(); + } else if (msg.type === 'session') { + if (!currentClient) return; + void currentClient.auth + .setSession({ + access_token: msg.accessToken, + refresh_token: msg.refreshToken, + }) + .catch((err: Error) => { + post({ + type: 'log', + level: 'warn', + message: `forwarded setSession failed: ${err.message}`, + }); + }); + } +}); + +export { sleep }; +export type { StartMessage, StopMessage }; diff --git a/src/lib/agents/wiki/prompt.ts b/src/lib/agents/wiki/prompt.ts index 9a7d407..8bd5eea 100644 --- a/src/lib/agents/wiki/prompt.ts +++ b/src/lib/agents/wiki/prompt.ts @@ -22,18 +22,27 @@ * Refer to the subject directly (their first name, the project * name) rather than "the user" so articles read as encyclopedia * entries rather than session-scoped notes. - * - "Search before write" workflow: every per-topic decision - * starts with wiki_search; existing relevant article -> update - * (preserving facts); no relevant article -> create; create's - * unique-violation -> search again -> update. This is the - * difference between a wiki that grows coherently and one that - * accretes near-duplicates. + * - "Update is the default; create is rare." Earlier production + * traffic showed the agent generating one new article per + * conversation - the per-thread shape biased it toward + * "this conversation is its own topic, write a new article". + * The prompt now leads with the bias hard the other way: + * search broadly with multiple query angles, prefer extending + * a loosely-related existing article over creating a new one, + * and treat zero-edits as a normal outcome on chitchat / + * tactical conversations. Create only fires when wiki_search + * returned nothing on at least two different angles AND the + * subject is one the user is genuinely likely to look up + * later. * - "Preserve facts unless contradicted" is load-bearing: a model * prone to rewriting will overwrite established information * each cycle. The wiki is meant to accrete, not churn. * - Conservative-on-create: the bar is "would the user later look * this up", not "did this come up". A throwaway question about - * the weather should not produce a "weather" article. + * the weather should not produce a "weather" article. A + * conversation that's mostly chitchat or a quick tactical + * exchange may produce zero wiki updates - that is a correct + * outcome, not a failure. */ export const WIKI_AUTONOMOUS_PROMPT = [ "You've just finished the conversation above. Now step out of that", @@ -48,6 +57,15 @@ export const WIKI_AUTONOMOUS_PROMPT = [ 'situation. Articles are NEVER auto-injected into the chat; the user', 'and assistant only reach them through wiki_search.', '', + '**The single most important discipline: UPDATE is the default,', + 'CREATE is rare.** A new article should be the exception, not the', + 'rule. Most conversations should result in zero or one wiki_update', + 'calls and zero wiki_create calls. Conversations that are mostly', + 'chitchat, tactical (a one-off question with a one-off answer), or', + "about something the user is unlikely to look up by name later", + 'should produce no wiki edits at all. That is a correct outcome,', + 'not a failure - reply with a single word and stop.', + '', '**Voice and tone**:', '', '- Encyclopedic, third-person, present tense, neutral. Like the lead', @@ -60,32 +78,48 @@ export const WIKI_AUTONOMOUS_PROMPT = [ '- One topic per article. If a conversation surfaces multiple topics,', ' consider multiple separate updates.', '', - '**Workflow for each topic the conversation surfaced**:', - '', - '1. Call wiki_search with a query that captures the topic. ALWAYS', - ' search first - the unique-key constraint and the wiki-coherence', - " discipline both depend on you knowing what's already there.", - '2. If a relevant article exists, consider wiki_update. **Preserve', - ' every existing fact unless the conversation explicitly', - ' contradicts it.** Add new information; do not rewrite for tone', - ' or condense. The wiki accretes.', - '3. If no relevant article exists, call wiki_create. If create', - " raises a unique-violation (the title collides with one you didn't", - ' surface), call wiki_search again with the exact title and then', - ' wiki_update on the result.', - '4. wiki_delete is only for consolidation: when an article you just', - ' updated now strictly subsumes another one. Never delete on the', - ' basis of "the user said something different today" alone - in', - ' that case, update.', + '**Workflow for each topic the conversation actually deserves an', + 'edit on**:', + '', + '1. **Search broadly first, with multiple query angles.** Call', + ' wiki_search at least twice with DIFFERENT phrasings before you', + ' conclude an article does not already exist. The user may have', + ' an article on the topic under a different title than the one', + ' that came up in conversation - "kombucha" might already exist', + ' as "fermented drinks", a person named "Maya" might be filed', + ' under "household" or by surname. Search for the topic, search', + ' for adjacent topics, search for the specific facts. Do not', + ' skip straight to wiki_create.', + '2. **If anything related exists, prefer wiki_update.** Even a', + ' loosely-related existing article is usually the right home', + ' for new information - extend it rather than fragment the wiki.', + ' A "Maya" article gains a paragraph about her job change; a', + ' "household" article gains a section about Maya. Preserve every', + ' existing fact unless the conversation explicitly contradicts', + ' it. Add new information; do not rewrite for tone or condense.', + '3. **wiki_create is the last resort.** Only call wiki_create', + ' when you have run wiki_search at least twice with different', + ' angles AND none of the results could plausibly be extended to', + ' cover this topic AND the user is genuinely likely to look', + ' this up by name later. A new article should be a new SUBJECT,', + ' not a new conversation summary. If wiki_create raises a', + ' unique-violation, that means a search angle missed - call', + ' wiki_search with the exact title and fall through to', + ' wiki_update.', + '4. wiki_delete is only for consolidation: when an article you', + ' just updated now strictly subsumes another one. Never delete', + ' on the basis of "the user said something different today"', + ' alone - in that case, update.', '', '**Do not fabricate.** Only assert facts that appear in the', - 'conversation above or in existing articles you read via wiki_search.', - "Don't import outside knowledge.", + 'conversation above or in existing articles you read via', + "wiki_search. Don't import outside knowledge.", '', - '**Be conservative.** The bar for creating an article is "would the', - 'user later look this up?", not "did this come up at all?". A', - 'one-off mention does not warrant an article. Fewer high-signal', - 'articles beat many noisy ones.', + '**Be conservative.** Fewer high-signal articles beat many noisy', + 'ones. The bar for updating is "the conversation added durable', + 'information about that subject", not "the conversation mentioned', + 'the subject". The bar for creating is "this is a coherent subject', + 'the user will want to look up by name later", not "this came up".', '', 'When you have nothing more to write, reply with a single word. The', 'word is discarded - only the tool calls matter.', diff --git a/src/lib/models/index.ts b/src/lib/models/index.ts index 3e1d655..a5d33b4 100644 --- a/src/lib/models/index.ts +++ b/src/lib/models/index.ts @@ -299,6 +299,7 @@ export type AgentRole = | 'journal' | 'reflection' | 'wiki' + | 'wikiLibrarian' | 'webSearch' | 'researchDocs' | 'intuition' @@ -389,6 +390,17 @@ export type AgentRole = * big window swallows the conversation, capacity isolation from * foreground tiers, and the JSON pin works on the manual path. * + * wikiLibrarian - deepseek-v4-flash. The wiki agent's bigger + * sibling: every ~12 hours it reads the full alphabetical list + * of articles, fact-checks individual claims via + * conversation_search, and consolidates duplicates / updates + * stale info via wiki_update / wiki_delete. The librarian needs + * the same big context window the per-conversation wiki agent + * uses (the article list + several full articles can run wide) + * and the same response shape (tool-driven, no structured + * final output). Pinned to the same id as `wiki` so a future + * swap of the wiki family flows through both surfaces. + * * webSearch - deepseek-v4-flash. The `web_search` tool's sub- * completion summarises Venice-provided results into 2-4 * sentences with citation markers. Bounded synthesis; the call @@ -434,6 +446,7 @@ export const AGENT_MODELS = { journal: 'deepseek-v4-flash', reflection: 'deepseek-v4-flash', wiki: 'deepseek-v4-flash', + wikiLibrarian: 'deepseek-v4-flash', webSearch: 'deepseek-v4-flash', researchDocs: 'deepseek-v4-flash', intuition: 'mistral-small-3-2-24b-instruct', diff --git a/src/lib/state.svelte.ts b/src/lib/state.svelte.ts index 2ea7a68..bd92b59 100644 --- a/src/lib/state.svelte.ts +++ b/src/lib/state.svelte.ts @@ -119,6 +119,9 @@ const journal = lazyManager(() => const wiki = lazyManager(() => import('./agents/wiki/manager').then((m) => m.wikiManager) ); +const wikiLibrarian = lazyManager(() => + import('./agents/wiki-librarian/manager').then((m) => m.wikiLibrarianManager) +); export type AppPhase = 'loading' | 'setup' | 'locked' | 'unlocked' | 'edit-config'; @@ -198,6 +201,14 @@ interface AppState { * `profiles.settings.wikiAutomaticEnabled` on unlock. */ wikiAutomaticEnabled: boolean; + /** + * Wiki librarian: when true, the periodic librarian agent runs in + * the background (12h minimum interval, atomically gated across + * devices). Independent of `wikiAutomaticEnabled` so the two wiki + * agents can be toggled separately. Default true; overwritten from + * Supabase `profiles.settings.wikiLibrarianEnabled` on unlock. + */ + wikiLibrarianEnabled: boolean; /** * IANA timezone used by the journaling feature to bucket entries. * Seeded from the browser's detected zone on activate() so a @@ -246,6 +257,7 @@ export const app = $state({ notifyOnComplete: false, journalAutomaticEnabled: true, wikiAutomaticEnabled: true, + wikiLibrarianEnabled: true, journalTimezone: detectTimezone(), userName: '', userLocation: '', @@ -371,6 +383,24 @@ export function setWikiAutomaticEnabled(enabled: boolean): void { } } +/** + * Flip the wiki librarian on/off. Independent of the per-conversation + * wiki worker - the user can disable autonomy on one or the other + * without losing the other. + */ +export function setWikiLibrarianEnabled(enabled: boolean): void { + app.wikiLibrarianEnabled = enabled; + if (!app.supabase || !app.config) return; + if (enabled) { + wikiLibrarian.start({ + supabase: app.supabase, + config: app.config, + }); + } else { + wikiLibrarian.stop(); + } +} + /** * Update color mode / accent in memory, apply to the DOM, and cache the * choice so the boot script can restore it instantly next load. Does NOT @@ -545,6 +575,18 @@ export async function persistWikiAutomaticEnabled(enabled: boolean): Promise { + if (!app.supabase) throw new Error(NOT_CONNECTED); + const prev = app.wikiLibrarianEnabled; + setWikiLibrarianEnabled(enabled); + try { + await app.supabase.updateSettings({ wikiLibrarianEnabled: enabled }); + } catch (err) { + setWikiLibrarianEnabled(prev); + throw err; + } +} + /** * Save the journal-day timezone. Caller is responsible for * normalizing user input to a valid IANA name before calling - @@ -625,6 +667,7 @@ export function applyServerSettings(s: UserSettings): void { // seed (also true). setJournalAutomaticEnabled(s.journalAutomaticEnabled ?? true); setWikiAutomaticEnabled(s.wikiAutomaticEnabled ?? true); + setWikiLibrarianEnabled(s.wikiLibrarianEnabled ?? true); if (s.journalTimezone) setJournalTimezone(s.journalTimezone); // Profile: empty string is the "not set" sentinel; always // assign so explicit absence in the blob clears any value @@ -678,6 +721,12 @@ function startBackgroundWorkers(config: AppConfig): void { timezone: app.journalTimezone || null, }); } + if (app.wikiLibrarianEnabled) { + wikiLibrarian.start({ + supabase: app.supabase, + config, + }); + } } /** @@ -711,6 +760,7 @@ export function activate(config: AppConfig, opts: { persist?: boolean } = {}): v app.notifyOnComplete = false; app.journalAutomaticEnabled = true; app.wikiAutomaticEnabled = true; + app.wikiLibrarianEnabled = true; app.journalTimezone = detectTimezone(); app.userName = ''; app.userLocation = ''; @@ -760,6 +810,7 @@ export function lock(): void { samskara.stop(); journal.stop(); wiki.stop(); + wikiLibrarian.stop(); // Tear down the usage poller and wipe the cache so rows billed // against the previous API key don't leak into a subsequent // unlock-with-different-config. @@ -778,6 +829,7 @@ export function lock(): void { // inheriting the previous account's choices. app.journalAutomaticEnabled = true; app.wikiAutomaticEnabled = true; + app.wikiLibrarianEnabled = true; app.journalTimezone = detectTimezone(); // Profile: same rationale - never leak the previous account's // name/location across a lock-then-unlock-as-someone-else flow. diff --git a/src/lib/supabase.ts b/src/lib/supabase.ts index bc4475e..2924beb 100644 --- a/src/lib/supabase.ts +++ b/src/lib/supabase.ts @@ -774,6 +774,15 @@ export interface UserSettings { * button are unaffected by this flag. */ wikiAutomaticEnabled?: boolean; + /** + * Wiki librarian: when true, a separate background agent runs every + * ~12 hours, reads the full wiki, and consolidates duplicates + + * fact-checks against conversation history. Independent of + * `wikiAutomaticEnabled` so the user can disable per-conversation + * autonomy while still getting periodic reorganisation, or vice + * versa. Default-on like the other wiki toggle. + */ + wikiLibrarianEnabled?: boolean; /** * Free-form display name the user wants the model to address them * by. Optional - absent / empty string means "no name supplied, @@ -869,6 +878,9 @@ export function coerceSettings(raw: unknown): UserSettings { if (typeof r.wikiAutomaticEnabled === 'boolean') { out.wikiAutomaticEnabled = r.wikiAutomaticEnabled; } + if (typeof r.wikiLibrarianEnabled === 'boolean') { + out.wikiLibrarianEnabled = r.wikiLibrarianEnabled; + } if ( typeof r.journalTimezone === 'string' && r.journalTimezone.length > 0 && @@ -1056,6 +1068,13 @@ export class SupabaseService { merged.wikiAutomaticEnabled = patch.wikiAutomaticEnabled; } } + if ('wikiLibrarianEnabled' in patch) { + if (patch.wikiLibrarianEnabled === undefined) { + delete merged.wikiLibrarianEnabled; + } else if (typeof patch.wikiLibrarianEnabled === 'boolean') { + merged.wikiLibrarianEnabled = patch.wikiLibrarianEnabled; + } + } if ('journalTimezone' in patch) { if (patch.journalTimezone === undefined) delete merged.journalTimezone; else if ( @@ -3011,6 +3030,27 @@ export class SupabaseService { return data === true; } + /** + * Atomic claim for one wiki-librarian run. Returns true if the + * caller acquired the run (the prior timestamp is older than + * `minIntervalSeconds`, or no run has happened yet); false + * otherwise. The worker calls this BEFORE invoking the agent so + * two devices waking up at the same moment don't both run the + * agent against the same wiki. + * + * The atomicity comes from the SQL UPDATE-with-WHERE shape - the + * update only matches when the interval has passed, so concurrent + * callers either both miss the predicate (one already won) or one + * matches and the others don't. + */ + async claimWikiLibrarianRun(minIntervalSeconds: number): Promise { + const { data, error } = await this.client.rpc('claim_wiki_librarian_run', { + p_min_interval_seconds: minIntervalSeconds, + }); + if (error) throw new SupabaseError(error.message); + return data === true; + } + // Background-worker pipeline -------------------------------------------- // // Methods in this block drive the background workers in diff --git a/src/lib/tools/index.ts b/src/lib/tools/index.ts index caadf07..2d421ff 100644 --- a/src/lib/tools/index.ts +++ b/src/lib/tools/index.ts @@ -529,6 +529,7 @@ export async function executeToolCall( // source paths, not through this barrel. export { memoryToolbox } from './memory_toolbox'; export { wikiToolbox } from './wiki_toolbox'; +export { wikiLibrarianToolbox } from './wiki_librarian_toolbox'; export { recallToolbox } from './recall_toolbox'; export { conversationRecallToolbox } from './conversation_recall_toolbox'; diff --git a/src/lib/tools/wiki_librarian_toolbox.ts b/src/lib/tools/wiki_librarian_toolbox.ts new file mode 100644 index 0000000..78e6379 --- /dev/null +++ b/src/lib/tools/wiki_librarian_toolbox.ts @@ -0,0 +1,51 @@ +/** + * Toolbox for the wiki librarian agent. Bigger surface than + * `wikiToolbox` because the librarian's job is to fact-check and + * reorganise articles against actual conversation history, not just + * land per-conversation edits: + * + * - wiki_search / wiki_update / wiki_delete - the read + mutate + * subset of the wiki agent's toolbox. NO wiki_create: the + * librarian's job is to consolidate what's already there, not + * to invent new articles. New articles flow from the per- + * conversation wiki agent (or the user). If the librarian + * decides an article should be split into two, it edits the + * existing article down to the bigger half and the next per- + * conversation cycle picks up the leftover topic. + * - conversation_search - read-only search over the user's prior + * threads' titles + summaries. Used to verify that a fact + * claimed in an article actually appears in some conversation, + * and to find threads relevant to a topic when the librarian + * is consolidating. + * + * Tool impls are lazy-loaded via `lazyTool`; only the schemas are + * eagerly imported here. Same chunking discipline as + * `memory_toolbox.ts` and `wiki_toolbox.ts` - importing this file + * from a worker bundle should not transitively pull in research_docs + * or other chat-only tools. + */ +import type { Toolbox } from './types'; +import { lazyTool } from './lazy'; +import { wikiSearchSchema } from './wiki_search.schema'; +import { wikiUpdateSchema } from './wiki_update.schema'; +import { wikiDeleteSchema } from './wiki_delete.schema'; +import { conversationSearchSchema } from './conversation_search.schema'; + +export const wikiLibrarianToolbox: Toolbox = { + name: 'wiki-librarian', + description: + "Read, update, and consolidate the signed-in user's wiki articles " + + 'while cross-referencing conversation history to fact-check. ' + + 'Includes wiki_search / wiki_update / wiki_delete plus ' + + 'conversation_search; no wiki_create (consolidation only).', + tools: [ + lazyTool(wikiSearchSchema, () => import('./wiki_search'), 'wikiSearch'), + lazyTool(wikiUpdateSchema, () => import('./wiki_update'), 'wikiUpdate'), + lazyTool(wikiDeleteSchema, () => import('./wiki_delete'), 'wikiDelete'), + lazyTool( + conversationSearchSchema, + () => import('./conversation_search'), + 'conversationSearch' + ), + ], +}; diff --git a/src/screens/Settings.svelte b/src/screens/Settings.svelte index 722dac6..ca935d5 100644 --- a/src/screens/Settings.svelte +++ b/src/screens/Settings.svelte @@ -59,6 +59,7 @@ persistNotifyOnComplete, persistJournalAutomaticEnabled, persistWikiAutomaticEnabled, + persistWikiLibrarianEnabled, persistJournalTimezone, persistSystemPrompts, persistTheme, @@ -196,6 +197,7 @@ // journal pane's timezone preference (one user-tz covers both // background subsystems), so this pane is just the toggle. let wikiAutomaticEnabled = $state(app.wikiAutomaticEnabled); + let wikiLibrarianEnabled = $state(app.wikiLibrarianEnabled); let wikiError = $state(null); let wikiInfo = $state(null); @@ -968,6 +970,22 @@ } } + async function onToggleWikiLibrarian(next: boolean): Promise { + wikiError = null; + wikiInfo = null; + const prev = wikiLibrarianEnabled; + wikiLibrarianEnabled = next; + try { + await persistWikiLibrarianEnabled(next); + wikiInfo = next + ? 'Wiki librarian enabled.' + : 'Wiki librarian disabled. Existing articles are unaffected.'; + } catch (err) { + wikiLibrarianEnabled = prev; + wikiError = err instanceof Error ? err.message : String(err); + } + } + async function onChangeJournalTimezone(next: string): Promise { journalError = null; journalInfo = null; @@ -1451,10 +1469,28 @@ onchange={(e) => onToggleWikiAutomatic(e.currentTarget.checked)} /> - Let Nak's wiki agent maintain articles automatically. - Turning this off stops the background agent; manual edits - and the per-article "ask agent to update" button still - work, and existing articles are untouched. + Let Nak's wiki agent maintain articles automatically as + you chat. Turning this off stops the per-conversation + agent; manual edits and the per-article "ask agent to + update" button still work, and existing articles are + untouched. + + + +

Librarian

+ diff --git a/src/screens/Wiki.svelte b/src/screens/Wiki.svelte index b84b57a..f5a1405 100644 --- a/src/screens/Wiki.svelte +++ b/src/screens/Wiki.svelte @@ -715,10 +715,10 @@ margin-top: 0.5rem; flex-wrap: wrap; } - .danger { - color: var(--error, #b00); - border-color: var(--error, #b00); - } + /* button.danger is styled globally in styles.css (solid red fill + + light text on top). No local override - the previous version + re-set `color` to red on top of the global red background, which + painted the button text invisible. */ .wiki-preview { margin-top: 1rem; padding-top: 1rem; diff --git a/supabase/schema.sql b/supabase/schema.sql index 561d477..46282e8 100644 --- a/supabase/schema.sql +++ b/supabase/schema.sql @@ -5273,6 +5273,56 @@ language sql stable security invoker as $$ limit match_limit $$; +-- Wiki librarian last-run timestamp + atomic-claim RPC. The wiki +-- librarian is a separate background agent that periodically +-- reorganises the user's wiki: consolidating duplicates, fact- +-- checking against conversation history, merging articles that +-- belong together. It runs on a long minimum interval (12 hours +-- by default) - far less often than the per-conversation wiki +-- agent - and there's no per-thread queue. Cross-device +-- coordination needs an atomic "is it time to run yet?" check +-- so two devices that both wake up don't both run the agent. +-- +-- Approach: store the last successful run timestamp on profiles +-- and gate the run via an UPDATE-with-WHERE that only matches +-- when `now() - last_run >= min_interval`. The UPDATE is atomic +-- per row, so only one device's call ever sees the row update; +-- the others see zero rows updated and skip. +alter table public.profiles + add column if not exists wiki_librarian_last_run_at timestamptz; + +-- Atomic claim. Returns true if this caller acquired the run +-- (i.e. the timestamp had aged past p_min_interval_seconds, OR +-- no prior run timestamp was stored), false otherwise. The +-- worker calls this BEFORE running the agent; if it returns +-- false the worker skips this cycle and naps until the next +-- check. +-- +-- security invoker so RLS scopes the row to the calling user. +-- profiles already has a self-update policy. +drop function if exists public.claim_wiki_librarian_run(int); +create or replace function public.claim_wiki_librarian_run( + p_min_interval_seconds int +) returns boolean +language plpgsql security invoker as $$ +declare + updated int; +begin + if auth.uid() is null then + return false; + end if; + update public.profiles + set wiki_librarian_last_run_at = now() + where user_id = auth.uid() + and ( + wiki_librarian_last_run_at is null + or wiki_librarian_last_run_at + < now() - make_interval(secs => p_min_interval_seconds) + ); + get diagnostics updated = row_count; + return updated > 0; +end $$; + -- Atomic assistant-message commit with conflict detection ----------------- -- -- Terminal assistant rows from the chat-loop are written through this