Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 42 additions & 6 deletions docs/dev/cookbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ user can do the same via the Cookbook modal.
Storage layout follows `memories` deliberately — recipes are
user-owned notes that the LLM can also author, same row-level
security posture, same "freeform text column as source of truth"
philosophy. No embeddings pipeline: ILIKE on `title` is fast enough
at cookbook scale (tens to low hundreds of rows per user).
philosophy. Embeddings pipeline added later (see "Embeddings"
below) for the drawer-side recipe search; the LLM tool path
(`recipe_list`, `recipe_search`) still uses ILIKE-on-title and is
unaffected.

## Files

Expand Down Expand Up @@ -207,6 +209,36 @@ sync` is a no-op.
user opted in to keeping every revision so the History panel reads
as a complete diary.

## Embeddings

The drawer's recipe search runs through the same embed-then-merge
pipeline as the wiki and journal sidebars (see `./embeddings.md`).
Added after the original `recipes` design, which omitted embeddings
on the rationale that ILIKE-on-title is enough for a small
single-user cookbook. That holds for the LLM tool path - the model
knows exactly what title it wrote - but the human drawer is a
different problem: a fuzzy query like "fluffy potato side" should
find "Mashed Potatoes" by meaning.

Columns: `embedding vector(2048)`, `embedding_model text`,
`embedding_claim_holder text`, `embedding_claim_expires timestamptz`
on `public.recipes`. A `clear_recipe_embedding_on_change` trigger
nulls the embedding (and the claim) whenever `title`, `cooklang`,
or `source` change; `recipe_update_with_version` updates these
columns inside its own RPC so the trigger fires automatically.

RPCs: `claim_next_pending_recipe`, `save_recipe_embedding_if_claimed`,
`search_recipes_by_embedding`. Same shape as the wiki RPCs and the
same `for update skip locked` claim discipline. The worker adapter
is `src/lib/embeddings/sources/recipes.ts`, registered in
`src/lib/embeddings/worker.ts` alongside the other five sources.

Search wrapper: `SupabaseService.searchRecipes({query, queryEmbedding,
limit})` merges semantic hits (RPC, cosine order) and ILIKE hits
(title only) deduped by id and capped at `limit`. The sidebar
(`src/components/RecipeList.svelte`) calls it on debounced
keystrokes; the LLM tool path keeps using `listRecipes`.

## Interactions

- **Tools** (`./tools.md`) — five recipe tools registered in
Expand Down Expand Up @@ -248,10 +280,14 @@ as a complete diary.
add textarea treats each line as an item — a "saucepan" row would
sit in the shopping list permanently. Instructions and
ingredients are the useful parts for transfer.
- **No embeddings.** ILIKE on `title` is fine at cookbook scale. If
a user ever needs semantic search, the escape hatch mirrors
memories exactly: add `embedding vector(2048)` + claim columns +
the worker-claim RPC pattern. Don't pre-emptively wire it.
- **Drawer search is semantic; LLM tool path is not.** The
embedding pipeline (added after the original "no embeddings"
design - see "Embeddings" below) feeds `searchRecipes` only.
The `recipe_list` and `recipe_search` tools the model uses
still go through ILIKE-on-title. Two reasons: the model knows
exactly what title it just wrote so meaning matching adds no
value on the tool path, and keeping the tool's behaviour
deterministic avoids fuzz on a path the agent reasons about.
- **Cooklang source is the source of truth.** The parsed
ingredients list you see in the render is derived, not stored.
This means a future parser bug is a pure read-path issue —
Expand Down
15 changes: 12 additions & 3 deletions docs/dev/embeddings.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,11 @@ under-ranked until the worker catches up.
- `src/lib/embeddings/types.ts` — `EmbeddingSource` interface
and shared constants.
- `src/lib/embeddings/sources/memories.ts`,
`sources/threads.ts` — per-table adapters. Each knows how to
claim one pending row, build the input string for Venice,
and save the result under a guard.
`sources/threads.ts`, `sources/journal.ts`,
`sources/wiki.ts`, `sources/samskara-substrate.ts`,
`sources/recipes.ts` — per-table adapters. Each knows how
to claim one pending row, build the input string for
Venice, and save the result under a guard.
- `supabase/schema.sql` — `worker_leases` table, lease RPCs,
`claim_next_pending_memory` /
`claim_next_pending_thread_embedding`,
Expand Down Expand Up @@ -165,6 +167,13 @@ extension.
`clear_journal_embedding_on_change` trigger fires when
`content | topics | mood` change so an edit reselects
the row. See `./journal.md`.
- **Cookbook** — `recipes` joined the registered-source list
so the drawer's recipe search can rank by meaning. The
`clear_recipe_embedding_on_change` trigger fires when
`title | cooklang | source` change. The LLM-facing
`recipe_list` tool still runs ILIKE-on-title only - the
embedding pipeline is for the human drawer search. See
`./cookbook.md`.
- **Auth-session** — worker startup is gated on an active
Supabase session; `manager.start()` pulls the session from
the Supabase client and passes tokens to the Worker. A
Expand Down
4 changes: 2 additions & 2 deletions docs/user/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ You can reach these pages two ways:
alongside your conversations, plus space for your own entries.
- [Cookbook](./cookbook.md) — a personal store for Cooklang recipes,
with a dedicated modal and a tab in the conversation drawer.
- [Search](./search.md) — finding a thread or a message across your
history.
- [Search](./search.md) — finding a thread, recipe, journal day, or
wiki article by meaning across your history.
- [Keyboard shortcuts](./shortcuts.md) — the keys that save you the
most time.

Expand Down
75 changes: 67 additions & 8 deletions docs/user/search.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,78 @@
# Search

Finding a thread or a specific message across your history. The search
field in the drawer matches thread titles and message bodies.
Every drawer tab in Nak - Chats, Recipes, Journal, Wiki - has a
search box at the top. They all work the same way: type a query,
wait briefly while Nak embeds your phrase, and the matches come
back in order of closeness rather than alphabetical or
chronological order.

## Searching from the drawer
## How matching works

## What matches and what doesn't
Each search runs in two passes and merges the results:

## Jumping to a matched message
1. **Meaning match.** Your query is sent to the embedding model
and compared to a stored vector for every item in that tab.
The closer the meaning, the higher the rank. This is how
"fluffy potato side" finds *Mashed Potatoes* and "anxious about
work last week" finds the right journal day.
2. **Word match.** A simple case-insensitive substring search
over the obvious fields (thread title, recipe title, article
body, journal content / mood). These appear after the meaning
matches, so a freshly added item that hasn't been embedded yet
is still findable - just not yet rankable by meaning.

The two passes are deduplicated by id and capped per tab.

## Loading feedback

When you type, Nak debounces briefly (so it isn't embedding every
keystroke), then shows a small scanner animation in place of the
list while the embedding round-trip and the database query run.
The actual results appear once they arrive, replacing the scanner.
Searches typically complete in under a second.

If your network is offline, or the embedding model is briefly
unreachable, Nak falls back to the substring-only pass quietly -
your search will still return matches, just ranked by recency
rather than meaning.

## Ordering

| Tab | Empty query | Active query |
| --- | --- | --- |
| **Chats** | Recent / Older / Archived buckets | Exact title hits first, then by similarity |
| **Recipes** | Most-recent or by rating (your choice) | By similarity to your query |
| **Journal** | Days newest-first | Days ranked by their best-matching entry |
| **Wiki** | Alphabetical by title | By similarity to your query |

The Wiki list switches from alphabetical to relevance order only
while there's text in the search box. Clearing the box returns it
to the alphabetical browse view.

The Recipe list hides its sort picker while you're searching - the
similarity ranking is the sort during a search.

The Journal list rolls multiple matching entries from the same day
into one row; days are ordered by their single best match, so a
day with one strong hit can rank above a day with three weak ones.

## Why a search might miss something

- The item was just added and the background embedding worker
hasn't reached it yet. Substring-on-the-obvious-field still
catches it; meaning matches will catch up within minutes.
- The phrase you typed is too short or too generic for the model
to anchor on. Try adding a noun or a context word.
- You typed a long query and the substring pass found nothing
literal; meaning matches still ranked but none were close.

## Where to go next

- [Threads](./threads.md) — the drawer that hosts the search field.
- [Keyboard shortcuts](./shortcuts.md) — the shortcut for focusing
search.
- [Chats](./threads.md) - the conversation drawer.
- [Recipes](./cookbook.md) - the cookbook drawer.
- [Journal](./journal.md) - the journal drawer.
- [Wiki](./wiki.md) - the wiki drawer.
- [Keyboard shortcuts](./shortcuts.md) - focusing the search box.

---
Back to the [index](./README.md).
Loading
Loading