Skip to content

feat: add memory system for persistent context across conversations#14

Open
42tg wants to merge 158 commits intomainfrom
feat/memory-system
Open

feat: add memory system for persistent context across conversations#14
42tg wants to merge 158 commits intomainfrom
feat/memory-system

Conversation

@42tg
Copy link
Copy Markdown
Owner

@42tg 42tg commented Mar 19, 2026

Summary

Adds a full-stack memory feature inspired by Claude Code's recently shipped memory system. Memories persist context across conversations — things like coding preferences, architectural decisions, project conventions, and facts that the agent should remember.

  • Contracts: Effect Schema types for Memory entity, 5 categories (preference/pattern/decision/fact/convention), project + global scopes, manual + auto sources, and 7 WS input/result schemas
  • Server: SQLite migration with FTS5 full-text search, sync triggers, composite indexes; MemoryRepository service + layer following the ReviewCommentRepository pattern; 7 WebSocket RPC handlers
  • Web: React Query hooks with cache invalidation; MemoryPanel with search + category filters + CRUD; MemoryCreateDialog; MemoryBadge in ChatHeader; Sidebar sheet integration

Design decisions

Decision Rationale
SQLite FTS5 (no vector DB) Keeps deployment simple, no external deps. Hybrid BM25 keyword + recency scoring covers v1 well
Project + global scopes Global memories carry across projects; project-scoped stay local
sanitizeFts5Query() wraps tokens in double-quotes Prevents FTS5 operator injection while still supporting multi-word search
Composite covering index on (project_id, archived_at, updated_at) Optimizes the most common query: list active memories for a project
Manual source only for now Auto-extraction from completed turns is designed but deferred

Research references

Studied implementations from Claude Code (file-based CLAUDE.md), OpenClaw (hybrid vector+BM25), ClawMem (hooks-based), Copilot (citation-verified), and Cursor (phase-based memory bank) to inform the design.

Files changed (18 files, +1,631 lines)

Contracts (packages/contracts/src/)

  • memory.ts — Domain schemas: MemoryId, Memory entity, WS inputs/results
  • ws.ts — 7 memory WS methods + request body union entries
  • ipc.ts — Memory namespace on NativeApi interface
  • index.ts — Barrel export

Server (apps/server/src/)

  • persistence/Migrations/019_Memories.ts — SQLite table + FTS5 virtual table + triggers + indexes
  • persistence/Migrations.ts — Migration registration
  • persistence/Services/MemoryRepository.ts — Service interface
  • persistence/Layers/MemoryRepository.ts — Full implementation with FTS5 search
  • serverLayers.ts — Layer registration
  • wsServer.ts — 7 WS route handlers

Web (apps/web/src/)

  • lib/memoryReactQuery.ts — Query keys, query options, mutation options
  • components/MemoryPanel.tsx — Main panel with search, filters, memory cards
  • components/MemoryCreateDialog.tsx — Dialog form for manual memory creation
  • components/MemoryBadge.tsx — Badge showing memory count in ChatHeader
  • components/Sidebar.tsx — Memory sheet integration
  • components/chat/ChatHeader.tsx — MemoryBadge integration
  • components/ChatView.tsx — Pass projectId to ChatHeader
  • wsNativeApi.ts — Memory namespace transport bindings

Test plan

  • Create a manual memory via the dialog — verify it appears in the panel
  • Search memories by keyword — verify FTS5 results are relevant
  • Filter by category — verify both list and search respect the filter
  • Archive a memory — verify it disappears from the active list
  • Delete a memory — verify confirmation dialog and permanent removal
  • Switch projects — verify project-scoped memories change, global memories persist
  • Check MemoryBadge in ChatHeader shows correct count
  • Verify bun typecheck, bun fmt, bun lint all pass (✅ confirmed)

juliusmarminge and others added 30 commits March 5, 2026 18:12
- Add per-provider model options, defaults, and slug aliases
- Add provider-aware model normalization/resolution helpers
- Preserve Codex-only constants/functions for backward compatibility
- Extend tests to cover Claude aliases and provider-specific fallback behavior
- introduce `ClaudeCodeAdapter` service and live layer wiring
- map runtime/session/request failures into provider adapter error types
- add coverage for validation, session-not-found mapping, lifecycle forwarding, and event passthrough
Add provider-service routing coverage for explicit claudeCode sessions.

Co-authored-by: codex <codex@users.noreply.github.com>
Add restart semantics when requested provider changes and cover with tests.

Co-authored-by: codex <codex@users.noreply.github.com>
Add provider-aware model options and include provider in turn-start dispatch.

Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
Co-authored-by: codex <codex@users.noreply.github.com>
- add `@anthropic-ai/claude-agent-sdk` dependency for `apps/server`
- replace placeholder Claude adapter with live session/query/event handling
- add comprehensive adapter tests for runtime events, approvals, resume, rollback, and model overrides
- Replace manual prompt async iterator with `Stream.fromQueue(...).toAsyncIterable`
- Use `Ref` for shared session context in tool approval callbacks
- Simplify timestamp/ID generation and close sessions via queue shutdown
- Stop synthesizing `resume` from generated thread IDs
- Persist resume session ID from query messages when available
- Validate resume/sessionId values as UUIDs before reuse
- Add tests for valid UUID resume passthrough and no synthesized resume
Co-authored-by: codex <codex@users.noreply.github.com>
- Do not pass `resumeCursor` when restarting a session after changing providers
- Treat synthetic Claude thread IDs (`claude-thread-*`) as unscoped during runtime ingestion
- Add tests covering provider-switch restart behavior and Claude turn lifecycle acceptance
Co-authored-by: codex <codex@users.noreply.github.com>
- Preserve active turn state when `session.started`/`thread.started` arrive mid-turn
- Emit `message.delta` from assistant text when stream deltas are missing, then complete the message
- Add Claude native SDK NDJSON observability logging and wire its log path in server layers
- Expand ingestion/adapter tests to cover mid-turn lifecycle and delta fallback behavior
- Default Claude sessions to bypass permissions when approval policy is `never`, while preserving explicit `permissionMode`
- Hide/send reasoning effort only for providers that support it in ChatView
- Add coverage for Claude permission-mode derivation and precedence, and update runtime event model docs
- add per-session `sessionSequence` on provider runtime events and persist activity `sequence`
- migrate `projection_thread_activities` with nullable `sequence` column + index
- sort server/web activity projections by sequence fallback to timestamp/id
- allow provider sessions/turns to start before a real threadId is emitted
- define `CursorAdapter` service contract and Cursor stream-json schema types
- add `CursorCliStreamEvent` decoding tests for system/thinking/tool/result/retry events
- add implementation plan for Cursor provider integration

Co-authored-by: codex <codex@users.noreply.github.com>
- Accept `cursor` as a first-class provider in orchestration, persistence, and session directory flows
- Update model/provider inference and normalization to handle Cursor model aliases
- Revamp chat provider/model picker with Cursor-specific trait controls and add coverage in tests
- add `scripts/cursor-acp-probe.mjs` to run ACP protocol probing scenarios
- update `package.json` scripts for probe execution
- include generated probe summaries/transcripts under `.tmp/acp-probe/`
42tg added 16 commits March 17, 2026 22:19
)

* feat: improve review popover UX with inline expand, sorting, and visual hierarchy

- Inline morph: clicking a row expands it in-place (no duplicate detail section)
  using CSS grid height animation for smooth reveal of action buttons
- Sort by status (in_review first, then pending) and PR number descending
  for stable, triage-optimized ordering
- Add sticky group headers ("In progress" / "Awaiting review") with backdrop blur
- De-emphasize repo name, promote PR title as primary visual element
- Fix nested button a11y: outer element is now div[role=button] with keyboard handlers
- Stabilize useMemo dependency with module-level empty array constant

* feat: improve notification panel mobile UX

- Larger panel: full-width on mobile, w-96 on desktop
- More height: 70dvh on mobile, 500px on desktop
- Block sidebar interaction behind open panel via z-indexed portal backdrop
- Raise popover positioner above Sheet (z-52) on mobile so it clears the sidebar drawer
- Close panel and mobile sidebar when navigating to a review
- Add positionerClassName prop to PopoverPopup for z-index overrides

* fix: suppress composer auto-focus on touch devices

Prevents the virtual keyboard from opening automatically when
navigating to a thread on mobile. focusComposer now bails early
on pointer:coarse devices, covering all call sites (navigation,
mode switches, model selection, drag-drop, terminal close).

* feat: persist and expose PR body and labels on review requests

Fetches body and labels from the GitHub CLI alongside existing PR
fields, stores them in two new columns (pr_body, pr_labels) added
via migration 018, and surfaces them through the ReviewRequest
contract so the notification panel can render PR description and
label chips in the expanded item view.
* feat: add review approve/request-changes actions in chat header

Add ReviewActionsControl component with a collapsible dropdown menu
(matching Git/Jira action patterns) that appears when the current
thread has an active PR review. Supports submitting APPROVE or
REQUEST_CHANGES via the GitHub API, then auto-dismisses the review
request locally.

* perf: reduce GitHub API rate-limit consumption

- Cache findLatestPr results per (cwd, branch) with 60s TTL, avoiding
  1-3 gh API calls every git status poll
- Cache repository clone URLs for 5 minutes (essentially static data
  fetched up to 4x per PR worktree setup)
- Cache PR head SHA for 2 minutes during review comment publishing
- Increase git status polling from 15s to 30s (stale 5s to 15s)
- Increase git branches polling from 60s to 120s (stale 15s to 30s)
- Increase review request polling from 60s to 120s (stale 30s to 60s)
- Invalidate PR cache on PR creation to ensure fresh data after mutations

* perf: further reduce GitHub API rate-limit consumption

- Extract createTtlCache to @t3tools/shared/cache (dedup GitManager + wsServer)
- Cache getDefaultBranch results for 15 minutes (essentially static)
- Server-side 60s cache for reviewRequestList (collapses multi-tab polls)
- Batch review comments into single POST /reviews call (N→1 API calls)
- Add refetchIntervalInBackground: false to all polled queries (stops
  polling when tab is in background)
- Add refetchOnWindowFocus to reviewRequestListQueryOptions
- Add stdin passthrough to GitHubCli.execute for batch API payloads
- Swipe-to-close: crisp 1px dark edge with tight depth shadow instead of
  blurry theme-color border; sidebar content live-translates under finger
- Swipe-to-open: edge indicator now uses sidebar background color, follows
  finger width directly, fades to fully opaque quickly
- Pull-to-reveal: bell icon indicator with smooth fade/scale feedback;
  PullToReveal component wired into ThreadSidebar
- NotificationBell: controlled open/onOpenChange props so pull gesture can
  drive it; mobile backdrop at z-51 blocks sidebar while panel is open;
  popover z-52 ensures it renders above the backdrop
Remove key={threadId} from ChatView so the DOM stays mounted across
thread switches (no unmount/remount blank frame). Add explicit resets
for all remaining state that isn't keyed by threadId: responding request
IDs, pending user input state, attachment preview handoffs, in-flight
send guard, and composer select lock.

On thread change, scroll the messages container to bottom synchronously
in useLayoutEffect (before paint) to avoid showing the previous thread's
scroll offset for a frame. Follow with a 120ms opacity fade-in so the
content change feels smooth rather than an abrupt swap.
- Add approved/changes_requested statuses to ReviewRequestStatus
- Show review outcome in chat header (approved, changes requested)
- Add "Done" tab in notification panel with proper outcome labels
- Fix "Go to Review" link for completed reviews with existing threads
- Unlink deleted threads from review requests on each poll
- Fix race condition: await linkThread before navigating to review thread
- Only auto-dismiss pending reviews (preserve completed review history)
- Add listReviewedPrs GitHub CLI method for future use
- Add "Restart" button next to "Done" in notification panel for in_review items
- Extract buildReviewPrompt and normalizePrReference to packages/shared/prReview
- Add implementation plan for future server-driven review flow
…eads

The query used NOT IN (alive threads), treating threads that don't exist
yet in projection_threads the same as deleted threads. This caused a race
where linkThread sets the threadId, but the next poll wipes it because the
client-side draft thread hasn't materialized on the server yet.

Changed to IN (explicitly deleted threads) so only threads that were
created and then deleted get unlinked. Also reset status back to pending
when unlinking so the UI stays consistent.
The server swallowed all gh api errors during comment publishing and
returned { published: 0 } as a success response. The client never
checked the count, so the UI always showed "Comment published to GitHub"
even when nothing was posted.

Now the server captures the actual gh error message and includes it in
the response. The client checks published count and throws with the real
error, which the existing error toast handler displays.
- ProviderRuntimeIngestion: use Effect.succeed(null) instead of
  Effect.succeed(undefined) to satisfy the effectSucceedWithVoid rule
  while preserving the truthiness check downstream
- ClaudeCodeAdapter: wrap JSON.parse in Effect.sync callback to move
  try/catch out of the generator scope (tryCatchInEffectGen rule)
- wsServer: use runProcess with allowNonZeroExit for batch comment
  publish so the GitHub API response body (stdout) is captured on 422
  errors — previously only stderr was shown, losing the actual error
  detail. Also add server-side logging for publish attempts.
Check which files are in the PR diff before attempting to publish.
If a comment targets a file outside the diff, return immediately with
a clear error message instead of hitting GitHub's 422 rejection.
Also adds file context hint to any remaining 422 errors.
The GitHub PR review API only accepts comments on files that are part of
the diff. Previously, the publish button appeared on all comments,
leading to confusing 422 errors for files not in the PR.

Now DiffPanel passes the set of branch diff file paths through to
useDiffAnnotations, which only attaches the onPublish callback (and
thus the button) to comments whose file is in the diff. Also keeps the
server-side pre-flight check as a safety net.
- Review prompt: remove /find-skills reference, condense tool docs
  to just mention review_comment tool. Keep skill check as Important.
- Timeline: normalize MCP tool labels from "Mcp__server__tool_name"
  to "Tool name" for readability (e.g. "Review comment" instead of
  "Mcp__review-comments__review_comment").
Introduces a full-stack memory feature inspired by Claude Code's memory
system. Memories can be scoped per-project or global, categorized
(preference, pattern, decision, fact, convention), and created manually
or (in a future phase) extracted automatically from conversations.

Key components:
- Effect Schema contracts for Memory entity, WS inputs/results
- SQLite migration with FTS5 full-text search + sync triggers
- MemoryRepository service/layer following the ReviewComment pattern
- 7 WebSocket RPC methods (list, search, create, update, archive, delete, getForThread)
- React Query hooks with cache invalidation and optimistic patterns
- MemoryPanel UI with search, category filters, and CRUD actions
- MemoryBadge in ChatHeader showing memory count per thread
- MemoryCreateDialog for manual memory creation
- Sidebar integration with sheet-based memory panel

Retrieval uses hybrid FTS5 keyword matching + relevance/recency scoring.
Auto-extraction from completed turns is designed but deferred to a
follow-up phase.
@github-actions github-actions bot added size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Mar 19, 2026
42tg added 5 commits March 19, 2026 08:14
Add batch memory extraction that analyzes conversation threads using
Claude Haiku and stores extracted knowledge as project-scoped memories
and daily summaries. Triggered manually via Extract button in MemoryPanel.

Key changes:
- Add MemoryExtraction service + layer with LLM-powered extraction
- Extract shared runAgentQuery utility from CodexTextGeneration
- Add daily scope + date field to memory schema (migration 020)
- Add optional source field to MemoryCreateInput for auto vs manual
- Add memory.extract WS endpoint, React Query mutation, and UI dialog
- Dedup via Jaccard word-similarity against existing project memories
- Two LLM passes: per-project knowledge extraction + global daily summary
Restructure the memory system from project/global/daily to
project/thread/daily scopes with autonomous extraction:

- Replace "global" scope with "thread" for per-thread summaries
- Add MemoryReactor that auto-generates thread summaries on
  turn.processing.quiesced (debounced 30s per thread)
- Skip re-summarization when thread has no new activity
- Periodic project + daily extraction after 10 thread summaries
- Improve extraction prompt quality bar (max 5, actionable only)
- Add thread_id column + unique index (migration 021)
- Add findThreadSummary + upsertThreadSummary to MemoryRepository
- Extract shared threadTranscript module
- Update UI: thread scope icon/labels, replace global references
…maries

- MemoryBadge: use Button size="xs" to align with other header controls
- MemoryPanel: remove Extract and Add buttons (extraction is autonomous)
- Daily summaries: delete existing entries for the date before inserting
  new ones, preventing duplicate daily memories on re-extraction
- Add deleteDailyByDate to MemoryRepository
Load existing daily summaries for today and pass them into the LLM
prompt so new extractions build on earlier context instead of replacing
it. The LLM produces a comprehensive merged summary, then the old
entries are deleted and replaced with the combined result.

- Add listDailyByDate to MemoryRepository
- Pass existing summaries to buildDailySummaryPrompt
- Prompt instructs LLM to incorporate earlier summaries
@42tg 42tg force-pushed the main branch 7 times, most recently from 54574d8 to 18c5f27 Compare April 7, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants