feat(store): add recall tracking and promoted observations endpoint

[sergiomarquezdev]

Linked Issue

Closes #95

PR Type

• New feature
• Bug fix
• Documentation
• Refactoring
• Tests
• CI/CD
• Other

Summary

• Add read-side instrumentation (recall_count, last_recalled_at) to observations, tracking how often each memory is retrieved via Search and GetObservation
• Expose a GET /promoted HTTP endpoint and mem_promoted MCP tool that surface frequently-recalled observations above a configurable threshold
• Enable clients to implement "memory promotion" strategies where high-value observations graduate to permanent context (e.g., system prompt injection)

Changes

File	Change
internal/store/store.go	Add recall_count/last_recalled_at to schema, struct, all 13 scan sites, migration. New incrementRecall() (fire-and-forget), PromotedObservations(), and private getObservation() (no side effects)
internal/store/store_test.go	4 new tests: recall increment on search, recall increment on get, promoted threshold filtering, deleted observation exclusion
internal/server/server.go	GET /promoted endpoint with project, scope, min_recalls, limit query params
internal/mcp/mcp.go	mem_promoted tool in agent profile (deferred loading), updated tool counts in package doc
internal/mcp/mcp_test.go	Updated tool count assertions (14→15 total, 11→12 agent)

Design Decisions

Fire-and-forget recall tracking: incrementRecall() uses _, _ = s.db.Exec(...) — recall tracking must never break reads. Errors are silently ignored.

Public vs private GetObservation: Split into GetObservation() (increments recall, for external callers) and getObservation() (no side effects, for internal use like Timeline()). This prevents navigational reads from inflating counts.

No sync for recall data: recall_count is analytics metadata, not content. No enqueueSyncMutationTx calls — avoids noise in cross-device sync.

Schema migration: Uses existing addColumnIfNotExists pattern. recall_count defaults to 0 for existing rows, last_recalled_at defaults to NULL. Safe for databases with existing data.

Test Plan

• TestRecallCountIncrementsOnSearch — verifies Search bumps recall_count
• TestRecallCountIncrementsOnGetObservation — verifies GetObservation bumps recall_count
• TestPromotedObservationsReturnsHighRecallOnly — verifies threshold filtering
• TestPromotedObservationsExcludesDeleted — verifies soft-deleted excluded
• Manual E2E: built binary, swapped on live system with 13 existing observations, verified zero data loss, recall tracking works, /promoted returns correct results

go test ./internal/store/... -v -run "TestRecall|TestPromoted"
go test ./internal/mcp/... ./internal/server/...

Automated Checks

Check	Status
Issue reference	⏳
Issue approved	⏳
Type label	⏳
Unit tests	⏳
E2E tests	⏳

Contributor Checklist

• Code follows existing patterns and conventions
• All scan sites (13) updated consistently
• Migration is idempotent (addColumnIfNotExists)
• MCP handler returns ToolResultError, not Go errors
• Error messages follow "Failed to [verb] [noun]: " pattern
• Tests use white-box style with newTestStore(t)
• Commit message follows conventional commit format
• No external test frameworks added

Notes for Reviewers

• The only failing test on Windows is TestNewErrorBranches/fails_when_migration_encounters_conflicting_object — a pre-existing Windows file-locking issue unrelated to this PR
• recall_count is monotonically increasing by design. Staleness filtering (e.g., "not recalled in last 30 days") is a natural follow-up but not in scope for v1
• The mem_promoted tool uses deferred loading to avoid bloating the initial tool context for agents that don't need it

[Alan-TheGentleman]

This is thoughtful work, but it is not a quick merge. You are changing persistence semantics, schema, endpoint surface, and MCP behavior in one shot. Before I merge something at that depth, I need the linked issue approved and I need the design justified explicitly around migration safety, read-side side effects, and product scope. Tighten the design case first, then we can review the implementation.

[sergiomarquezdev]

Thanks for the review @Alan-TheGentleman. I've posted a detailed design justification on the linked issue (#95) covering the three areas you raised:

1. Migration safety — uses the existing addColumnIfNotExists pattern, same as the other 9 columns added through that path. Safe defaults, no backfill, no destructive changes.
2. Read-side side effects — fire-and-forget semantics, no sync propagation, public/private split to prevent inflation from internal reads.
3. Product scope — natural complement to existing write-side analytics (revision_count, duplicate_count). Data layer only — no promotion strategy imposed.

Full details: #95 (comment)

Once the issue is reviewed, I'll rebase on current main (no conflicts expected — the 21 upstream commits don't touch store/mcp/server) and force-push.

[sergiomarquezdev]

Closing: upstream removed recall tracking in b6b1f6f (perf(mcp): defer 4 rare tools). Feature didn't generate practical value after testing. Syncing fork with upstream direction.