Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
f2d4f8b
Merge pull request #13 from sovantica/dev
przemarzec Jun 3, 2026
f93d026
Merge pull request #15 from sovantica/dev
przemarzec Jun 3, 2026
7d1118d
Merge pull request from dev: docs accuracy fix + documentation-exampl…
przemarzec Jun 3, 2026
3db5530
Merge pull request #18 from sovantica/dev
przemarzec Jun 4, 2026
f07fef4
Merge pull request #21 from sovantica/dev
przemarzec Jun 5, 2026
b2f1691
Merge pull request #23 from sovantica/dev
przemarzec Jun 6, 2026
d7eaff0
build(deps): raise pydantic floor to >=2.11
przemarzec Jun 7, 2026
1c8ffb4
feat(mindql): add store-level execute_mindql entry point
przemarzec Jun 7, 2026
68a8085
feat(mcp): add MCP server with read tools (engrava[mcp] extra)
przemarzec Jun 7, 2026
82c65f0
Merge MCP server (read tools) + MindQL execution entry point into v0.4.0
przemarzec Jun 8, 2026
79d7604
feat(mcp): add write tools, opt-in read-only mode, and per-tool safet…
przemarzec Jun 8, 2026
3723d10
Merge MCP write tools, read-only mode and safety annotations into v0.4.0
przemarzec Jun 8, 2026
84b87f6
feat(mcp): add delete_thought and delete_edge tools
przemarzec Jun 8, 2026
76598bb
Merge MCP delete tools (delete_thought, delete_edge) into v0.4.0
przemarzec Jun 8, 2026
c54dcf7
feat(mcp): expose memory as resources (thought, stats, recent)
przemarzec Jun 8, 2026
62c7639
Merge MCP resources (thought, stats, recent) into v0.4.0
przemarzec Jun 8, 2026
1f6fa36
feat(mcp): add guided memory prompts
przemarzec Jun 8, 2026
2afef80
Merge MCP guided memory prompts into v0.4.0
przemarzec Jun 8, 2026
57357b7
feat(mcp): add memory filters and pagination
przemarzec Jun 8, 2026
62b2b60
Merge MCP memory filters and pagination into v0.4.0
przemarzec Jun 8, 2026
8b615cc
feat(mcp): map known failures to typed, actionable tool errors
przemarzec Jun 8, 2026
5f4ea20
fix(mcp): keep query_memory parse errors FIND-only
przemarzec Jun 9, 2026
8c6811b
Merge MCP structured tool errors into v0.4.0
przemarzec Jun 9, 2026
b2bc18b
docs(mcp): add MCP server guide and client-config examples
przemarzec Jun 9, 2026
7ccdbd8
Merge MCP server guide and client-config examples into v0.4.0
przemarzec Jun 9, 2026
897c46f
fix(embeddings): retry transient errors with bounded backoff
przemarzec Jun 9, 2026
f959844
Merge embedding-provider transient-error retry into v0.4.0
przemarzec Jun 9, 2026
730c2bb
test(docs): execute the tutorial end-to-end and a search round-trip
przemarzec Jun 9, 2026
c582bf0
Merge doc-block execution (tutorial end-to-end + search round-trip) i…
przemarzec Jun 9, 2026
456bcb6
feat: add bi-temporal valid-time to thoughts and edges
przemarzec Jun 10, 2026
534cb5c
Merge bi-temporal valid-time storage into v0.4.0
przemarzec Jun 10, 2026
86de77f
feat: add temporal query predicates and invalidate primitive
przemarzec Jun 10, 2026
c160736
Merge temporal query predicates and invalidate primitive into v0.4.0
przemarzec Jun 10, 2026
8fba769
feat: reflections inherit temporal extent from members
przemarzec Jun 10, 2026
e220f0a
Merge reflection temporal-extent inheritance into v0.4.0
przemarzec Jun 10, 2026
036e225
docs: add bi-temporal model guide and 0.3 to 0.4 upgrade notes
przemarzec Jun 10, 2026
a04b75c
Merge bi-temporal documentation and 0.3 to 0.4 upgrade notes into v0.4.0
przemarzec Jun 10, 2026
035e860
test: bound temporal query overhead and confirm index use
przemarzec Jun 10, 2026
0e4e176
fix: assert plan-shape invariant for temporal queries, not scan-vs-index
przemarzec Jun 10, 2026
cd4ecc2
fix: match exact table token in query-plan helpers
przemarzec Jun 10, 2026
4779c9c
Merge temporal-query performance gate into v0.4.0
przemarzec Jun 10, 2026
3a0eaa3
test: pin native thread pools session-wide to stop full-suite hang
przemarzec Jun 10, 2026
114e85a
Merge session-wide test thread-pinning into v0.4.0
przemarzec Jun 10, 2026
bb6b729
fix: let natural-language queries reach the full-text index
przemarzec Jun 12, 2026
ba1b288
Merge full-text query repair into v0.4.0
przemarzec Jun 12, 2026
862e7bc
test: add functional contract suite for search behavior
przemarzec Jun 12, 2026
30027f4
Merge search functional contract suite into v0.4.0
przemarzec Jun 13, 2026
36e08e7
fix: embed full thought content without duplication or silent truncation
przemarzec Jun 13, 2026
825a9fc
Merge embedding-input repair into v0.4.0
przemarzec Jun 13, 2026
d88043d
fix: keep quoted MindQL values as strings and reject malformed condit…
przemarzec Jun 13, 2026
3f3fbeb
Merge MindQL correctness fixes into v0.4.0
przemarzec Jun 13, 2026
1256303
perf: tune sqlite pragmas and add hot-path indexes
przemarzec Jun 13, 2026
66afaf0
Merge sqlite pragma tuning and hot-path indexes into v0.4.0
przemarzec Jun 13, 2026
c3cf339
test: isolate subprocess examples (offline, single-thread, no stdin)
przemarzec Jun 13, 2026
59dcf5d
docs: drop reference to non-existent purity-check script in CONTRIBUTING
przemarzec Jun 14, 2026
6794436
fix(mcp): map write-tool errors and complete the 0.4.0 documentation
przemarzec Jun 14, 2026
be9b110
feat(api): add remember() and recall() convenience methods on the store
przemarzec Jun 15, 2026
bd6402f
Merge convenience API (remember/recall) and cycle defaults into v0.4.0
przemarzec Jun 15, 2026
edfe620
docs: lead quickstart with remember()/recall() short path
przemarzec Jun 15, 2026
8c00368
docs: fix quickstart cycle note — remember() takes no created_cycle; …
przemarzec Jun 15, 2026
1a89476
Merge quickstart remember()/recall() short path into v0.4.0
przemarzec Jun 15, 2026
3f415e2
docs: lead README Basic Usage with remember()/recall()
przemarzec Jun 15, 2026
d42a852
docs: correct reflection_boost default to 1.0; add test verifying doc…
przemarzec Jun 15, 2026
2e80d04
docs: document percept/utterance/thought metadata helpers in api-refe…
przemarzec Jun 15, 2026
22bf027
Merge README quickstart remember()/recall() into v0.4.0
przemarzec Jun 15, 2026
8ff4145
Merge documented-defaults verification and metadata-helper docs into …
przemarzec Jun 15, 2026
c55f0ad
docs: list all installable extras in README (mcp + ollama/hf embeddin…
przemarzec Jun 16, 2026
c6fa946
build: drop no-op 'dreaming' extra (empty deps; dreaming is in the ba…
przemarzec Jun 16, 2026
13c2448
Merge README extras list + drop no-op dreaming extra into v0.4.0
przemarzec Jun 16, 2026
ed8b184
docs(tests): de-reference internal principle name in doc-test rationale
przemarzec Jun 16, 2026
cd5df19
docs: bring architecture + CLI docs up to 0.4.0 (bi-temporal + MCP)
przemarzec Jun 16, 2026
bb5dff1
docs: fix 0.4.0 drift found in the full doc audit
przemarzec Jun 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 134 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,139 @@ and this project adheres to [Semantic Versioning 2.0.0](https://semver.org/spec/

## [Unreleased]

### Added

- **`remember` and `recall`: store and retrieve in one call.** Two ergonomic
convenience methods on the store let you persist a string and get relevant
strings back without hand-building a `ThoughtRecord` or wiring up
`search_hybrid`. `remember(text, *, metadata=None, deduplicate=False)` stores
a string as a thought (deriving the essence from its opening) and honours
opt-in content deduplication; `recall(query, *, top_k=10, current_cycle=None)`
returns the ranked matches. Passing `current_cycle` to `recall` blends in the
recency signal; on a large store recalled without it, a single DEBUG log line
points out that a cycle would let recent thoughts rank higher. `ThoughtRecord`
now defaults `created_cycle` and `updated_cycle` to `0`, so callers that do
not track cognitive cycles can omit them.

- **Bi-temporal model: track when a fact is *true*, not just when you stored
it.** `ThoughtRecord` and `EdgeRecord` gain two optional, nullable ISO-8601
fields — `valid_from` and `valid_until` — describing the half-open real-world
interval during which a fact holds (the upper bound is exclusive; a `None`
bound is treated as ±∞, so facts you never annotate keep matching every query).
Four opt-in MindQL `WHERE` predicates query this *valid time* on the `thoughts`
and `edges` tables: `valid_now`, `valid_at <ts>`, `valid_within <start> <end>`
(interval overlap), and `valid_between <start> <end>` (fully contained — the one
predicate that excludes open-bounded rows). Two new store primitives,
`invalidate_thought(id, valid_until)` and `invalidate_edge(id, valid_until)`,
retire a fact by closing its interval instead of deleting it — deterministic,
idempotent, non-cascading, and fully auditable (the row stays on file and a
point-in-time query before the cut-off still finds it). Reflections built by
dreaming inherit their members' valid-time extent (open-on-either-side is
contagious). A query that uses no temporal predicate behaves exactly as before.
See the [Bi-temporal Model](docs/bitemporal.md) guide.

- **MCP server: connect any MCP client to an engrava store.** A new optional
`mcp` extra (`pip install "engrava[mcp]"`) ships a Model Context Protocol
server that exposes a store over stdio to Claude Desktop, Claude Code, Cursor,
Windsurf, VS Code, and other MCP clients. Two entry points, `engrava-mcp` and
`python -m engrava.mcp`, build the same server. It registers eleven tools (six
read, five write), two static `engrava://` resources plus an
`engrava://thought/{thought_id}` resource template, and three prompt templates,
resolving its store from `ENGRAVA_MCP_CONFIG` (an `engrava.yaml`) or
`ENGRAVA_DB_PATH` (a bare database file). A read-only mode
(`ENGRAVA_MCP_READ_ONLY`) drops the five write tools entirely, leaving a
retrieval-only surface. The server is a pure API consumer — plain
`pip install engrava` is unaffected and stays dependency-light. See the
[MCP server](docs/guides/mcp.md) guide.

- **`execute_mindql` on the store.** `SqliteEngravaCore.execute_mindql(query, *,
extensions=None)` runs a parsed `MindQLQuery` directly against the store's own
connection, returning a `MindQLResult` — a convenience over constructing a
`MindQLExecutor` by hand. See the [API Reference](docs/api-reference.md#mindql).

### Performance

- **Hot-path indexes and tuned SQLite PRAGMAs make the common reads faster.**
Four indexes now back the equality filters and the sort column hit on every
common read — looking up edges by their target thought, fetching a thought's
embedding, listing thoughts in recency order, and filtering thoughts by type
— turning what were full table scans into index lookups. The connection is
also opened with `synchronous=NORMAL` (the documented-safe companion to WAL:
durable across an application crash, only at risk of losing the most recent
transactions on an OS crash or power loss) and `busy_timeout=5000`, so a
second connection waits briefly for a lock instead of failing immediately
with a "database is locked" error. The index changes are an additive schema
migration that runs automatically on first open with zero data loss; see the
[upgrade guide](docs/upgrade.md#03---04).

### Fixed

- **MindQL no longer mistypes quoted values or silently ignores malformed
conditions.** A single-quoted WHERE value is now kept verbatim as a string,
so a zero-padded identifier such as `WHERE source = '007'` matches the stored
string `'007'` instead of being coerced to the integer `7` and matching
nothing; an *unquoted* bare value (for example `WHERE created_cycle = 12`) is
still coerced to a number as before. A WHERE fragment must now match the
`field op value` grammar in full: trailing content after a condition (such as
`WHERE priority = 'P1' OR 1=1`) previously matched only the leading prefix and
silently discarded the rest, which could change the result set unnoticed — it
now raises a parse error. Finally, a `FIND` query with no `LIMIT` clause is
capped at a sane default (100 rows) rather than running an unbounded scan; an
explicit `LIMIT` always overrides the default, and `COUNT` queries are
unaffected.

- **Long memories are now embedded in full, and a thought's opening is no
longer double-counted.** Two silent recall killers in the vector arm of
search are fixed. First, when a thought's `essence` is just the opening of
its `content` (a common convention, e.g. `essence = content[:200]`),
auto-embed used to concatenate the two and encode that opening twice, letting
it dominate the vector and dilute the discriminative tail; the redundant
prefix is now dropped and `content` is embedded alone, while a genuinely
distinct `essence` is still encoded alongside the content as before. Second,
the local `sentence-transformers` provider now raises `max_seq_length` to the
model's true architecture maximum after loading (derived from the model, not
hard-coded), instead of accepting a conservative shipped default — the bundled
`all-MiniLM-L12-v2` reported `128` while its backbone supports `512`, so the
tail of any longer thought was silently truncated away before encoding.
Existing stored embeddings are unaffected until a thought is re-written
(re-create or an `essence`/`content` update), at which point it is re-embedded
with the corrected input.

- **Natural-language queries now reach the full-text index.** `search_fts`
previously joined the words of a bare query with FTS5's implicit `AND`, so a
question only matched documents that contained *every* word — including
function words like "what", "was" and "my" — and a relevantly-phrased answer
was missed. Bare queries are now matched with `OR`: a document is returned
when it shares any content word, and BM25's IDF weighting ranks the documents
that share the most distinctive words first, so no stopword list or stemmer is
needed in any language. Contractions and clitics no longer silently miss
(`sister's` matches a stored `sister's dog`; `l'école` matches `l'école
française`) because unsafe characters now split a token into separate terms
instead of being deleted into an unindexed word. Pasting a URL or a timestamp
into search no longer raises: only the real `essence:` and `content:` column
filters are honoured, while tokens such as `http://example.com` or `12:30` are
treated as ordinary search terms. Expert syntax is unchanged — quoted phrases,
uppercase `AND`/`OR`/`NOT`, hyphenated identifiers and the `essence:`/
`content:` column filters all keep their existing behaviour. As a final
safeguard, a malformed full-text expression is now logged and degraded to no
full-text hits instead of propagating, so the rest of a hybrid search still
returns results.

- **Transient errors from an OpenAI-compatible embeddings endpoint no
longer abort the whole call.** `OpenAICompatibleProvider` now retries a
single embeddings request with bounded exponential backoff when the
endpoint reports a transient failure — a read timeout or network blip,
or a transient HTTP status (`408`, `409`, `425`, `429`, `500`, `502`,
`503`, `504`). A short outage is absorbed instead of failing the
caller's ingest. Non-transient errors (such as `400`, `401`, `403`,
`404`) still surface immediately with no retry, and a transient failure
that persists across every attempt is still raised, so the call never
loops forever. The behaviour is tunable through two new keyword-only
constructor arguments, `max_attempts` (default `3`) and
`base_retry_delay_s` (default `1.0`); the defaults leave the success
path at a single request, so existing callers see no change. The other
embedding providers are unaffected.

- **sqlite-vec backend no longer crashes on configuration.** Selecting the
`sqlite-vec` vector backend (via `engrava[vec]`) previously raised a
thread error when building a store from configuration, because the
Expand Down Expand Up @@ -656,13 +787,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **`search_hybrid()` — `include_reflections` + `reflection_boost`** — callers
can now pass `include_reflections=False` to exclude REFLECTION thoughts from
hybrid search results, or a custom `reflection_boost` multiplier to re-rank
them. The default boost (`SearchConfig.reflection_boost = 1.2`) gives
REFLECTION thoughts a mild up-ranking.
them. The default boost (`SearchConfig.reflection_boost = 1.0`) leaves
REFLECTION thoughts competing on equal footing until raised.
- **`search_reflections_only()`** — convenience method on `SqliteEngravaCore`
that returns only `ThoughtType.REFLECTION` thoughts ranked by hybrid score.
- **`list_edges()`** — new `SqliteEngravaCore` method for querying edges by
optional `edge_type` / `source` filters with configurable `limit`.
- **`SearchConfig.reflection_boost`** — new field (default `1.2`) parsed from
- **`SearchConfig.reflection_boost`** — new field (default `1.0`) parsed from
YAML `search.reflection_boost`.

- **Dream-created edges** — `run_consolidation()` now creates
Expand Down
9 changes: 2 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,8 @@ Public Engrava artifacts must stay free of internal tier references.
- Avoid internal branded example names in comments, docstrings, snippets, and tests.
- Prefer neutral names like `ThirdPartyHooks`, `CustomPlugin`, or `ConsumerApp`.

Run the purity check before opening a PR:

```bash
python scripts/assert_purity.py
```

CI also runs the same check and will fail if a forbidden reference leaks into the public tree.
Before opening a PR, re-read your diff against the three rules above and remove any internal
reference. Maintainers verify this invariant during review.

## Code Style

Expand Down
67 changes: 44 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,28 @@ pip install engrava
Optional extras:

```bash
pip install engrava[vec] # sqlite-vec vector search backend
pip install engrava[embeddings-local] # sentence-transformers embeddings
pip install engrava[embeddings-openai] # OpenAI-compatible embeddings
pip install engrava[vec] # sqlite-vec vector search backend
pip install engrava[mcp] # MCP server (engrava-mcp) for Claude Desktop/Code, Cursor, …
pip install engrava[embeddings-local] # sentence-transformers embeddings (local model)
pip install engrava[embeddings-openai] # OpenAI-compatible embeddings API
pip install engrava[embeddings-ollama] # Ollama local embeddings server
pip install engrava[embeddings-hf] # HuggingFace Inference API embeddings
```

Dreaming/consolidation and the knowledge graph need **no extra** — they are part
of the base install.

### Basic Usage

Store a memory and search for it in two calls — no IDs to generate, no record
to assemble:

```python
import asyncio
import uuid

import aiosqlite

from engrava import LifecycleStatus, Priority, SqliteEngravaCore, ThoughtRecord, ThoughtType
from engrava import SqliteEngravaCore


async def main() -> None:
Expand All @@ -56,29 +64,25 @@ async def main() -> None:
store = SqliteEngravaCore(conn)
await store.ensure_schema()

# Build a ThoughtRecord and persist it with create_thought.
observation = ThoughtRecord(
thought_id=str(uuid.uuid4()),
thought_type=ThoughtType.OBSERVATION,
essence="Python is great for AI",
content="Python's async ecosystem makes it ideal for AI agents.",
priority=Priority.P2,
lifecycle_status=LifecycleStatus.ACTIVE,
created_cycle=0,
updated_cycle=0,
source="human",
)
await store.create_thought(observation)

# Retrieve it
thought = await store.get_thought(observation.thought_id)
if thought is not None:
print(f"Stored: {thought.essence}")
await store.remember("Python is great for AI agents")
await store.remember("SQLite needs no server")

result = await store.recall("what language is good for agents?")
for thought_id, score in result.results:
thought = await store.get_thought(thought_id)
if thought is not None:
print(f"{thought.essence} (score: {score:.3f})")


asyncio.run(main())
```

`remember()` stores the text as a thought (generating its ID for you) and
returns the stored `ThoughtRecord`; `recall()` runs the same hybrid search as
`search_hybrid()` and returns the ranked results. For full control — setting
priority, thought type, metadata, or the cognitive cycle on a write — build a
`ThoughtRecord` yourself and call `create_thought()`.

From here, link thoughts with [typed edges](#edge-based-knowledge-graph),
query them with [MindQL](#mindql-query-language), or run the full
ingest → dream → search tour in the [Quick Start guide](docs/quickstart.md).
Expand Down Expand Up @@ -199,6 +203,21 @@ async with EngravaManager(data_dir=Path("./data")) as mgr:
# Completely isolated databases
```

### MCP Server

Expose a store to any MCP client — Claude Desktop, Claude Code, Cursor,
Windsurf, VS Code — via the `engrava[mcp]` extra. A native stdio server (no HTTP
shim) with read tools, optional write tools, attachable `engrava://` resources,
and guided prompts:

```bash
pip install "engrava[mcp]"
engrava-mcp # spawned by your MCP client over stdio
```

→ See [`docs/guides/mcp.md`](docs/guides/mcp.md) for install, client
configuration, the full tool/resource/prompt reference, and read-only mode.

## CLI

```bash
Expand Down Expand Up @@ -228,13 +247,15 @@ See the [CLI reference](docs/cli.md) for every command and option.
## Documentation

- [Core Concepts](docs/concepts.md) — the mental model (thought, edge, reflection, cycle, …) — start here
- [The Bi-temporal Model](docs/bitemporal.md) — the optional valid-time axis: query a fact as of any instant, `invalidate` without deleting
- [Positioning](docs/positioning.md) — when Engrava is (and isn't) the right tool, and how it compares
- [Quick Start](docs/quickstart.md) — 5-minute setup guide
- [Tutorial](docs/tutorial.md) — build a small notes memory end to end
- [Recipes](docs/recipes/index.md) — copy-paste snippets for common tasks (store a turn, retrieve context, TTL, dedup, …)
- [Building a memory-backed agent](docs/guides/agent-memory.md) — the end-to-end agent turn loop (ingest → retrieve → generate → consolidate)
- [Migrating from another memory system](docs/guides/migrating-from-other-memory.md) — concept mapping, porting calls, bulk import, and scoping/multi-tenancy
- [Embeddings](docs/guides/embeddings.md) — wiring a real embedding provider (local / OpenAI / Ollama / HuggingFace / custom)
- [MCP server](docs/guides/mcp.md) — expose a store to MCP clients (Claude Desktop, Claude Code, Cursor, Windsurf, VS Code): install, run, client config, tools/resources/prompts
- [Configuration](docs/configuration.md) — YAML config format and options
- [Upgrade Guide](docs/upgrade.md) — compatibility matrix, backups, and troubleshooting
- [Extensions](docs/extensions.md) — Writing custom extensions and hooks
Expand Down
Loading
Loading