Update MIRIX backend to use async native approach#119
Merged
LiaoJianhe merged 30 commits intoMirix-AI:re-orgfrom Mar 12, 2026
Merged
Update MIRIX backend to use async native approach#119LiaoJianhe merged 30 commits intoMirix-AI:re-orgfrom
LiaoJianhe merged 30 commits intoMirix-AI:re-orgfrom
Conversation
* feat: multi scope clients * fix: format and passing tests * fix: fix langfuse tests * fix: fix local client tests * fix: fix tests * chore: format * feat: scoped core memory * fix: fix some test bugs * chore: tests * Apply suggestion from @L-u-k-e * Apply suggestion from @L-u-k-e --------- Co-authored-by: Jianhe Liao <jianhe_liao@intuit.com>
feat: Allow clients to add `filter_tags` to blocks and use them for cross user searches
* feat: support multiple filter operators in tag search * chore: remove integration test * fix: messageToDict * feat: block filter tag updates - always-apply on save plus new update mode options (#58) * gfeat: update block filter tags * fix: fix bugs
Made-with: Cursor
Made-with: Cursor
…single-user search) - Pull latest re-org (PR #60: feat VEPAGE-518 include core memory in single user search) - Resolve conflicts: keep async-native patterns (await block_manager.get_blocks, await client.search) - remote_client: add include_core_memory param to async search(), keep return await _request - rest_api: use await server.block_manager.get_blocks (no asyncio.to_thread) - test_block_filter_*: keep agent.user = SimpleNamespace(id='test-user') from async branch - test_search_single_user_core_memory: convert to async (MirixClient.create, await, asyncio.sleep) Made-with: Cursor
Change Mirix code base to be async-native
Add artifacts to make the code base AI Coding friendly
fix: strip sslmode from URL param
fix: update setup_server.py
[VEPAGE-326] Update MIRIX with async cache provider changes Built and pushed package jl-ecms-*-0.5.1, validated the package.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MIRIX Async-Native Rewrite
1. Why Async-Native
MIRIX is a multi-agent system where every user request fans out into
database queries, Redis lookups, LLM API calls, embedding computations,
and Kafka messages -- all I/O-bound. The previous sync codebase serialized
these operations: each blocked thread sat idle waiting for a network
response, and concurrency was limited to the thread-pool size.
Rewriting the stack to be async-native delivers several concrete benefits:
Higher throughput on the same hardware.
A single event loop multiplexes thousands of in-flight I/O operations
without dedicating a thread to each one. Connection pools (asyncpg, Redis,
httpx) are shared across all coroutines, so the server handles more
concurrent users with fewer file descriptors, less memory, and less
context-switching overhead.
End-to-end consistency with FastAPI / Uvicorn.
FastAPI is async-first. When route handlers are
async defand directlyawaitthe server/manager/agent/LLM chain, there is no implicit offloadto a thread-pool executor. This removes an entire class of subtle bugs
(thread-safety of shared state, session leaks across threads) and makes
the call stack easy to reason about.
Natural streaming and SSE.
LLM token streaming and Server-Sent Events map directly to async
generators. No background thread is needed to feed an SSE response; the
generator yields tokens as they arrive from the LLM provider.
In-process background workers.
Queue consumers (memory extraction, cleanup) run as
asyncio.Tasks inthe same process. This simplifies deployment (one container, one process)
while keeping workers non-blocking.
Lower tail latency.
asyncio.sleep-based retries and exponential back-off do not occupy athread during the wait, freeing the loop to serve other requests.
2. High-Level Changes
2.1 External Library Migrations
Layer | Sync (before) | Async (after) -- | -- | -- Database driver | pg8000 / psycopg2-binary | asyncpg (PostgreSQL), aiosqlite (SQLite) SQLAlchemy | create_engine, sessionmaker, Session | create_async_engine, async_sessionmaker, AsyncSession Redis | redis.Redis | redis.asyncio.Redis with hiredis HTTP client | requests | httpx.AsyncClient OpenAI | openai.OpenAI | openai.AsyncOpenAI Anthropic | anthropic.Anthropic | anthropic.AsyncAnthropic Azure OpenAI | AzureOpenAI | AsyncAzureOpenAI Google AI | sync genai calls | async genai + httpx.AsyncClient Kafka | sync kafka-python (if used) | aiokafka (AIOKafkaProducer, AIOKafkaConsumer) Web search | duckduckgo_search | asyncddgs Google APIs | sync google-api-python-client | aiogoogle Test runner | sync pytest | pytest-asyncio (asyncio_mode = "auto")3.6 Server Class Naming (Resolved)
The class formerly named
SyncServerhas been renamed toAsyncServeras part of this change set. All imports, type hints, docstrings, and tests
have been updated. A backward-compatible alias
SyncServer = AsyncServeris retained in
mirix/server/server.py.4. Summary
The MIRIX application is async-native from the HTTP boundary through the
server, agents, service managers, ORM, database, Redis, Kafka, and
LLM/embedding clients. The only remaining sync touch-points are:
asyncio.to_thread; low impact.to_thread; rare.run_sync; no runtime impact.asyncio.run()inmain; separate process.async defwould add overhead, not benefit.SyncServerrenamed toAsyncServer; alias kept.None of these limit MIRIX's ability to scale request throughput or
concurrent users. The critical path is fully async.