Implement Task Memory (raw memory) into Mirix

[LiaoJianhe]

This is the pahse 1: basic Mirix part of implemention for the Task Memory design.
The search implementation is not implemented in this phase.

Status: - All core components, REST APIs, cleanup jobs, and comprehensive tests implemented and verified.
Implementation Date: January 2026
Total Lines of Code: ~1,800 lines across 5 new files + 5 modified files

Overview

Task Memory (Raw Memory) is a new memory type in MIRIX that stores unprocessed task context without LLM extraction, designed specifically for agent-to-agent task sharing use cases.

This implementation adds a complete memory subsystem following MIRIX's standard architecture patterns, including ORM models, service managers, REST APIs, Redis caching, TTL enforcement, and comprehensive test coverage.

---

High-Level Design

Core Design Principles

1. Direct Storage Path: Bypasses the LLM extraction queue and writes directly to PostgreSQL
2. 14-Day TTL: Enforced by nightly cleanup job (memories deleted based on updated_at timestamp)
3. Scope-Based Access Control: Uses filter_tags.scope for multi-tenant isolation (same pattern as other memory types)
4. Redis JSON Caching: Performance optimization with cache-first reads and invalidation on updates/deletes
5. Standard MIRIX Pattern: Follows identical architecture to episodic, semantic, procedural, resource, and knowledge_vault memories
6. Consistent Naming: Uses *Item naming convention (RawMemoryItem, not RawMemory) to match other memory types

Architecture Flow

REST API (GET/PATCH/DELETE) 
    ↓
RawMemoryManager (synchronous CRUD operations)
    ↓
PostgreSQL (source of truth) + Redis (JSON cache)
    ↓
Nightly Cleanup Job (deletes records > 14 days old)

Database Schema

• Table: raw_memory
• Key Fields: id, user_id, organization_id, context, filter_tags (JSONB), timestamps
• Indexes: Organization, timestamp, GIN on filter_tags, scope extraction
• Audit Fields: _created_by_id, _last_update_by_id (track which client created/updated)

---

Files Added (New Implementations)

1. mirix/orm/raw_memory.py (152 lines)

Purpose: SQLAlchemy ORM model for the raw_memory table

Key Features:

• Uses OrganizationMixin and UserMixin (no AgentMixin)
• Includes filter_tags JSONB field for scope-based access control
• Audit fields (_created_by_id, _last_update_by_id)
• Standard MIRIX index patterns (organization, timestamp, GIN, scope extraction)
• Maps to PydanticRawMemoryItem for Pydantic serialization

Why: Defines database schema, relationships (organization, user), and enables ORM-based CRUD operations with proper indexing for performance.

---

2. mirix/schemas/raw_memory.py (131 lines)

Purpose: Pydantic schemas for raw memory validation and serialization

Key Classes:

• RawMemoryItemBase - Base schema with context and filter_tags
• RawMemoryItem - Full response schema (excludes audit fields per Pydantic v2 requirements)
• RawMemoryItemCreate - Creation schema with optional ID and timestamps
• RawMemoryItemUpdate - Update schema with context/tag merge modes

Why: Defines API request/response contracts, provides validation, and ensures type safety across the stack. Uses *Item naming to match other MIRIX memory types.

---

3. mirix/services/raw_memory_manager.py (360 lines)

Purpose: Service manager implementing business logic for raw memory CRUD

Key Methods:

• create_raw_memory() - Direct PostgreSQL write with Redis caching
• get_raw_memory_by_id() - Cache-first read with fallback to DB
• update_raw_memory() - Supports append/replace modes with cache invalidation
• delete_raw_memory() - Hard delete with cache removal

Key Features:

• All methods are synchronous (matching existing MIRIX managers)
• Redis JSON caching with get_json() and set_json()
• Cache invalidation on updates/deletes
• Comprehensive logging and error handling
• Optional parameter fallbacks (client_id, user_id)
• Timestamp defaulting for occurred_at, created_at, updated_at

Why: Centralizes all operations with Redis caching, access control, logging, and error handling. Provides consistent interface for both API and internal usage.

---

4. mirix/jobs/cleanup_raw_memories.py (107 lines)

Purpose: Nightly cleanup job for TTL enforcement

Key Function: delete_stale_raw_memories(days_threshold=14)

Features:

• Queries raw memories older than threshold (based on updated_at)
• Hard deletes records using manager's delete_raw_memory()
• Returns statistics (deleted count, error count, cutoff date)
• Configurable threshold (default: 14 days)

Why: Automatically enforces the 14-day TTL requirement for raw memories, preventing database bloat and ensuring compliance with data retention policies.

---

5. tests/test_raw_memory.py (889 lines, 18 tests)

Purpose: Comprehensive test suite covering all functionality

Test Coverage:

Manager Unit Tests (6 tests):

• test_manager_create_raw_memory - Create operations
• test_manager_get_raw_memory_by_id - Read operations
• test_manager_update_raw_memory_replace - Update with replace mode
• test_manager_update_raw_memory_append - Update with append mode
• test_manager_update_raw_memory_merge_tags - Update with tag merging
• test_manager_delete_raw_memory - Delete operations

Cleanup Job Tests (2 tests):

• test_cleanup_job_deletes_stale_memories - Verifies 14-day TTL deletion
• test_cleanup_job_respects_custom_threshold - Tests custom day thresholds

Redis Caching Tests (5 tests):

• test_raw_memory_create_with_redis - Verifies creation caches to Redis JSON
• test_raw_memory_cache_hit_performance - Benchmarks cache hit speed (< 10ms target)
• test_raw_memory_update_invalidates_cache - Verifies cache invalidation on updates
• test_raw_memory_delete_removes_cache - Verifies cache removal on deletes
• test_raw_memory_works_without_redis - Tests graceful fallback when Redis disabled

REST API Integration Tests (5 tests):

• test_api_create_and_get_raw_memory - GET endpoint
• test_api_update_raw_memory_replace - PATCH endpoint (replace mode)
• test_api_update_raw_memory_append_and_merge - PATCH endpoint (append/merge modes)
• test_api_delete_raw_memory - DELETE endpoint
• test_api_get_nonexistent_memory - 404 error handling

Why: Validates all functionality, ensures Redis caching works correctly, verifies API authentication/authorization, and provides regression protection.

---

Files Modified (Existing Files Extended)

6. mirix/server/rest_api.py (3 new endpoints)

Changes Added:

• GET /memory/raw/{memory_id} - Fetch single raw memory
• PATCH /memory/raw/{memory_id} - Update context/filter_tags with append/replace/merge modes
• DELETE /memory/raw/{memory_id} - Hard delete memory

Authentication: Uses get_client_from_jwt_or_api_key() (supports both JWT and API keys)

Why: Exposes raw memory functionality via HTTP API following existing MIRIX endpoint patterns (/memory/episodic/, /memory/semantic/, etc.).

---

7. mirix/database/redis_client.py (1 line added)

Change: Added RAW_MEMORY_PREFIX = "raw_memory:" constant

Why: Enables Redis caching with namespace isolation. Required for key generation in manager and ORM caching logic.

---

8. mirix/orm/sqlalchemy_base.py (1 line added - CRITICAL FIX)

Change: Added "raw_memory": redis_client.RAW_MEMORY_PREFIX to the memory_tables dictionary in _update_redis_cache() method (line ~1010)

Why: This is the most critical change - without this entry, Redis caching doesn't work at all. The ORM's create_with_redis() method calls _update_redis_cache(), which checks this dictionary to determine if a table uses JSON-based caching. Without this entry, records are logged as "Cached" but not actually stored in Redis.

Impact: This was the root cause of initial Redis caching failure discovered during testing.

---

9. mirix/server/server.py (registration)

Change: Added self.raw_memory_manager = RawMemoryManager() in server initialization

Why: Makes the manager accessible to REST API endpoints via server.raw_memory_manager.

---

10. mirix/jobs/__init__.py (new directory)

Change: Created jobs package directory with __init__.py

Why: Standard Python package structure for scheduled jobs. Enables import of cleanup job module.

---