feat: 改进内存测试系统，添加环境变量控制和优雅降级机制

.gitignore

@@ -33,3 +33,5 @@ local_evaluations
 .claude/
 mirix.egg-info
 .local
+logs/
+mirix_qwen3.yaml

LLM_DEBUG_README.md

@@ -0,0 +1,172 @@
+# LLM 调试日志记录功能
+
+这个功能提供了详细的 LLM 调用和响应日志记录，帮助诊断和解决 LLM API 调用中的问题。
+
+## 功能特性
+
+- 📝 **详细的请求记录**: 记录发送给 LLM 的完整请求内容
+- 📥 **完整的响应记录**: 记录 LLM 返回的完整响应内容
+- ⏱️ **性能监控**: 记录请求响应时间
+- ❌ **错误跟踪**: 详细记录错误信息和上下文
+- 🔧 **JSON 解析错误诊断**: 专门记录 JSON 解析失败的情况
+- 📁 **文件日志**: 将日志保存到文件中，便于后续分析
+- 🖥️ **控制台输出**: 实时显示关键信息
+
+## 使用方法
+
+### 方法 1: 使用环境变量
+
+```bash
+# 启用调试日志记录
+source enable_llm_debug.sh
+
+# 运行测试
+python -m tests.run_specific_memory_test episodic_memory_indirect --config mirix/configs/mirix_qwen3.yaml
+```
+
+### 方法 2: 在代码中启用
+
+```python
+from mirix.llm_api.llm_debug_logger import enable_llm_debug_logging
+
+# 启用调试日志记录
+enable_llm_debug_logging(
+    log_dir="./logs/llm_debug",
+    enable_file_logging=True
+)
+
+# 运行你的测试代码
+```
+
+## 日志文件说明
+
+调试日志会保存到 `./logs/llm_debug/` 目录下：
+
+- `llm_requests_YYYYMMDD.log`: 所有 LLM 请求的详细记录
+- `llm_responses_YYYYMMDD.log`: 所有 LLM 响应的详细记录
+- `llm_errors_YYYYMMDD.log`: 所有错误和异常的详细记录
+- `session_*.json`: 完整的调试会话记录（如果使用会话功能）
+
+## 日志内容示例
+
+### 请求日志
+```json
+{
+  "timestamp": "2025-01-08T12:30:00.000Z",
+  "request_id": "req_1704715800000",
+  "model_name": "qwen-plus",
+  "endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1",
+  "request_data": {
+    "model": "qwen-plus",
+    "messages": [...],
+    "tools": [...],
+    "temperature": 0.6,
+    "max_tokens": 4096
+  },
+  "additional_info": {
+    "tools_count": 5,
+    "messages_count": 3,
+    "stream": false
+  }
+}
+```
+
+### 响应日志
+```json
+{
+  "timestamp": "2025-01-08T12:30:01.500Z",
+  "request_id": "req_1704715800000",
+  "response_data": {
+    "choices": [...],
+    "usage": {
+      "prompt_tokens": 1000,
+      "completion_tokens": 200,
+      "total_tokens": 1200
+    }
+  },
+  "response_time_ms": 1500.25,
+  "additional_info": {
+    "success": true
+  }
+}
+```
+
+### 错误日志
+```json
+{
+  "timestamp": "2025-01-08T12:30:01.500Z",
+  "request_id": "req_1704715800000",
+  "error_type": "JSONDecodeError",
+  "error_message": "Extra data: line 1 column 473 (char 472)",
+  "error_context": {
+    "stage": "response_conversion",
+    "response_data_keys": ["choices", "usage"]
+  }
+}
+```
+
+## 控制台输出示例
+
+```
+🚀 LLM Request [req_1704715800000]
+   Model: qwen-plus
+   Endpoint: https://dashscope.aliyuncs.com/compatible-mode/v1
+   Messages Count: 3
+   Message 1: user - This is a test memory about going to the grocery store...
+   Message 2: assistant - I understand you want to store this information...
+   Message 3: user - Please add this to episodic memory
+   Tools: 5 tools available
+     - episodic_memory_insert
+     - episodic_memory_search
+     - episodic_memory_update
+     - episodic_memory_delete
+     - finish_memory_update
+
+📥 LLM Response [req_1704715800000]
+   Response Time: 1500.25ms
+   Choices Count: 1
+   Choice 1: assistant
+     Content: I'll help you store this information in episodic memory...
+     Tool Calls: 1
+       Tool 1: episodic_memory_insert
+         Args: {"items": [{"actor": "user", "details": "The user mentioned going to the grocery store..."}]}
+   Usage: 1000 prompt + 200 completion = 1200 total
+```
+
+## 故障排除
+
+### 常见问题
+
+1. **JSON 解析错误**
+   - 查看 `llm_errors_*.log` 文件中的 JSON 解析错误详情
+   - 检查 LLM 返回的 JSON 格式是否正确
+
+2. **API 调用失败**
+   - 查看请求日志确认发送的数据格式
+   - 查看错误日志了解具体的错误原因
+
+3. **响应时间过长**
+   - 查看响应日志中的 `response_time_ms` 字段
+   - 分析是否有网络或模型性能问题
+
+### 调试技巧
+
+1. **关联请求和响应**: 使用 `request_id` 字段关联请求和响应
+2. **分析错误模式**: 查看错误日志中的重复错误模式
+3. **性能分析**: 使用响应时间数据分析性能瓶颈
+4. **内容验证**: 检查请求和响应内容是否符合预期
+
+## 环境变量配置
+
+| 变量名 | 默认值 | 说明 |
+|--------|--------|------|
+| `LLM_DEBUG_ENABLE_FILE` | `true` | 是否启用文件日志记录 |
+| `LLM_DEBUG_LOG_DIR` | `./logs/llm_debug` | 日志文件保存目录 |
+| `LOG_LEVEL` | `INFO` | 日志级别 |
+
+## 注意事项
+
+- 调试日志记录会增加一些性能开销
+- 日志文件可能会变得很大，建议定期清理
+- 敏感信息（如 API 密钥）会被自动过滤
+- 在生产环境中建议关闭详细的调试日志记录

enable_llm_debug.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+# 启用 LLM 调试日志记录的环境变量设置脚本
+
+echo "🔍 启用 LLM 调试日志记录..."
+
+# 设置环境变量
+export LLM_DEBUG_ENABLE_FILE=true
+export LLM_DEBUG_LOG_DIR="./logs/llm_debug"
+export LOG_LEVEL=DEBUG
+
+echo "✅ 环境变量已设置:"
+echo "   LLM_DEBUG_ENABLE_FILE=$LLM_DEBUG_ENABLE_FILE"
+echo "   LLM_DEBUG_LOG_DIR=$LLM_DEBUG_LOG_DIR"
+echo "   LOG_LEVEL=$LOG_LEVEL"
+echo ""
+echo "📁 日志文件将保存到: $LLM_DEBUG_LOG_DIR"
+echo ""
+echo "🚀 现在可以运行测试了:"
+echo "   python -m tests.run_specific_memory_test episodic_memory_indirect --config mirix/configs/mirix_qwen3.yaml"

mirix/agent/agent_wrapper.py

@@ -801,6 +801,7 @@ def _create_llm_config_for_provider(self, model_name: str, provider: str, custom
                     model_endpoint=custom_agent_config['model_endpoint'],
                     model_wrapper=None,
                     api_key=custom_agent_config.get('api_key'),
+                    put_inner_thoughts_in_kwargs=custom_agent_config.get('put_inner_thoughts_in_kwargs', True),
                     **custom_agent_config.get('generation_config', {})
                 )
             else:
@@ -811,6 +812,7 @@ def _create_llm_config_for_provider(self, model_name: str, provider: str, custom
                     model_endpoint=self.agent_config['model_endpoint'],
                     model_wrapper=None,
                     api_key=self.agent_config.get('api_key'),
+                    put_inner_thoughts_in_kwargs=self.agent_config.get('put_inner_thoughts_in_kwargs', True),
                     **self.agent_config.get('generation_config', {})
                 )
 

mirix/agent/app_constants.py

@@ -1,4 +1,7 @@
-TEMPORARY_MESSAGE_LIMIT = 20
+import os
+
+# 可以通过环境变量 TEMPORARY_MESSAGE_LIMIT 设置，默认为 20
+TEMPORARY_MESSAGE_LIMIT = int(os.environ.get('TEMPORARY_MESSAGE_LIMIT', '20'))
 MAXIMUM_NUM_IMAGES_IN_CLOUD = 600
 
 GEMINI_MODELS = ['gemini-2.0-flash', 'gemini-2.5-flash-lite', 'gemini-1.5-pro', 'gemini-2.0-flash-lite', 'gemini-2.5-flash']

mirix/constants.py

@@ -194,4 +194,4 @@
 CHAINING_FOR_MEMORY_UPDATE = False
 
 LOAD_IMAGE_CONTENT_FOR_LAST_MESSAGE_ONLY = False
-BUILD_EMBEDDINGS_FOR_MEMORY = True
+BUILD_EMBEDDINGS_FOR_MEMORY = os.getenv("BUILD_EMBEDDINGS_FOR_MEMORY", "true").lower() in ("true", "1", "yes", "on")

mirix/llm_api/helpers.py

@@ -1,5 +1,6 @@
 import copy
 import json
+import time
 import warnings
 from collections import OrderedDict
 from typing import Any, List, Union
@@ -293,6 +294,19 @@ def unpack_inner_thoughts_from_kwargs(choice: Choice, inner_thoughts_key: str) -
                 warnings.warn(f"Did not find inner thoughts in tool call: {str(tool_call)}")
 
         except json.JSONDecodeError as e:
+            # 记录详细的 JSON 解析错误信息
+            try:
+                from mirix.llm_api.llm_debug_logger import get_llm_debug_logger
+                debug_logger = get_llm_debug_logger()
+                debug_logger.log_json_parse_error(
+                    request_id=f"inner_thoughts_{int(time.time() * 1000)}",
+                    json_string=tool_call.function.arguments,
+                    error=e,
+                    context="inner_thoughts_parsing"
+                )
+            except ImportError:
+                pass  # 如果调试日志记录器不可用，继续使用原来的警告
+
             warnings.warn(f"Failed to strip inner thoughts from kwargs: {e}")
             raise e
     else:

mirix/llm_api/llm_client_base.py

@@ -1,3 +1,4 @@
+import time
 from abc import abstractmethod
 from typing import Dict, List, Optional, Union
 
@@ -7,6 +8,7 @@
 from mirix.schemas.openai.chat_completion_response import ChatCompletionResponse
 from mirix.services.cloud_file_mapping_manager import CloudFileMappingManager
 from mirix.services.file_manager import FileManager
+from mirix.llm_api.llm_debug_logger import get_llm_debug_logger
 
 class LLMClientBase:
     """
@@ -38,19 +40,72 @@ def send_llm_request(
         """
         Issues a request to the downstream model endpoint and parses response.
         """
+        # 获取调试日志记录器
+        debug_logger = get_llm_debug_logger()
+
         request_data = self.build_request_data(messages, self.llm_config, tools, force_tool_call, existing_file_uris=existing_file_uris)
 
         if get_input_data_for_debugging:
             return request_data
 
+        # 记录请求
+        request_id = debug_logger.log_request(
+            model_name=self.llm_config.model,
+            endpoint=self.llm_config.model_endpoint,
+            request_data=request_data,
+            additional_info={
+                "tools_count": len(tools) if tools else 0,
+                "messages_count": len(messages),
+                "stream": stream,
+                "force_tool_call": force_tool_call
+            }
+        )
+
+        start_time = time.time()
+
         try:
             response_data = self.request(request_data)
+            response_time_ms = (time.time() - start_time) * 1000
+
+            # 记录成功响应
+            debug_logger.log_response(
+                request_id=request_id,
+                response_data=response_data,
+                response_time_ms=response_time_ms,
+                additional_info={
+                    "success": True
+                }
+            )
+
         except Exception as e:
+            response_time_ms = (time.time() - start_time) * 1000
+
+            # 记录错误
+            debug_logger.log_error(
+                request_id=request_id,
+                error=e,
+                error_context={
+                    "response_time_ms": response_time_ms,
+                    "model_name": self.llm_config.model,
+                    "endpoint": self.llm_config.model_endpoint
+                }
+            )
             raise self.handle_llm_error(e)
 
-        chat_completion_data = self.convert_response_to_chat_completion(response_data, messages)
-
-        return chat_completion_data
+        try:
+            chat_completion_data = self.convert_response_to_chat_completion(response_data, messages)
+            return chat_completion_data
+        except Exception as e:
+            # 记录响应转换错误
+            debug_logger.log_error(
+                request_id=request_id,
+                error=e,
+                error_context={
+                    "stage": "response_conversion",
+                    "response_data_keys": list(response_data.keys()) if isinstance(response_data, dict) else "not_dict"
+                }
+            )
+            raise
 
     @abstractmethod
     def build_request_data(