diff --git a/dify-plugin-tdai-memory/.difyignore b/dify-plugin-tdai-memory/.difyignore new file mode 100644 index 00000000..670d19f9 --- /dev/null +++ b/dify-plugin-tdai-memory/.difyignore @@ -0,0 +1,14 @@ +__pycache__/ +*.pyc +.pytest_cache/ +.venv/ +tests/ +scripts/ +.git/ +.DS_Store +.idea/ +.vscode/ +*.egg-info/ +dist/ +.env +.env.* diff --git a/dify-plugin-tdai-memory/ARCHITECTURE.md b/dify-plugin-tdai-memory/ARCHITECTURE.md new file mode 100644 index 00000000..3d195d22 --- /dev/null +++ b/dify-plugin-tdai-memory/ARCHITECTURE.md @@ -0,0 +1,67 @@ +# Dify Adapter Architecture + +The Dify adapter is intentionally thin. It does not embed memory logic and does +not call an LLM directly. It maps Dify tool invocations onto the existing +TencentDB Agent Memory Gateway API. + +## Call Chain + +```text +Dify workflow + -> Dify tool plugin + -> TdaiGatewayClient + -> TencentDB Agent Memory Gateway + -> TdaiCore + -> L0/L1/L2/L3 memory pipeline and stores +``` + +## Responsibilities + +| Layer | Responsibility | +| --- | --- | +| Dify workflow | Supplies `conversation_id`, user text, assistant text, and decides where recalled context is injected. | +| Dify tool plugin | Validates tool parameters, truncates prompt-bound output, and returns non-throwing JSON payloads. | +| `TdaiGatewayClient` | Sends JSON requests to Gateway endpoints and normalizes HTTP failures. | +| Gateway | Owns HTTP auth, request validation, session flush, recall, capture, and search routes. | +| `TdaiCore` | Owns host-neutral memory orchestration and the progressive memory pipeline. | + +## Recall Flow + +1. Dify calls `tdai_recall` before an LLM node. +2. The plugin sends `POST /recall` with `query` and `session_key`. +3. Gateway calls `TdaiCore.handleBeforeRecall`. +4. The plugin returns `{ "ok": true, "context": "..." }`. +5. The Dify workflow injects `context` into the LLM prompt. + +## Capture Flow + +1. Dify calls `tdai_capture` after the assistant response is available. +2. The plugin sends `POST /capture` with the user message, assistant response, + and the same `session_key`. +3. Gateway calls `TdaiCore.handleTurnCommitted`. +4. L0 conversation capture happens synchronously; downstream extraction is + scheduled by the existing core pipeline. + +## Session Model + +Use Dify `conversation_id` as `session_key`. Do not use Dify run IDs because +they change between workflow executions and would fragment memory. + +The current Gateway request types accept `user_id`, but core isolation still +follows the existing Gateway behavior. Treat `session_key` as the primary Dify +isolation key until Gateway/Core user scoping is extended. + +## Credential Validation + +`GET /health` is intentionally unauthenticated. To verify Bearer credentials, +the provider validation path sends one read-only memory search request: + +```text +POST /search/memories +query = "__dify_credential_validation__" +limit = 1 +``` + +This does not write memory, but it can appear in Gateway logs, traces, or +metrics. A future Gateway `OPTIONS` or authenticated handshake endpoint could +replace this probe without changing the tool API. diff --git a/dify-plugin-tdai-memory/README.md b/dify-plugin-tdai-memory/README.md new file mode 100644 index 00000000..f0339522 --- /dev/null +++ b/dify-plugin-tdai-memory/README.md @@ -0,0 +1,61 @@ +# TencentDB Agent Memory for Dify + +This Dify tool plugin connects Dify workflows to TencentDB Agent Memory through +the local HTTP Gateway. It does not start the Gateway by itself. + +## Quickstart + +For a runnable smoke test that starts the Gateway, starts a mock Dify server, +and invokes the real `tdai_capture` and `tdai_recall` tool classes: + +```bash +npm install +bash dify-plugin-tdai-memory/scripts/quickstart-gateway-mock-e2e.sh +``` + +Manual setup: + +1. Start the Gateway from the repository root: + + ```bash + npx tsx src/gateway/server.ts + ``` + +2. In Dify, install this plugin directory and configure provider credentials: + + - `gateway_url`: `http://127.0.0.1:8420` + - `gateway_api_key`: optional; must match `TDAI_GATEWAY_API_KEY` if Gateway auth is enabled + - `gateway_timeout_seconds`: optional, default `10` + +3. Recommended workflow wiring: + + - Before the LLM node: call `tdai_recall` with the current user message and Dify `conversation_id`. + - Inject the returned `context` into the system prompt or context field. + - After the assistant response: call `tdai_capture` with user/assistant content and the same `conversation_id`. + - At workflow `End`: call `tdai_session_end` to flush the current session buffer. + +## Tools + +| Tool | Gateway endpoint | Purpose | +| --- | --- | --- | +| `tdai_health` | `GET /health` | Check Gateway availability | +| `tdai_recall` | `POST /recall` | Recall memory context before generation | +| `tdai_capture` | `POST /capture` | Store a completed conversation turn | +| `tdai_memory_search` | `POST /search/memories` | Search structured L1 memories | +| `tdai_conversation_search` | `POST /search/conversations` | Search raw L0 conversations | +| `tdai_session_end` | `POST /session/end` | Flush the current session buffer | + +## Notes + +- Use Dify `conversation_id` as `session_key`; do not use transient run IDs. +- Tool calls return `{ "ok": false, "error": "..." }` on Gateway/network failure so workflows can continue without memory. +- Search outputs are truncated by default to 2000 characters to avoid oversized prompt injection. Set `max_chars` to `0` for unlimited output. +- The current Gateway accepts `user_id` in recall/capture request bodies, but core identity handling still uses the existing Gateway behavior. Treat `session_key` as the primary isolation key until Gateway/Core user scoping is extended. +- Provider credential validation sends a read-only `POST /search/memories` probe because `GET /health` is intentionally unauthenticated. The probe does not write memory, but it can appear in Gateway logs or metrics. + +## Architecture Docs + +- [Dify adapter architecture](ARCHITECTURE.md) +- Repository guide: `docs/dify-plugin-installation-guide.md` +- [Dify workflow diagram](../docs/dify-workflow-diagram.md) +- [Cross-platform adapter comparison](../docs/cross-platform-comparison.md) diff --git a/dify-plugin-tdai-memory/_assets/icon.svg b/dify-plugin-tdai-memory/_assets/icon.svg new file mode 100644 index 00000000..ef5c159c --- /dev/null +++ b/dify-plugin-tdai-memory/_assets/icon.svg @@ -0,0 +1,5 @@ + + + + + diff --git a/dify-plugin-tdai-memory/main.py b/dify-plugin-tdai-memory/main.py new file mode 100644 index 00000000..b232d8cb --- /dev/null +++ b/dify-plugin-tdai-memory/main.py @@ -0,0 +1,21 @@ +import os + +from dify_plugin import DifyPluginEnv, Plugin + + +DEFAULT_REQUEST_TIMEOUT = 120 + + +def _max_request_timeout() -> int: + try: + timeout = int(os.environ.get("MAX_REQUEST_TIMEOUT", str(DEFAULT_REQUEST_TIMEOUT))) + except (TypeError, ValueError): + return DEFAULT_REQUEST_TIMEOUT + return timeout if timeout > 0 else DEFAULT_REQUEST_TIMEOUT + + +plugin = Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=_max_request_timeout())) + + +if __name__ == "__main__": + plugin.run() diff --git a/dify-plugin-tdai-memory/manifest.yaml b/dify-plugin-tdai-memory/manifest.yaml new file mode 100644 index 00000000..1da38704 --- /dev/null +++ b/dify-plugin-tdai-memory/manifest.yaml @@ -0,0 +1,34 @@ +version: 0.0.1 +type: plugin +author: tencentdb-agent-memory +name: tdai_memory +label: + en_US: TencentDB Agent Memory + zh_Hans: 腾讯云数据库 Agent Memory +description: + en_US: Connect Dify workflows to TencentDB Agent Memory through the local HTTP Gateway. + zh_Hans: 通过本地 HTTP Gateway 将 Dify 工作流接入 TencentDB Agent Memory。 +created_at: 2026-07-03T00:00:00.000Z +icon: icon.svg +tags: + - utilities +resource: + memory: 1048576 + permission: + tool: + enabled: true + model: + enabled: false + llm: false +plugins: + tools: + - provider/tdai_memory.yaml +meta: + version: 0.0.1 + arch: + - amd64 + - arm64 + runner: + language: python + version: "3.12" + entrypoint: main diff --git a/dify-plugin-tdai-memory/provider/tdai_memory.py b/dify-plugin-tdai-memory/provider/tdai_memory.py new file mode 100644 index 00000000..868ad0c2 --- /dev/null +++ b/dify-plugin-tdai-memory/provider/tdai_memory.py @@ -0,0 +1,30 @@ +"""Dify provider for TencentDB Agent Memory Gateway credentials.""" + +from __future__ import annotations + +from typing import Any + +from dify_plugin import ToolProvider +from dify_plugin.errors.tool import ToolProviderCredentialValidationError + +from tools.client import TdaiGatewayClient, TdaiGatewayError + + +class TdaiMemoryProvider(ToolProvider): + """Validate Dify provider credentials against the Gateway.""" + + def _validate_credentials(self, credentials: dict[str, Any]) -> None: + gateway_url = str(credentials.get("gateway_url") or "").strip() + if not gateway_url: + raise ToolProviderCredentialValidationError("Gateway URL is required") + + try: + client = TdaiGatewayClient.from_credentials(credentials) + # `/health` is intentionally unauthenticated, so use a read-only + # search request to validate Bearer credentials when auth is on. + client.search_memories("__dify_credential_validation__", limit=1) + except TdaiGatewayError as exc: + message = "Gateway credential validation failed" + if exc.status_code: + message = f"{message} (HTTP {exc.status_code})" + raise ToolProviderCredentialValidationError(message) from exc diff --git a/dify-plugin-tdai-memory/provider/tdai_memory.yaml b/dify-plugin-tdai-memory/provider/tdai_memory.yaml new file mode 100644 index 00000000..9c9bc207 --- /dev/null +++ b/dify-plugin-tdai-memory/provider/tdai_memory.yaml @@ -0,0 +1,56 @@ +identity: + author: tencentdb-agent-memory + name: tdai_memory + label: + en_US: TencentDB Agent Memory + zh_Hans: 腾讯云数据库 Agent Memory + description: + en_US: Local long-term memory tools backed by TencentDB Agent Memory Gateway. + zh_Hans: 基于 TencentDB Agent Memory Gateway 的本地长期记忆工具。 + icon: icon.svg + tags: + - utilities +credentials_for_provider: + gateway_url: + type: text-input + required: true + label: + en_US: Gateway URL + zh_Hans: Gateway 地址 + placeholder: + en_US: http://127.0.0.1:8420 + zh_Hans: http://127.0.0.1:8420 + help: + en_US: Start the TDAI Gateway before using these tools. + zh_Hans: 使用这些工具前请先启动 TDAI Gateway。 + gateway_api_key: + type: secret-input + required: false + label: + en_US: Gateway API Key + zh_Hans: Gateway API Key + placeholder: + en_US: Optional Bearer token + zh_Hans: 可选 Bearer Token + help: + en_US: Must match TDAI_GATEWAY_API_KEY when Gateway auth is enabled. + zh_Hans: Gateway 开启鉴权时必须与 TDAI_GATEWAY_API_KEY 一致。 + gateway_timeout_seconds: + type: text-input + required: false + label: + en_US: Timeout Seconds + zh_Hans: 超时秒数 + placeholder: + en_US: "10" + zh_Hans: "10" +extra: + python: + source: provider/tdai_memory.py +tools: + - tools/tdai_health.yaml + - tools/tdai_recall.yaml + - tools/tdai_capture.yaml + - tools/tdai_memory_search.yaml + - tools/tdai_conversation_search.yaml + - tools/tdai_session_end.yaml diff --git a/dify-plugin-tdai-memory/pyproject.toml b/dify-plugin-tdai-memory/pyproject.toml new file mode 100644 index 00000000..ce6f5853 --- /dev/null +++ b/dify-plugin-tdai-memory/pyproject.toml @@ -0,0 +1,17 @@ +[project] +name = "dify-plugin-tdai-memory" +version = "0.0.1" +description = "Connect Dify workflows to TencentDB Agent Memory through the local HTTP Gateway." +readme = "README.md" +requires-python = ">=3.12" +authors = [ + { name = "tencentdb-agent-memory" }, +] +license = { text = "MIT" } +dependencies = [ + "dify_plugin~=0.9.0", +] + +[build-system] +requires = ["setuptools>=64.0", "wheel"] +build-backend = "setuptools.build_meta" diff --git a/dify-plugin-tdai-memory/scripts/mock_dify_plugin_server.py b/dify-plugin-tdai-memory/scripts/mock_dify_plugin_server.py new file mode 100644 index 00000000..51da6a3d --- /dev/null +++ b/dify-plugin-tdai-memory/scripts/mock_dify_plugin_server.py @@ -0,0 +1,208 @@ +#!/usr/bin/env python3 +"""Small local server that invokes the Dify tool classes without Dify. + +This is a quickstart/e2e harness, not production runtime. It installs a tiny +`dify_plugin` stub, loads the real tool classes, and forwards invocations to +the configured TencentDB Agent Memory Gateway. +""" + +from __future__ import annotations + +import argparse +import importlib +import json +import os +import sys +import threading +import types +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from types import SimpleNamespace +from typing import Any + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +PLUGIN_ROOT_TEXT = str(PLUGIN_ROOT) +if PLUGIN_ROOT_TEXT not in sys.path: + sys.path.insert(0, PLUGIN_ROOT_TEXT) + +TOOL_CLASSES = { + "tdai_health": ("tools.tdai_health", "TdaiHealthTool"), + "tdai_recall": ("tools.tdai_recall", "TdaiRecallTool"), + "tdai_capture": ("tools.tdai_capture", "TdaiCaptureTool"), + "tdai_memory_search": ("tools.tdai_memory_search", "TdaiMemorySearchTool"), + "tdai_conversation_search": ("tools.tdai_conversation_search", "TdaiConversationSearchTool"), + "tdai_session_end": ("tools.tdai_session_end", "TdaiSessionEndTool"), +} +MAX_BODY_BYTES = 1_048_576 +_api_key_cache: str | None = None +_api_key_lock = threading.Lock() + + +class _StubTool: + def create_json_message(self, payload: dict[str, Any]) -> dict[str, Any]: + return payload + + +class _StubToolProvider: + pass + + +class _StubToolProviderCredentialValidationError(Exception): + pass + + +def install_dify_stubs() -> None: + dify_plugin = sys.modules.get("dify_plugin") or types.ModuleType("dify_plugin") + if not hasattr(dify_plugin, "Tool"): + dify_plugin.Tool = _StubTool + if not hasattr(dify_plugin, "ToolProvider"): + dify_plugin.ToolProvider = _StubToolProvider + + entities = sys.modules.get("dify_plugin.entities") or types.ModuleType("dify_plugin.entities") + tool_module = sys.modules.get("dify_plugin.entities.tool") or types.ModuleType("dify_plugin.entities.tool") + if not hasattr(tool_module, "ToolInvokeMessage"): + tool_module.ToolInvokeMessage = dict + + errors = sys.modules.get("dify_plugin.errors") or types.ModuleType("dify_plugin.errors") + error_tool_module = sys.modules.get("dify_plugin.errors.tool") or types.ModuleType("dify_plugin.errors.tool") + if not hasattr(error_tool_module, "ToolProviderCredentialValidationError"): + error_tool_module.ToolProviderCredentialValidationError = _StubToolProviderCredentialValidationError + + sys.modules["dify_plugin"] = dify_plugin + sys.modules["dify_plugin.entities"] = entities + sys.modules["dify_plugin.entities.tool"] = tool_module + sys.modules["dify_plugin.errors"] = errors + sys.modules["dify_plugin.errors.tool"] = error_tool_module + + +install_dify_stubs() + + +def gateway_credentials() -> dict[str, Any]: + return { + "gateway_url": os.environ.get("TDAI_DIFY_GATEWAY_URL", "http://127.0.0.1:8420"), + "gateway_api_key": _gateway_api_key(), + "gateway_timeout_seconds": os.environ.get("TDAI_DIFY_GATEWAY_TIMEOUT_SECONDS", "10"), + } + + +def _gateway_api_key() -> str: + global _api_key_cache + if _api_key_cache is not None: + return _api_key_cache + + with _api_key_lock: + if _api_key_cache is not None: + return _api_key_cache + + key_file = os.environ.get("TDAI_DIFY_GATEWAY_API_KEY_FILE", "") + if key_file: + key_path = Path(key_file) + try: + _api_key_cache = key_path.read_text(encoding="utf-8").strip() + except (FileNotFoundError, PermissionError) as exc: + raise ValueError(f"cannot read gateway API key file: {key_path}") from exc + key_path.unlink(missing_ok=True) + return _api_key_cache + + _api_key_cache = os.environ.get("TDAI_DIFY_GATEWAY_API_KEY", "") + return _api_key_cache + + +def invoke_tool(tool_name: str, parameters: dict[str, Any], credentials: dict[str, Any]) -> dict[str, Any]: + if not str(credentials.get("gateway_url") or "").strip(): + return {"ok": False, "operation": tool_name, "error": "gateway_url is required"} + module_name, class_name = TOOL_CLASSES[tool_name] + module = importlib.import_module(module_name) + tool_class = getattr(module, class_name) + tool = tool_class() + tool.runtime = SimpleNamespace(credentials=credentials) + messages = list(tool._invoke(parameters)) + if not messages: + return {"ok": False, "operation": tool_name, "error": "tool produced no output"} + message = messages[0] + if isinstance(message, dict): + return message + return {"ok": False, "operation": tool_name, "error": f"unexpected output: {type(message).__name__}"} + + +class MockDifyHandler(BaseHTTPRequestHandler): + server_version = "TdaiDifyMock/0.1" + + def do_GET(self) -> None: + if self.path == "/health": + self._send({"ok": True, "tools": sorted(TOOL_CLASSES)}) + return + self._send({"ok": False, "error": "not found"}, status=404) + + def do_POST(self) -> None: + prefix = "/invoke/" + if not self.path.startswith(prefix): + self._send({"ok": False, "error": "not found"}, status=404) + return + tool_name = self.path[len(prefix) :] + if tool_name not in TOOL_CLASSES: + self._send({"ok": False, "error": f"unknown tool: {tool_name}"}, status=404) + return + try: + parameters = self._read_json() + result = invoke_tool(tool_name, parameters, gateway_credentials()) + self._send(result) + except ValueError as exc: + self._send({"ok": False, "error": str(exc)}, status=400) + except Exception as exc: + self._send({"ok": False, "error": f"mock server failed: {exc}"}, status=500) + + def log_message(self, format: str, *args: Any) -> None: + if self.path != "/health": + super().log_message(format, *args) + + def _read_json(self) -> dict[str, Any]: + raw_length = self.headers.get("Content-Length", "0") + try: + length = int(raw_length) + except (TypeError, ValueError) as exc: + raise ValueError(f"invalid Content-Length header: {raw_length!r}") from exc + if length < 0: + raise ValueError("invalid Content-Length header: negative length") + if length > MAX_BODY_BYTES: + raise ValueError(f"request body exceeds {MAX_BODY_BYTES} bytes") + try: + raw = self.rfile.read(length).decode("utf-8") + except UnicodeDecodeError as exc: + raise ValueError("request body must be valid UTF-8") from exc + if not raw: + return {} + parsed = json.loads(raw) + if not isinstance(parsed, dict): + raise ValueError("request body must be a JSON object") + return parsed + + def _send(self, body: dict[str, Any], *, status: int = 200) -> None: + data = json.dumps(body, ensure_ascii=False).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +def main() -> None: + parser = argparse.ArgumentParser(description="Run a local mock Dify server for TDAI tools.") + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=18420) + args = parser.parse_args() + + server = ThreadingHTTPServer((args.host, args.port), MockDifyHandler) + print(f"Mock Dify server listening on http://{args.host}:{args.port}", flush=True) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + finally: + server.server_close() + + +if __name__ == "__main__": + main() diff --git a/dify-plugin-tdai-memory/scripts/quickstart-gateway-mock-e2e.sh b/dify-plugin-tdai-memory/scripts/quickstart-gateway-mock-e2e.sh new file mode 100644 index 00000000..b86cc5be --- /dev/null +++ b/dify-plugin-tdai-memory/scripts/quickstart-gateway-mock-e2e.sh @@ -0,0 +1,195 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PLUGIN_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +REPO_ROOT="$(cd "$PLUGIN_DIR/.." && pwd)" + +GATEWAY_HOST="${TDAI_GATEWAY_HOST:-127.0.0.1}" +GATEWAY_PORT="${TDAI_GATEWAY_PORT:-8420}" +GATEWAY_URL="${TDAI_DIFY_GATEWAY_URL:-http://${GATEWAY_HOST}:${GATEWAY_PORT}}" +MOCK_HOST="${TDAI_DIFY_MOCK_HOST:-127.0.0.1}" +MOCK_PORT="${TDAI_DIFY_MOCK_PORT:-18420}" +MOCK_URL="http://${MOCK_HOST}:${MOCK_PORT}" +SESSION_KEY="${TDAI_DIFY_E2E_SESSION_KEY:-dify-quickstart-session}" + +TMP_DIR="" +GATEWAY_PID="" +MOCK_PID="" + +cleanup() { + if [ -n "$MOCK_PID" ] && kill -0 "$MOCK_PID" 2>/dev/null; then + kill "$MOCK_PID" 2>/dev/null || true + fi + if [ -n "$GATEWAY_PID" ] && kill -0 "$GATEWAY_PID" 2>/dev/null; then + kill "$GATEWAY_PID" 2>/dev/null || true + fi + if [ -z "${TMP_DIR:-}" ]; then + return + fi + rm -f "$TMP_DIR/gateway-api-key" + if [ "${TDAI_E2E_KEEP_LOGS:-0}" != "1" ]; then + rm -rf "$TMP_DIR" + else + echo "Logs kept in $TMP_DIR" + fi +} +trap cleanup EXIT INT TERM + +TMP_DIR="$(mktemp -d)" || { + echo "Failed to create temp directory" >&2 + exit 1 +} + +find_python() { + if command -v python3 >/dev/null 2>&1; then + command -v python3 + return + fi + if command -v python >/dev/null 2>&1; then + command -v python + return + fi + echo "python3 or python is required" >&2 + exit 1 +} + +find_curl() { + if command -v curl >/dev/null 2>&1; then + return + fi + echo "curl is required" >&2 + exit 1 +} + +PYTHON_BIN="${PYTHON_BIN:-$(find_python)}" +find_curl + +wait_for_url() { + local url="$1" + local label="$2" + local attempts="${3:-60}" + for ((i = 0; i < attempts; i++)); do + if curl -fsS "$url" >/dev/null 2>&1; then + return 0 + fi + sleep 1 + done + echo "Timed out waiting for $label at $url" >&2 + return 1 +} + +post_json() { + local url="$1" + local body="$2" + curl -fsS -X POST "$url" \ + -H "Content-Type: application/json" \ + --data "$body" +} + +assert_ok_json() { + "$PYTHON_BIN" -c ' +import json +import sys + +payload = json.load(sys.stdin) +if payload.get("ok") is not True: + raise SystemExit(f"tool invocation failed: {payload}") +print(json.dumps(payload, ensure_ascii=False, sort_keys=True), file=sys.stderr) +' +} + +make_capture_body() { + SESSION_KEY="$SESSION_KEY" "$PYTHON_BIN" <<'PY' +import json +import os + +print(json.dumps({ + "user_content": "Please remember that Dify uses conversation_id as session_key.", + "assistant_content": "Stored. Use the same session_key for recall and capture.", + "session_key": os.environ["SESSION_KEY"], +})) +PY +} + +make_recall_body() { + SESSION_KEY="$SESSION_KEY" "$PYTHON_BIN" <<'PY' +import json +import os + +print(json.dumps({ + "query": "How should Dify identify the memory session?", + "session_key": os.environ["SESSION_KEY"], + "max_chars": 2000, +})) +PY +} + +echo "[1/4] Starting or reusing TencentDB Agent Memory Gateway at $GATEWAY_URL" +if curl -fsS "$GATEWAY_URL/health" >/dev/null 2>&1; then + echo "Gateway already healthy" +else + if ! command -v npx >/dev/null 2>&1; then + echo "npx is required to start the Gateway. Install Node.js, which includes npx." >&2 + exit 1 + fi + if [ ! -d "$REPO_ROOT/node_modules" ]; then + echo "node_modules not found. Run 'npm install' from $REPO_ROOT before starting a new Gateway." >&2 + echo "If a Gateway is already running elsewhere, set TDAI_DIFY_GATEWAY_URL to reuse it." >&2 + exit 1 + fi + export TDAI_GATEWAY_HOST="$GATEWAY_HOST" + export TDAI_GATEWAY_PORT="$GATEWAY_PORT" + export TDAI_DATA_DIR="${TDAI_DATA_DIR:-$TMP_DIR/memory-tdai}" + ( + cd "$REPO_ROOT" + exec npx tsx src/gateway/server.ts + ) >"$TMP_DIR/gateway.log" 2>&1 & + GATEWAY_PID="$!" + wait_for_url "$GATEWAY_URL/health" "Gateway" 90 || { + echo "--- Gateway log ---" >&2 + cat "$TMP_DIR/gateway.log" >&2 || true + exit 1 + } +fi + +echo "[2/4] Starting mock Dify plugin server at $MOCK_URL" +API_KEY="${TDAI_DIFY_GATEWAY_API_KEY:-${TDAI_GATEWAY_API_KEY:-}}" +API_KEY_FILE="" +if [ -n "$API_KEY" ]; then + API_KEY_FILE="$TMP_DIR/gateway-api-key" + (umask 077 && printf '%s' "$API_KEY" >"$API_KEY_FILE") || { + echo "Failed to write API key to $API_KEY_FILE" >&2 + exit 1 + } +fi +unset TDAI_DIFY_GATEWAY_API_KEY TDAI_GATEWAY_API_KEY +TDAI_DIFY_GATEWAY_URL="$GATEWAY_URL" \ +TDAI_DIFY_GATEWAY_API_KEY_FILE="$API_KEY_FILE" \ +"$PYTHON_BIN" "$PLUGIN_DIR/scripts/mock_dify_plugin_server.py" \ + --host "$MOCK_HOST" \ + --port "$MOCK_PORT" >"$TMP_DIR/mock-dify.log" 2>&1 & +MOCK_PID="$!" +wait_for_url "$MOCK_URL/health" "mock Dify server" 30 || { + echo "--- Mock Dify server log ---" >&2 + cat "$TMP_DIR/mock-dify.log" >&2 || true + exit 1 +} + +echo "[3/4] Capturing a completed Dify turn through tdai_capture" +CAPTURE_BODY="$(make_capture_body)" +CAPTURE_RESPONSE="$(post_json "$MOCK_URL/invoke/tdai_capture" "$CAPTURE_BODY")" || { + echo "Failed to invoke tdai_capture" >&2 + exit 1 +} +printf '%s' "$CAPTURE_RESPONSE" | assert_ok_json + +echo "[4/4] Recalling memory through tdai_recall" +RECALL_BODY="$(make_recall_body)" +RECALL_RESPONSE="$(post_json "$MOCK_URL/invoke/tdai_recall" "$RECALL_BODY")" || { + echo "Failed to invoke tdai_recall" >&2 + exit 1 +} +printf '%s' "$RECALL_RESPONSE" | assert_ok_json + +echo "Quickstart e2e succeeded: Gateway -> mock Dify server -> capture -> recall" diff --git a/dify-plugin-tdai-memory/tests/test_gateway_client.py b/dify-plugin-tdai-memory/tests/test_gateway_client.py new file mode 100644 index 00000000..ce710ef7 --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_gateway_client.py @@ -0,0 +1,345 @@ +from __future__ import annotations + +import json +import sys +import threading +import unittest +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +PLUGIN_ROOT_TEXT = str(PLUGIN_ROOT) +if PLUGIN_ROOT_TEXT not in sys.path: + sys.path.insert(0, PLUGIN_ROOT_TEXT) + +from tools.client import TdaiGatewayClient, TdaiGatewayError # noqa: E402 + + +class _GatewayHandler(BaseHTTPRequestHandler): + response_status = 200 + response_body: dict[str, Any] = {} + response_content_type = "application/json" + response_raw_body: str | None = None + response_headers: dict[str, str] = {} + requests: list[dict[str, Any]] = [] + _state_lock = threading.Lock() + + def do_GET(self) -> None: + self._record_request(None) + self._send_json() + + def do_POST(self) -> None: + length = int(self.headers.get("Content-Length", "0")) + raw = self.rfile.read(length).decode("utf-8") + self._record_request(json.loads(raw) if raw else None) + self._send_json() + + def log_message(self, format: str, *args: Any) -> None: + return + + def _record_request(self, body: dict[str, Any] | None) -> None: + with self._state_lock: + self.requests.append( + { + "method": self.command, + "path": self.path, + "headers": dict(self.headers), + "body": body, + } + ) + + def _send_json(self) -> None: + with self._state_lock: + raw = self.response_raw_body + body = self.response_body + status = self.response_status + content_type = self.response_content_type + headers = dict(self.response_headers) + data = (raw if raw is not None else json.dumps(body)).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", content_type) + for name, value in headers.items(): + self.send_header(name, value) + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +class TdaiGatewayClientTest(unittest.TestCase): + def setUp(self) -> None: + with _GatewayHandler._state_lock: + _GatewayHandler.response_status = 200 + _GatewayHandler.response_body = {} + _GatewayHandler.response_content_type = "application/json" + _GatewayHandler.response_raw_body = None + _GatewayHandler.response_headers = {} + _GatewayHandler.requests = [] + self.server = ThreadingHTTPServer(("127.0.0.1", 0), _GatewayHandler) + self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) + self.thread.start() + host, port = self.server.server_address + self.base_url = f"http://{host}:{port}" + + def tearDown(self) -> None: + self.server.shutdown() + self.server.server_close() + self.thread.join(timeout=5) + + def _gateway_requests(self) -> list[dict[str, Any]]: + with _GatewayHandler._state_lock: + return list(_GatewayHandler.requests) + + def _set_gateway_response( + self, + body: dict[str, Any], + *, + status: int = 200, + content_type: str = "application/json", + raw_body: str | None = None, + headers: dict[str, str] | None = None, + ) -> None: + with _GatewayHandler._state_lock: + _GatewayHandler.response_status = status + _GatewayHandler.response_body = body + _GatewayHandler.response_content_type = content_type + _GatewayHandler.response_raw_body = raw_body + _GatewayHandler.response_headers = headers or {} + + def test_recall_posts_json_with_bearer_auth(self) -> None: + self._set_gateway_response( + { + "context": "remember TypeScript imports", + "strategy": "hybrid", + "memory_count": 2, + } + ) + + client = TdaiGatewayClient(self.base_url, api_key="secret-token", timeout=2) + result = client.recall("What should I remember?", "dify-conv-1", user_id="u-42") + + self.assertEqual(result["context"], "remember TypeScript imports") + requests = self._gateway_requests() + self.assertEqual(len(requests), 1) + request = requests[0] + self.assertEqual(request["method"], "POST") + self.assertEqual(request["path"], "/recall") + self.assertEqual(request["headers"]["Authorization"], "Bearer secret-token") + self.assertEqual(request["headers"]["Content-Type"], "application/json") + self.assertEqual( + request["body"], + { + "query": "What should I remember?", + "session_key": "dify-conv-1", + "user_id": "u-42", + }, + ) + + def test_gateway_error_exposes_status_code_and_error_code(self) -> None: + self._set_gateway_response({"error": "Unauthorized", "code": "UNAUTHORIZED"}, status=401) + + client = TdaiGatewayClient(self.base_url, api_key="wrong", timeout=2) + + with self.assertRaises(TdaiGatewayError) as caught: + client.search_memories("anything", limit=3) + + self.assertEqual(caught.exception.status_code, 401) + self.assertEqual(caught.exception.code, "UNAUTHORIZED") + self.assertIn("Unauthorized", str(caught.exception)) + + def test_health_uses_get_without_json_body(self) -> None: + self._set_gateway_response( + { + "status": "ok", + "version": "0.3.6", + "uptime": 10, + "stores": {"vectorStore": True, "embeddingService": False}, + } + ) + + client = TdaiGatewayClient(self.base_url, timeout=2) + result = client.health() + + self.assertEqual(result["status"], "ok") + requests = self._gateway_requests() + self.assertEqual(len(requests), 1) + request = requests[0] + self.assertEqual(request["method"], "GET") + self.assertEqual(request["path"], "/health") + self.assertNotIn("Content-Type", request["headers"]) + + def test_non_json_success_response_exposes_status_and_body(self) -> None: + self._set_gateway_response({}, content_type="text/plain", raw_body="proxy returned html") + + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError) as caught: + client.health() + + self.assertEqual(caught.exception.status_code, 200) + self.assertEqual(caught.exception.response, "proxy returned html") + self.assertIn("Unexpected Content-Type", str(caught.exception)) + + def test_json_suffix_content_type_is_accepted(self) -> None: + self._set_gateway_response({"status": "ok"}, content_type="application/vnd.api+json") + + client = TdaiGatewayClient(self.base_url, timeout=2) + result = client.health() + + self.assertEqual(result["status"], "ok") + + def test_empty_non_json_success_response_is_rejected(self) -> None: + self._set_gateway_response({}, content_type="text/html", raw_body="") + + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError) as caught: + client.health() + + self.assertEqual(caught.exception.status_code, 200) + self.assertIn("Unexpected Content-Type", str(caught.exception)) + + def test_empty_json_success_response_returns_empty_object(self) -> None: + self._set_gateway_response({}, raw_body="") + + client = TdaiGatewayClient(self.base_url, timeout=2) + result = client.health() + + self.assertEqual(result, {}) + + def test_non_object_json_success_response_is_rejected(self) -> None: + self._set_gateway_response({}, raw_body='["not", "an", "object"]') + + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError) as caught: + client.health() + + self.assertEqual(caught.exception.status_code, 200) + self.assertEqual(caught.exception.response, ["not", "an", "object"]) + self.assertIn("Expected JSON object", str(caught.exception)) + self.assertIn("HTTP 200", str(caught.exception)) + + def test_redirect_response_is_not_followed(self) -> None: + self._set_gateway_response({}, status=302, headers={"Location": "/redirected"}) + + client = TdaiGatewayClient(self.base_url, api_key="secret-token", timeout=2) + + with self.assertRaises(TdaiGatewayError) as caught: + client.recall("query", "dify-conv-1") + + self.assertEqual(caught.exception.status_code, 302) + self.assertEqual(len(self._gateway_requests()), 1) + + def test_from_credentials_uses_default_timeout_for_non_positive_values(self) -> None: + zero_timeout_client = TdaiGatewayClient.from_credentials( + {"gateway_url": self.base_url, "gateway_timeout_seconds": "0"} + ) + negative_timeout_client = TdaiGatewayClient.from_credentials( + {"gateway_url": self.base_url, "gateway_timeout_seconds": "-1"} + ) + + self.assertEqual(zero_timeout_client.timeout, 10) + self.assertEqual(negative_timeout_client.timeout, 10) + + def test_from_credentials_accepts_missing_credentials(self) -> None: + client = TdaiGatewayClient.from_credentials(None) + + self.assertEqual(client.base_url, "http://127.0.0.1:8420") + self.assertEqual(client.api_key, "") + self.assertEqual(client.timeout, 10) + + def test_capture_rejects_empty_content_without_gateway_request(self) -> None: + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError): + client.capture("", "assistant reply", "dify-conv-1") + with self.assertRaises(TdaiGatewayError): + client.capture("user message", "", "dify-conv-1") + with self.assertRaises(TdaiGatewayError): + client.capture(" ", "assistant reply", "dify-conv-1") + with self.assertRaises(TdaiGatewayError): + client.capture("user message", " ", "dify-conv-1") + with self.assertRaises(TdaiGatewayError): + client.capture(None, "assistant reply", "dify-conv-1") # type: ignore[arg-type] + with self.assertRaises(TdaiGatewayError): + client.capture("user message", None, "dify-conv-1") # type: ignore[arg-type] + with self.assertRaises(TdaiGatewayError): + client.capture(42, "assistant reply", "dify-conv-1") # type: ignore[arg-type] + + self.assertEqual(self._gateway_requests(), []) + + def test_capture_posts_json_on_success(self) -> None: + self._set_gateway_response({"l0_recorded": 1, "scheduler_notified": True}) + + client = TdaiGatewayClient(self.base_url, timeout=2) + result = client.capture( + "remember this", + "stored", + "dify-conv-1", + session_id="dify-session-id", + user_id="user-1", + ) + + self.assertEqual(result["l0_recorded"], 1) + requests = self._gateway_requests() + self.assertEqual(len(requests), 1) + self.assertEqual(requests[0]["path"], "/capture") + self.assertEqual( + requests[0]["body"], + { + "user_content": "remember this", + "assistant_content": "stored", + "session_key": "dify-conv-1", + "session_id": "dify-session-id", + "user_id": "user-1", + }, + ) + + def test_search_and_session_methods_post_expected_payloads(self) -> None: + client = TdaiGatewayClient(self.base_url, timeout=2) + + self._set_gateway_response({"results": "memory", "total": 1, "strategy": "keyword"}) + memory_result = client.search_memories("remember", limit=99, type_filter="fact", scene="dev") + + self._set_gateway_response({"total": 3, "conversations": ["conv-1"]}) + conversation_result = client.search_conversations("thread", limit=0, session_key="dify-conv-1") + + self._set_gateway_response({"flushed": True}) + session_result = client.end_session("dify-conv-1", user_id="user-1") + + self.assertEqual(memory_result["results"], "memory") + self.assertEqual(conversation_result["total"], 3) + self.assertEqual(session_result["flushed"], True) + requests = self._gateway_requests() + self.assertEqual([request["path"] for request in requests], ["/search/memories", "/search/conversations", "/session/end"]) + self.assertEqual(requests[0]["body"], {"query": "remember", "limit": 50, "type": "fact", "scene": "dev"}) + self.assertEqual(requests[1]["body"], {"query": "thread", "limit": 1, "session_key": "dify-conv-1"}) + self.assertEqual(requests[2]["body"], {"session_key": "dify-conv-1", "user_id": "user-1"}) + + def test_session_key_validation_rejects_empty_values(self) -> None: + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError): + client.recall("query", "") + with self.assertRaises(TdaiGatewayError): + client.capture("user", "assistant", " ") + with self.assertRaises(TdaiGatewayError): + client.end_session("") + + self.assertEqual(self._gateway_requests(), []) + + def test_search_memories_rejects_invalid_limit(self) -> None: + client = TdaiGatewayClient(self.base_url, timeout=2) + + with self.assertRaises(TdaiGatewayError): + client.search_memories("query", limit="bad") + + def test_rejects_unsupported_gateway_url_scheme(self) -> None: + with self.assertRaises(TdaiGatewayError): + TdaiGatewayClient("file:///tmp/gateway.sock") + + def test_rejects_api_key_over_remote_plain_http(self) -> None: + with self.assertRaises(TdaiGatewayError): + TdaiGatewayClient("http://example.com:8420", api_key="secret-token") diff --git a/dify-plugin-tdai-memory/tests/test_main_entrypoint.py b/dify-plugin-tdai-memory/tests/test_main_entrypoint.py new file mode 100644 index 00000000..2b942456 --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_main_entrypoint.py @@ -0,0 +1,74 @@ +from __future__ import annotations + +import importlib +import os +import sys +import types +import unittest +from pathlib import Path +from typing import Any +from unittest.mock import patch + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +PLUGIN_ROOT_TEXT = str(PLUGIN_ROOT) +if PLUGIN_ROOT_TEXT not in sys.path: + sys.path.insert(0, PLUGIN_ROOT_TEXT) + + +class _StubDifyPluginEnv: + def __init__(self, **kwargs: Any) -> None: + self.kwargs = kwargs + + +class _StubPlugin: + def __init__(self, env: _StubDifyPluginEnv) -> None: + self.env = env + + def run(self) -> None: + return None + + +def _install_dify_entrypoint_stubs() -> None: + dify_plugin = sys.modules.get("dify_plugin") or types.ModuleType("dify_plugin") + if not hasattr(dify_plugin, "DifyPluginEnv"): + dify_plugin.DifyPluginEnv = _StubDifyPluginEnv + if not hasattr(dify_plugin, "Plugin"): + dify_plugin.Plugin = _StubPlugin + sys.modules["dify_plugin"] = dify_plugin + + +class MainEntrypointTest(unittest.TestCase): + def tearDown(self) -> None: + sys.modules.pop("main", None) + sys.modules.pop("dify_plugin", None) + + @patch.dict(os.environ, {}, clear=True) + def test_main_module_imports_plugin(self) -> None: + _install_dify_entrypoint_stubs() + + module = importlib.import_module("main") + + self.assertIsNotNone(module.plugin) + self.assertEqual(module.plugin.env.kwargs["MAX_REQUEST_TIMEOUT"], 120) + + @patch.dict(os.environ, {"MAX_REQUEST_TIMEOUT": "240"}, clear=False) + def test_main_module_respects_timeout_env_override(self) -> None: + _install_dify_entrypoint_stubs() + module = importlib.import_module("main") + module = importlib.reload(module) + self.assertEqual(module.plugin.env.kwargs["MAX_REQUEST_TIMEOUT"], 240) + + @patch.dict(os.environ, {"MAX_REQUEST_TIMEOUT": "0"}, clear=False) + def test_main_module_falls_back_on_non_positive_timeout(self) -> None: + _install_dify_entrypoint_stubs() + module = importlib.import_module("main") + module = importlib.reload(module) + self.assertEqual(module.plugin.env.kwargs["MAX_REQUEST_TIMEOUT"], 120) + + @patch.dict(os.environ, {"MAX_REQUEST_TIMEOUT": "invalid"}, clear=False) + def test_main_module_falls_back_on_invalid_timeout(self) -> None: + _install_dify_entrypoint_stubs() + module = importlib.import_module("main") + module = importlib.reload(module) + self.assertEqual(module.plugin.env.kwargs["MAX_REQUEST_TIMEOUT"], 120) diff --git a/dify-plugin-tdai-memory/tests/test_mock_dify_server.py b/dify-plugin-tdai-memory/tests/test_mock_dify_server.py new file mode 100644 index 00000000..10e85c19 --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_mock_dify_server.py @@ -0,0 +1,144 @@ +from __future__ import annotations + +import importlib.util +import json +import os +import tempfile +import threading +import unittest +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any +from unittest import mock + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] + + +class _GatewayHandler(BaseHTTPRequestHandler): + requests: list[dict[str, Any]] = [] + _requests_lock = threading.Lock() + + def do_POST(self) -> None: + try: + length = int(self.headers.get("Content-Length", "0")) + except ValueError: + self._send({"error": "invalid Content-Length"}, status=400) + return + raw = self.rfile.read(length).decode("utf-8") + try: + body = json.loads(raw) if raw else {} + except ValueError: + self._send({"error": "invalid JSON"}, status=400) + return + with self._requests_lock: + self.requests.append({"path": self.path, "body": body}) + + if self.path == "/capture": + self._send({"l0_recorded": 1, "scheduler_notified": True}) + return + if self.path == "/recall": + self._send( + { + "context": "remember Dify sessions", + "strategy": "hybrid", + "memory_count": 1, + "debug_secret": "internal-field", + } + ) + return + self._send({"error": "not found"}, status=404) + + def log_message(self, format: str, *args: Any) -> None: + return + + def _send(self, body: dict[str, Any], *, status: int = 200) -> None: + data = json.dumps(body).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +class MockDifyServerTest(unittest.TestCase): + def setUp(self) -> None: + with _GatewayHandler._requests_lock: + _GatewayHandler.requests = [] + self.server = ThreadingHTTPServer(("127.0.0.1", 0), _GatewayHandler) + self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) + self.thread.start() + host, port = self.server.server_address + self.gateway_url = f"http://{host}:{port}" + + def tearDown(self) -> None: + self.server.shutdown() + self.server.server_close() + self.thread.join(timeout=5) + + def _load_mock_server_module(self) -> Any: + module_path = PLUGIN_ROOT / "scripts" / "mock_dify_plugin_server.py" + self.assertTrue(module_path.is_file(), str(module_path)) + spec = importlib.util.spec_from_file_location("mock_dify_plugin_server", module_path) + self.assertIsNotNone(spec) + self.assertIsNotNone(spec.loader) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + def test_mock_server_invokes_capture_and_recall_tools_through_gateway(self) -> None: + module = self._load_mock_server_module() + + credentials = {"gateway_url": self.gateway_url, "gateway_timeout_seconds": 2} + capture = module.invoke_tool( + "tdai_capture", + { + "user_content": "Please remember Dify session wiring.", + "assistant_content": "I will store it through TencentDB Agent Memory.", + "session_key": "dify-e2e-session", + }, + credentials, + ) + recall = module.invoke_tool( + "tdai_recall", + {"query": "What should Dify remember?", "session_key": "dify-e2e-session"}, + credentials, + ) + + self.assertEqual(capture["ok"], True) + self.assertEqual(capture["l0_recorded"], 1) + self.assertEqual(recall["ok"], True) + self.assertIn("Dify sessions", recall["context"]) + self.assertNotIn("debug_secret", recall) + with _GatewayHandler._requests_lock: + request_paths = [request["path"] for request in _GatewayHandler.requests] + self.assertEqual(request_paths, ["/capture", "/recall"]) + + def test_mock_server_returns_tool_error_payload_as_successful_invocation(self) -> None: + module = self._load_mock_server_module() + + result = module.invoke_tool("tdai_capture", {}, {"gateway_url": ""}) + + self.assertEqual(result["ok"], False) + self.assertIn("gateway_url", result["error"]) + + def test_mock_server_reads_api_key_file_once_then_deletes_it(self) -> None: + module = self._load_mock_server_module() + with tempfile.NamedTemporaryFile("w", delete=False, encoding="utf-8") as key_file: + key_file.write("secret-token") + key_path = key_file.name + self.addCleanup(lambda: Path(key_path).unlink(missing_ok=True)) + + with mock.patch.dict(os.environ, {"TDAI_DIFY_GATEWAY_API_KEY_FILE": key_path}): + self.assertEqual(module._gateway_api_key(), "secret-token") + self.assertFalse(Path(key_path).exists()) + self.assertEqual(module._gateway_api_key(), "secret-token") + + def test_mock_server_reports_missing_api_key_file(self) -> None: + module = self._load_mock_server_module() + missing_path = str(Path(tempfile.gettempdir()) / "tdai-missing-key-file") + + with mock.patch.dict(os.environ, {"TDAI_DIFY_GATEWAY_API_KEY_FILE": missing_path}): + with self.assertRaises(ValueError) as caught: + module._gateway_api_key() + self.assertIn("cannot read gateway API key file", str(caught.exception)) diff --git a/dify-plugin-tdai-memory/tests/test_plugin_manifest.py b/dify-plugin-tdai-memory/tests/test_plugin_manifest.py new file mode 100644 index 00000000..e3a944fc --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_plugin_manifest.py @@ -0,0 +1,141 @@ +from __future__ import annotations + +import re +import unittest +from pathlib import Path + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +REPO_ROOT = PLUGIN_ROOT.parent +MAX_LLM_DESCRIPTION_LENGTH = 30 + + +def _inline_yaml_values(text: str, key: str) -> list[str]: + # These tests intentionally use a tiny parser to avoid adding PyYAML just + # for controlled Dify YAML fixtures; block scalars are rejected explicitly. + values: list[str] = [] + pattern = re.compile(rf"^\s*{re.escape(key)}:\s*(.+?)\s*$") + for line in text.splitlines(): + match = pattern.match(line) + if not match: + continue + value = match.group(1).strip() + if value in {"|", ">", "|-", ">-", "|+", ">+"}: + raise AssertionError(f"{key} must stay inline for prompt-budget tests") + if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: + value = value[1:-1] + values.append(value) + return values + + +def _parameter_blocks(text: str) -> dict[str, str]: + # This covers the simple `parameters: - name: ...` shape used by Dify tools. + blocks: dict[str, str] = {} + for block in re.split(r"\n\s*-\s+name:\s+", f"\n{text}")[1:]: + name, _, rest = block.partition("\n") + blocks[name.strip()] = rest + return blocks + + +class PluginManifestTest(unittest.TestCase): + def test_manifest_references_provider_yaml(self) -> None: + manifest = (PLUGIN_ROOT / "manifest.yaml").read_text(encoding="utf-8") + + self.assertIn("provider/tdai_memory.yaml", manifest) + self.assertTrue((PLUGIN_ROOT / "provider" / "tdai_memory.yaml").is_file()) + + def test_provider_and_tool_yaml_sources_exist(self) -> None: + provider_yaml = (PLUGIN_ROOT / "provider" / "tdai_memory.yaml").read_text(encoding="utf-8") + self.assertIn("source: provider/tdai_memory.py", provider_yaml) + self.assertTrue((PLUGIN_ROOT / "provider" / "tdai_memory.py").is_file()) + + tool_paths = re.findall(r"-\s+(tools/[-_a-zA-Z0-9.]+\.yaml)", provider_yaml) + expected_tool_paths = {f"tools/{path.name}" for path in (PLUGIN_ROOT / "tools").glob("*.yaml")} + self.assertEqual(set(tool_paths), expected_tool_paths) + for relative_tool_path in tool_paths: + tool_yaml_path = PLUGIN_ROOT / relative_tool_path + self.assertTrue(tool_yaml_path.is_file(), relative_tool_path) + tool_yaml = tool_yaml_path.read_text(encoding="utf-8") + source_match = re.search(r"source:\s+(tools/[-_a-zA-Z0-9.]+\.py)", tool_yaml) + self.assertIsNotNone(source_match, relative_tool_path) + self.assertTrue((PLUGIN_ROOT / source_match.group(1)).is_file(), source_match.group(1)) + + def test_manifest_and_tools_reference_existing_icon_asset(self) -> None: + icon = "icon.svg" + self.assertTrue((PLUGIN_ROOT / "_assets" / icon).is_file()) + + manifest = (PLUGIN_ROOT / "manifest.yaml").read_text(encoding="utf-8") + provider_yaml = (PLUGIN_ROOT / "provider" / "tdai_memory.yaml").read_text(encoding="utf-8") + + self.assertIn(f"icon: {icon}", manifest) + self.assertIn(f"icon: {icon}", provider_yaml) + self.assertNotIn("icon: _assets/", manifest) + self.assertNotIn("icon: _assets/", provider_yaml) + + def test_quickstart_and_architecture_docs_are_present(self) -> None: + quickstart = PLUGIN_ROOT / "scripts" / "quickstart-gateway-mock-e2e.sh" + mock_server = PLUGIN_ROOT / "scripts" / "mock_dify_plugin_server.py" + architecture = PLUGIN_ROOT / "ARCHITECTURE.md" + install_guide = REPO_ROOT / "docs" / "dify-plugin-installation-guide.md" + workflow = REPO_ROOT / "docs" / "dify-workflow-diagram.md" + comparison = REPO_ROOT / "docs" / "cross-platform-comparison.md" + + for path in [quickstart, mock_server, architecture, install_guide, workflow, comparison]: + self.assertTrue(path.is_file(), str(path)) + + quickstart_text = quickstart.read_text(encoding="utf-8") + self.assertIn("src/gateway/server.ts", quickstart_text) + self.assertIn("mock_dify_plugin_server.py", quickstart_text) + self.assertIn("tdai_capture", quickstart_text) + self.assertIn("tdai_recall", quickstart_text) + self.assertIn("json.dumps", quickstart_text) + self.assertIn("find_curl", quickstart_text) + self.assertIn("trap cleanup EXIT INT TERM", quickstart_text) + self.assertIn("Failed to invoke tdai_capture", quickstart_text) + self.assertIn("Failed to invoke tdai_recall", quickstart_text) + self.assertNotIn("CAPTURE_BODY=$(cat < None: + for tool_yaml_path in (PLUGIN_ROOT / "tools").glob("*.yaml"): + text = tool_yaml_path.read_text(encoding="utf-8") + for description in _inline_yaml_values(text, "llm_description"): + self.assertLessEqual( + len(description.strip()), + MAX_LLM_DESCRIPTION_LENGTH, + f"{tool_yaml_path.name}: {description}", + ) + + def test_max_chars_zero_semantics_are_documented(self) -> None: + for tool_yaml_path in (PLUGIN_ROOT / "tools").glob("*.yaml"): + text = tool_yaml_path.read_text(encoding="utf-8") + max_chars_block = _parameter_blocks(text).get("max_chars") + if max_chars_block is not None: + self.assertIn("0 means unlimited", max_chars_block, tool_yaml_path.name) + + def test_dify_dependency_is_pinned(self) -> None: + pyproject = (PLUGIN_ROOT / "pyproject.toml").read_text(encoding="utf-8") + + self.assertIn('"dify_plugin~=0.9.0"', pyproject) + self.assertIn("[build-system]", pyproject) + self.assertIn('description = "Connect Dify workflows', pyproject) + self.assertIn('readme = "README.md"', pyproject) + self.assertIn('license = { text = "MIT" }', pyproject) diff --git a/dify-plugin-tdai-memory/tests/test_provider_credentials.py b/dify-plugin-tdai-memory/tests/test_provider_credentials.py new file mode 100644 index 00000000..8e33badd --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_provider_credentials.py @@ -0,0 +1,143 @@ +from __future__ import annotations + +import json +import sys +import threading +import types +import unittest +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +PLUGIN_ROOT_TEXT = str(PLUGIN_ROOT) +if PLUGIN_ROOT_TEXT not in sys.path: + sys.path.insert(0, PLUGIN_ROOT_TEXT) + + +class _StubToolProvider: + pass + + +class _StubToolProviderCredentialValidationError(Exception): + pass + + +def _install_dify_stubs() -> None: + dify_plugin = sys.modules.get("dify_plugin") or types.ModuleType("dify_plugin") + if not hasattr(dify_plugin, "ToolProvider"): + dify_plugin.ToolProvider = _StubToolProvider + + errors = sys.modules.get("dify_plugin.errors") or types.ModuleType("dify_plugin.errors") + error_tool_module = sys.modules.get("dify_plugin.errors.tool") or types.ModuleType("dify_plugin.errors.tool") + if not hasattr(error_tool_module, "ToolProviderCredentialValidationError"): + error_tool_module.ToolProviderCredentialValidationError = _StubToolProviderCredentialValidationError + + sys.modules["dify_plugin"] = dify_plugin + sys.modules["dify_plugin.errors"] = errors + sys.modules["dify_plugin.errors.tool"] = error_tool_module + + +_install_dify_stubs() + +from dify_plugin.errors.tool import ToolProviderCredentialValidationError # noqa: E402 +from provider.tdai_memory import TdaiMemoryProvider # noqa: E402 + + +class _ValidationGatewayHandler(BaseHTTPRequestHandler): + response_status = 200 + response_body: dict[str, Any] = {"results": "", "total": 0, "strategy": "keyword"} + requests: list[dict[str, Any]] = [] + _state_lock = threading.Lock() + + def do_POST(self) -> None: + length = int(self.headers.get("Content-Length", "0")) + raw = self.rfile.read(length).decode("utf-8") + body = json.loads(raw) if raw else {} + with self._state_lock: + self.requests.append({"path": self.path, "body": body}) + response_status = self.response_status + response_body = self.response_body + + if self.path != "/search/memories": + self._send({"error": "not found"}, status=404) + return + self._send(response_body, status=response_status) + + def log_message(self, format: str, *args: Any) -> None: + return + + def _send(self, body: dict[str, Any], *, status: int = 200) -> None: + data = json.dumps(body).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +class ProviderCredentialsTest(unittest.TestCase): + @classmethod + def tearDownClass(cls) -> None: + sys.modules.pop("dify_plugin", None) + sys.modules.pop("dify_plugin.errors", None) + sys.modules.pop("dify_plugin.errors.tool", None) + + def setUp(self) -> None: + with _ValidationGatewayHandler._state_lock: + _ValidationGatewayHandler.response_status = 200 + _ValidationGatewayHandler.response_body = {"results": "", "total": 0, "strategy": "keyword"} + _ValidationGatewayHandler.requests = [] + self.server = ThreadingHTTPServer(("127.0.0.1", 0), _ValidationGatewayHandler) + self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) + self.thread.start() + host, port = self.server.server_address + self.gateway_url = f"http://{host}:{port}" + + def tearDown(self) -> None: + self.server.shutdown() + self.server.server_close() + self.thread.join(timeout=5) + + def _requests(self) -> list[dict[str, Any]]: + with _ValidationGatewayHandler._state_lock: + return list(_ValidationGatewayHandler.requests) + + def test_validate_credentials_requires_gateway_url(self) -> None: + provider = TdaiMemoryProvider() + + with self.assertRaises(ToolProviderCredentialValidationError): + provider._validate_credentials({}) + + def test_validate_credentials_uses_read_only_search_handshake(self) -> None: + provider = TdaiMemoryProvider() + + provider._validate_credentials({"gateway_url": self.gateway_url, "gateway_timeout_seconds": 2}) + + requests = self._requests() + self.assertEqual(len(requests), 1) + self.assertEqual(requests[0]["path"], "/search/memories") + self.assertEqual( + requests[0]["body"], + {"query": "__dify_credential_validation__", "limit": 1}, + ) + + def test_validate_credentials_rejects_gateway_http_error(self) -> None: + with _ValidationGatewayHandler._state_lock: + _ValidationGatewayHandler.response_status = 401 + _ValidationGatewayHandler.response_body = {"error": "Unauthorized", "code": "UNAUTHORIZED"} + provider = TdaiMemoryProvider() + + with self.assertRaises(ToolProviderCredentialValidationError) as caught: + provider._validate_credentials({"gateway_url": self.gateway_url, "gateway_timeout_seconds": 2}) + self.assertEqual(str(caught.exception), "Gateway credential validation failed (HTTP 401)") + self.assertNotIn("Unauthorized", str(caught.exception)) + + def test_validate_credentials_rejects_api_key_over_remote_http(self) -> None: + provider = TdaiMemoryProvider() + + with self.assertRaises(ToolProviderCredentialValidationError): + provider._validate_credentials( + {"gateway_url": "http://example.invalid:8420", "gateway_api_key": "secret-token"} + ) diff --git a/dify-plugin-tdai-memory/tests/test_tool_helpers.py b/dify-plugin-tdai-memory/tests/test_tool_helpers.py new file mode 100644 index 00000000..8129157d --- /dev/null +++ b/dify-plugin-tdai-memory/tests/test_tool_helpers.py @@ -0,0 +1,134 @@ +from __future__ import annotations + +import sys +import unittest +from pathlib import Path + + +PLUGIN_ROOT = Path(__file__).resolve().parents[1] +PLUGIN_ROOT_TEXT = str(PLUGIN_ROOT) +if PLUGIN_ROOT_TEXT not in sys.path: + sys.path.insert(0, PLUGIN_ROOT_TEXT) + +from tools.base import ( # noqa: E402 + DEFAULT_MAX_CHARS, + MAX_CHARS_LIMIT, + TEXT_TRUNCATED_MARKER, + TdaiToolMixin, + build_error_payload, + normalize_limit, + truncate_text, +) +from tools.client import TdaiGatewayError # noqa: E402 + + +class _StubRuntime: + def __init__(self, credentials: dict[str, object] | None = None) -> None: + self.credentials = credentials or {} + + +class ToolHelpersTest(unittest.TestCase): + def test_truncate_text_keeps_short_text_unchanged(self) -> None: + self.assertEqual(truncate_text("short memory", 50), "short memory") + self.assertEqual(truncate_text("", 10), "") + + def test_truncate_text_marks_long_text(self) -> None: + self.assertEqual(truncate_text("abcdef", 4), f"abcd{TEXT_TRUNCATED_MARKER}") + + def test_truncate_text_exact_boundary_is_not_truncated(self) -> None: + self.assertEqual(truncate_text("abcd", 4), "abcd") + + def test_truncate_text_defaults_max_chars(self) -> None: + short_text = "x" * 1500 + long_text = "x" * 2500 + + self.assertEqual(truncate_text(short_text, None), short_text) + self.assertEqual(truncate_text(long_text, None), "x" * DEFAULT_MAX_CHARS + TEXT_TRUNCATED_MARKER) + + def test_truncate_text_non_positive_limit_returns_full_text(self) -> None: + self.assertEqual(truncate_text("hello", 0), "hello") + self.assertEqual(truncate_text("hello", -1), "hello") + + def test_normalize_limit_bounds_values(self) -> None: + self.assertEqual(normalize_limit(None), 5) + self.assertEqual(normalize_limit(-5), 1) + self.assertEqual(normalize_limit(0), 1) + self.assertEqual(normalize_limit("0"), 1) + self.assertEqual(normalize_limit("200"), 50) + self.assertEqual(normalize_limit("bad"), 5) + + def test_normalize_limit_preserves_values_within_bounds(self) -> None: + self.assertEqual(normalize_limit("10"), 10) + self.assertEqual(normalize_limit(25), 25) + + def test_build_error_payload_keeps_tool_result_non_throwing(self) -> None: + payload = build_error_payload("recall", RuntimeError("gateway down secret=abc")) + + self.assertEqual(payload["ok"], False) + self.assertEqual(payload["operation"], "recall") + self.assertEqual(payload["error"], "recall failed: RuntimeError") + self.assertEqual(payload["error_type"], "RuntimeError") + self.assertNotIn("secret=abc", payload["error"]) + + def test_build_error_payload_uses_exception_class_for_empty_message(self) -> None: + payload = build_error_payload("recall", RuntimeError()) + + self.assertEqual(payload["error"], "recall failed: RuntimeError") + + def test_build_error_payload_formats_gateway_error_with_status(self) -> None: + error = TdaiGatewayError("Unauthorized secret=abc", status_code=401, code="UNAUTHORIZED") + + payload = build_error_payload("recall", error) + + self.assertEqual(payload["error"], "recall failed: Gateway returned HTTP 401") + self.assertEqual(payload["error_type"], "TdaiGatewayError") + self.assertEqual(payload["status_code"], 401) + self.assertEqual(payload["code"], "UNAUTHORIZED") + self.assertNotIn("secret=abc", payload["error"]) + + def test_build_error_payload_formats_gateway_error_without_status(self) -> None: + payload = build_error_payload("recall", TdaiGatewayError("connection refused")) + + self.assertEqual(payload["error"], "recall failed: Gateway request failed") + self.assertEqual(payload["error_type"], "TdaiGatewayError") + self.assertNotIn("status_code", payload) + self.assertNotIn("code", payload) + + def test_tool_mixin_text_helper_normalizes_values(self) -> None: + self.assertEqual(TdaiToolMixin._text({"query": " hello "}, "query"), "hello") + self.assertEqual(TdaiToolMixin._text({"query": None}, "query", "fallback"), "fallback") + self.assertEqual(TdaiToolMixin._text({}, "query", "fallback"), "fallback") + self.assertEqual(TdaiToolMixin._text({"query": " "}, "query"), "") + self.assertEqual(TdaiToolMixin._text({"query": 42}, "query"), "42") + + def test_tool_mixin_max_chars_bounds_values(self) -> None: + self.assertEqual(TdaiToolMixin._max_chars({}), DEFAULT_MAX_CHARS) + self.assertEqual(TdaiToolMixin._max_chars({"max_chars": 0}), 0) + self.assertEqual(TdaiToolMixin._max_chars({"max_chars": "500"}), 500) + self.assertEqual(TdaiToolMixin._max_chars({"max_chars": "bad"}), DEFAULT_MAX_CHARS) + self.assertEqual(TdaiToolMixin._max_chars({"max_chars": 30_000}), MAX_CHARS_LIMIT) + + def test_tool_mixin_client_requires_runtime_credentials(self) -> None: + mixin = TdaiToolMixin() + with self.assertRaises(TdaiGatewayError): + mixin._client() + + mixin.runtime = _StubRuntime() + with self.assertRaises(TdaiGatewayError): + mixin._client() + + def test_tool_mixin_client_uses_runtime_credentials(self) -> None: + mixin = TdaiToolMixin() + mixin.runtime = _StubRuntime( + { + "gateway_url": "http://127.0.0.1:8420", + "gateway_api_key": "test-key", + "gateway_timeout_seconds": 5, + } + ) + + client = mixin._client() + + self.assertEqual(client.base_url, "http://127.0.0.1:8420") + self.assertEqual(client.api_key, "test-key") + self.assertEqual(client.timeout, 5) diff --git a/dify-plugin-tdai-memory/tools/__init__.py b/dify-plugin-tdai-memory/tools/__init__.py new file mode 100644 index 00000000..134318a5 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/__init__.py @@ -0,0 +1 @@ +"""TencentDB Agent Memory Dify tools.""" diff --git a/dify-plugin-tdai-memory/tools/base.py b/dify-plugin-tdai-memory/tools/base.py new file mode 100644 index 00000000..ace0340b --- /dev/null +++ b/dify-plugin-tdai-memory/tools/base.py @@ -0,0 +1,116 @@ +"""Shared helpers for TencentDB Agent Memory Dify tools. + +The Dify Tool classes should never leak Gateway exceptions into normal agent +flows. They convert failures into structured JSON payloads so a Dify workflow +can continue without memory instead of aborting the whole conversation. + +Design: +- Keep validation and truncation local to the Dify adapter layer. +- Reuse `TdaiGatewayClient` for every HTTP call so auth/header behavior stays + consistent across tools. + +Usage: + text = truncate_text(result["results"], 2000) + yield self.create_json_message({"ok": True, "results": text}) +""" + +from __future__ import annotations + +from typing import Any + +from tools.client import TdaiGatewayClient, TdaiGatewayError + + +DEFAULT_SEARCH_LIMIT = 5 +MAX_SEARCH_LIMIT = 50 +DEFAULT_MAX_CHARS = 2000 +MAX_CHARS_LIMIT = 20_000 +MAX_ERROR_CHARS = 500 +TEXT_TRUNCATED_MARKER = "\n\n[truncated]" +ERROR_TRUNCATED_MARKER = "\n[truncated]" + + +def truncate_text(text: str, max_chars: int | None = None) -> str: + """Trim long Gateway text fields. + + The `[truncated]` marker is appended after the content limit so callers can + distinguish a real suffix from adapter truncation. + """ + try: + limit = int(max_chars) if max_chars is not None else DEFAULT_MAX_CHARS + except (TypeError, ValueError): + limit = DEFAULT_MAX_CHARS + if limit <= 0 or len(text) <= limit: + return text + return f"{text[:limit]}{TEXT_TRUNCATED_MARKER}" + + +def normalize_limit( + value: Any, + *, + default: int = DEFAULT_SEARCH_LIMIT, + minimum: int = 1, + maximum: int = MAX_SEARCH_LIMIT, +) -> int: + """Clamp user supplied search limits to protect the Gateway and prompt.""" + try: + parsed = int(value) + except (TypeError, ValueError): + parsed = default + return max(minimum, min(maximum, parsed)) + + +def build_error_payload(operation: str, error: Exception) -> dict[str, Any]: + """Return a non-throwing tool payload for Gateway failures. + + Gateway details are reduced to status/code fields so Dify users do not see + raw backend messages or credentials embedded in exception strings. + """ + if isinstance(error, TdaiGatewayError): + if error.status_code is not None: + message = f"{operation} failed: Gateway returned HTTP {error.status_code}" + else: + message = f"{operation} failed: Gateway request failed" + else: + message = f"{operation} failed: {error.__class__.__name__}" + if len(message) > MAX_ERROR_CHARS: + message = f"{message[:MAX_ERROR_CHARS]}{ERROR_TRUNCATED_MARKER}" + payload: dict[str, Any] = { + "ok": False, + "operation": operation, + "error": message, + "error_type": error.__class__.__name__, + } + if isinstance(error, TdaiGatewayError): + if error.status_code is not None: + payload["status_code"] = error.status_code + if error.code: + payload["code"] = error.code + return payload + + +class TdaiToolMixin: + """Common Dify Tool helpers backed by provider credentials.""" + + def _client(self) -> TdaiGatewayClient: + runtime = getattr(self, "runtime", None) + if runtime is None: + raise TdaiGatewayError("Tool runtime is not initialized; provider credentials unavailable") + credentials = getattr(runtime, "credentials", {}) or {} + if not credentials: + raise TdaiGatewayError("Provider credentials are empty; configure Gateway settings") + return TdaiGatewayClient.from_credentials(credentials) + + @staticmethod + def _text(params: dict[str, Any], name: str, default: str = "") -> str: + value = params.get(name, default) + return str(value).strip() if value is not None else default + + @staticmethod + def _max_chars(params: dict[str, Any]) -> int: + return normalize_limit( + params.get("max_chars"), + default=DEFAULT_MAX_CHARS, + minimum=0, + maximum=MAX_CHARS_LIMIT, + ) diff --git a/dify-plugin-tdai-memory/tools/client.py b/dify-plugin-tdai-memory/tools/client.py new file mode 100644 index 00000000..78abbaef --- /dev/null +++ b/dify-plugin-tdai-memory/tools/client.py @@ -0,0 +1,288 @@ +"""TdaiGatewayClient - HTTP client for TencentDB Agent Memory Gateway. + +This module is the protocol boundary for the Dify plugin. It intentionally +uses only Python's standard library so tests can run without the Dify runtime +or additional HTTP dependencies. + +Design: +- Keep Gateway request/response field names identical to `src/gateway/types.ts`. +- Raise structured errors in the client; Dify tool classes decide how to + degrade without blocking an agent workflow. + +Usage: + client = TdaiGatewayClient("http://127.0.0.1:8420", api_key="...") + context = client.recall("query", "conversation-id") +""" + +from __future__ import annotations + +import json +import ssl +import urllib.error +import urllib.request +from ipaddress import ip_address +from typing import Any +from urllib.parse import urlparse + + +DEFAULT_GATEWAY_URL = "http://127.0.0.1:8420" +DEFAULT_TIMEOUT_SECONDS = 10 +MAX_ERROR_BODY_BYTES = 65_536 + + +class _NoRedirectHandler(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req: Any, fp: Any, code: int, msg: str, headers: Any, newurl: str) -> None: + raise urllib.error.HTTPError(req.full_url, code, msg, headers, fp) + + +class TdaiGatewayError(RuntimeError): + """Gateway error with HTTP status and optional gateway error code. + + `response` stores the raw, untrusted Gateway response for internal + debugging and must not be surfaced to agent users without sanitization. + """ + + def __init__( + self, + message: str, + *, + status_code: int | None = None, + code: str | None = None, + response: Any | None = None, + ) -> None: + super().__init__(message) + self.status_code = status_code + self.code = code + self.response = response + + +class TdaiGatewayClient: + """Small HTTP wrapper around the TDAI Gateway API.""" + + def __init__( + self, + base_url: str = DEFAULT_GATEWAY_URL, + *, + api_key: str | None = None, + timeout: int | float = DEFAULT_TIMEOUT_SECONDS, + ) -> None: + self.base_url = (base_url or DEFAULT_GATEWAY_URL).strip().rstrip("/") + parsed = urlparse(self.base_url) + if parsed.scheme not in ("http", "https") or not parsed.netloc: + raise TdaiGatewayError(f"Unsupported Gateway URL: {self.base_url}") + self.api_key = (api_key or "").strip() + self.timeout = _normalize_timeout(timeout) + self._ssl_context = ssl.create_default_context() + if self.api_key and parsed.scheme == "http" and not _is_loopback(parsed.hostname): + raise TdaiGatewayError("Gateway API key requires HTTPS for non-local Gateway URLs") + + @classmethod + def from_credentials(cls, credentials: dict[str, Any] | None = None) -> "TdaiGatewayClient": + """Create a client from Dify provider credentials.""" + credentials = credentials or {} + base_url = str(credentials.get("gateway_url") or DEFAULT_GATEWAY_URL).strip() + return cls( + base_url, + api_key=str(credentials.get("gateway_api_key") or ""), + timeout=_normalize_timeout(credentials.get("gateway_timeout_seconds")), + ) + + def health(self) -> dict[str, Any]: + """Call `GET /health`.""" + return self._request("GET", "/health") + + def recall(self, query: str, session_key: str, *, user_id: str = "") -> dict[str, Any]: + """Call `POST /recall`.""" + _require_non_empty_string(session_key, "session_key") + body = {"query": query, "session_key": session_key} + if user_id: + body["user_id"] = user_id + return self._request("POST", "/recall", body) + + def capture( + self, + user_content: str, + assistant_content: str, + session_key: str, + *, + session_id: str = "", + user_id: str = "", + ) -> dict[str, Any]: + """Call `POST /capture`.""" + _require_non_empty_string(user_content, "user_content") + _require_non_empty_string(assistant_content, "assistant_content") + _require_non_empty_string(session_key, "session_key") + body = { + "user_content": user_content, + "assistant_content": assistant_content, + "session_key": session_key, + } + if session_id: + body["session_id"] = session_id + if user_id: + body["user_id"] = user_id + return self._request("POST", "/capture", body) + + def search_memories( + self, + query: str, + *, + limit: int = 5, + type_filter: str = "", + scene: str = "", + ) -> dict[str, Any]: + """Call `POST /search/memories`.""" + body: dict[str, Any] = {"query": query, "limit": _normalize_gateway_limit(limit)} + if type_filter: + # Field name is fixed by the Gateway API contract. + body["type"] = type_filter + if scene: + body["scene"] = scene + return self._request("POST", "/search/memories", body) + + def search_conversations( + self, + query: str, + *, + limit: int = 5, + session_key: str = "", + ) -> dict[str, Any]: + """Call `POST /search/conversations`.""" + body: dict[str, Any] = {"query": query, "limit": _normalize_gateway_limit(limit)} + if session_key: + body["session_key"] = session_key + return self._request("POST", "/search/conversations", body) + + def end_session(self, session_key: str, *, user_id: str = "") -> dict[str, Any]: + """Call `POST /session/end`.""" + _require_non_empty_string(session_key, "session_key") + body = {"session_key": session_key} + if user_id: + body["user_id"] = user_id + return self._request("POST", "/session/end", body) + + def _request(self, method: str, path: str, body: dict[str, Any] | None = None) -> dict[str, Any]: + if not path.startswith("/"): + raise TdaiGatewayError(f"path must start with '/', got: {path!r}") + data = None + headers: dict[str, str] = {} + if body is not None: + data = json.dumps(body).encode("utf-8") + headers["Content-Type"] = "application/json" + if self.api_key: + headers["Authorization"] = f"Bearer {self.api_key}" + + request = urllib.request.Request( + f"{self.base_url}{path}", + data=data, + headers=headers, + method=method, + ) + try: + opener = urllib.request.build_opener( + _NoRedirectHandler(), + urllib.request.HTTPSHandler(context=self._ssl_context), + ) + with opener.open(request, timeout=self.timeout) as response: + raw = response.read().decode("utf-8") + content_type = response.headers.get("Content-Type", "") + status_code = response.status + if content_type and "json" not in content_type.lower(): + raise TdaiGatewayError( + f"Unexpected Content-Type: {content_type or '(none)'}", + status_code=status_code, + response=raw, + ) + if not raw: + return {} + try: + parsed = json.loads(raw) + except json.JSONDecodeError as exc: + raise TdaiGatewayError( + f"Invalid JSON response: {exc}", + status_code=status_code, + response=raw, + ) from exc + if not isinstance(parsed, dict): + raise TdaiGatewayError( + f"Expected JSON object response from Gateway (HTTP {status_code})", + status_code=status_code, + response=parsed, + ) + return parsed + except urllib.error.HTTPError as exc: + detail = self._read_error_body(exc) + message = self._error_message(detail) or exc.reason or f"HTTP {exc.code}" + raise TdaiGatewayError( + str(message), + status_code=exc.code, + code=detail.get("code") if isinstance(detail, dict) else None, + response=detail, + ) from exc + except TdaiGatewayError: + raise + except (urllib.error.URLError, OSError, TimeoutError) as exc: + raise TdaiGatewayError(f"Gateway request failed: {exc}") from exc + + @staticmethod + def _read_error_body(exc: urllib.error.HTTPError) -> Any: + try: + raw = exc.read(MAX_ERROR_BODY_BYTES).decode("utf-8", errors="replace") + except Exception: + return None + if not raw: + return None + try: + return json.loads(raw) + except json.JSONDecodeError: + return raw + + @staticmethod + def _error_message(detail: Any) -> str: + if isinstance(detail, dict): + for key in ("error", "message", "detail", "description"): + if detail.get(key): + return _safe_error_snippet(str(detail[key])) + return "" + return _safe_error_snippet(str(detail or "")) + + +def _is_loopback(host: str | None) -> bool: + if not host: + return False + if host.lower() == "localhost": + return True + try: + return ip_address(host).is_loopback + except ValueError: + return False + + +def _normalize_timeout(value: int | float | str | None) -> float: + try: + timeout = float(value) if value not in (None, "") else DEFAULT_TIMEOUT_SECONDS + except (TypeError, ValueError): + return float(DEFAULT_TIMEOUT_SECONDS) + if timeout <= 0: + return float(DEFAULT_TIMEOUT_SECONDS) + return timeout + + +def _safe_error_snippet(message: str) -> str: + return message.splitlines()[0][:200] + + +def _require_non_empty_string(value: Any, name: str) -> None: + if not isinstance(value, str): + raise TdaiGatewayError(f"{name} must be a string, got {type(value).__name__}") + stripped = value.strip() + if not stripped: + raise TdaiGatewayError(f"{name} must not be empty (value={value!r})") + + +def _normalize_gateway_limit(value: Any) -> int: + try: + parsed = int(value) + except (TypeError, ValueError) as exc: + raise TdaiGatewayError("limit must be an integer") from exc + return max(1, min(50, parsed)) diff --git a/dify-plugin-tdai-memory/tools/tdai_capture.py b/dify-plugin-tdai-memory/tools/tdai_capture.py new file mode 100644 index 00000000..b53d85e1 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_capture.py @@ -0,0 +1,30 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload + + +class TdaiCaptureTool(TdaiToolMixin, Tool): + """Capture a completed Dify conversation turn into TDAI memory.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().capture( + self._text(tool_parameters, "user_content"), + self._text(tool_parameters, "assistant_content"), + self._text(tool_parameters, "session_key"), + session_id=self._text(tool_parameters, "session_id"), + user_id=self._text(tool_parameters, "user_id"), + ) + payload = {"ok": True} + for key in ("l0_recorded", "scheduler_notified"): + if key in result: + payload[key] = result[key] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("capture", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_capture.yaml b/dify-plugin-tdai-memory/tools/tdai_capture.yaml new file mode 100644 index 00000000..86e6e296 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_capture.yaml @@ -0,0 +1,70 @@ +identity: + name: tdai_capture + author: tencentdb-agent-memory + label: + en_US: Capture Conversation Turn + zh_Hans: 捕获对话轮次 +description: + human: + en_US: Store a completed user/assistant turn in TencentDB Agent Memory. + zh_Hans: 将完成的一轮用户/助手对话写入 TencentDB Agent Memory。 + llm: Store a completed turn. +parameters: + - name: user_content + type: string + required: true + label: + en_US: User Content + zh_Hans: 用户内容 + human_description: + en_US: The user's original message. + zh_Hans: 用户原始消息。 + llm_description: User message. + form: llm + - name: assistant_content + type: string + required: true + label: + en_US: Assistant Content + zh_Hans: 助手内容 + human_description: + en_US: The assistant response. + zh_Hans: 助手回复。 + llm_description: Assistant reply. + form: llm + - name: session_key + type: string + required: true + label: + en_US: Session Key + zh_Hans: 会话 Key + human_description: + en_US: Stable Dify conversation ID. + zh_Hans: 稳定的 Dify conversation_id。 + llm_description: Stable session key. + form: llm + - name: session_id + type: string + required: false + label: + en_US: Session ID + zh_Hans: 会话 ID + human_description: + en_US: Optional sub-session identifier. + zh_Hans: 可选的子会话标识。 + llm_description: Optional sub-session ID. + form: llm + - name: user_id + type: string + required: false + label: + en_US: User ID + zh_Hans: 用户 ID + human_description: + en_US: Optional end-user identifier. + zh_Hans: 可选的终端用户标识。 + llm_description: Optional user ID. + form: llm +extra: + python: + source: tools/tdai_capture.py diff --git a/dify-plugin-tdai-memory/tools/tdai_conversation_search.py b/dify-plugin-tdai-memory/tools/tdai_conversation_search.py new file mode 100644 index 00000000..e349d944 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_conversation_search.py @@ -0,0 +1,33 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload, normalize_limit, truncate_text + + +class TdaiConversationSearchTool(TdaiToolMixin, Tool): + """Search L0 raw conversations through the Gateway.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().search_conversations( + self._text(tool_parameters, "query"), + limit=normalize_limit(tool_parameters.get("limit")), + session_key=self._text(tool_parameters, "session_key"), + ) + raw_results = result.get("results") + results = truncate_text( + str(raw_results) if raw_results is not None else "", + self._max_chars(tool_parameters), + ) + payload: dict[str, Any] = {"ok": True, "results": results} + for key in ("total", "strategy"): + if key in result: + payload[key] = result[key] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("conversation_search", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_conversation_search.yaml b/dify-plugin-tdai-memory/tools/tdai_conversation_search.yaml new file mode 100644 index 00000000..92b6f1ee --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_conversation_search.yaml @@ -0,0 +1,65 @@ +identity: + name: tdai_conversation_search + author: tencentdb-agent-memory + label: + en_US: Search Raw Conversations + zh_Hans: 搜索原始对话 +description: + human: + en_US: Search L0 raw conversations. + zh_Hans: 搜索 L0 原始对话。 + llm: Search raw conversations. +parameters: + - name: query + type: string + required: true + label: + en_US: Query + zh_Hans: 查询内容 + human_description: + en_US: Conversation search query. + zh_Hans: 对话搜索查询。 + llm_description: Conversation query. + form: llm + - name: limit + type: number + required: false + min: 1 + max: 50 + default: 5 + label: + en_US: Limit + zh_Hans: 数量限制 + human_description: + en_US: Maximum number of results. Default 5, max 50. + zh_Hans: 最大返回数量。默认 5,最大 50。 + llm_description: Result count. + form: llm + - name: session_key + type: string + required: false + label: + en_US: Session Key + zh_Hans: 会话 Key + human_description: + en_US: Optional stable conversation ID filter. + zh_Hans: 可选稳定会话 ID 过滤。 + llm_description: Optional session filter. + form: llm + - name: max_chars + type: number + required: false + min: 0 + max: 20000 + default: 2000 + label: + en_US: Max Characters + zh_Hans: 最大字符数 + human_description: + en_US: Maximum result characters. 0 means unlimited. Default 2000. + zh_Hans: 返回结果的最大字符数,默认 2000。 + llm_description: Result char cap. + form: llm +extra: + python: + source: tools/tdai_conversation_search.py diff --git a/dify-plugin-tdai-memory/tools/tdai_health.py b/dify-plugin-tdai-memory/tools/tdai_health.py new file mode 100644 index 00000000..d4951d35 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_health.py @@ -0,0 +1,24 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload + + +class TdaiHealthTool(TdaiToolMixin, Tool): + """Check whether the TencentDB Agent Memory Gateway is reachable.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().health() + payload = {"ok": True} + for key in ("status", "version", "uptime", "stores"): + if key in result: + payload[key] = result[key] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("health", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_health.yaml b/dify-plugin-tdai-memory/tools/tdai_health.yaml new file mode 100644 index 00000000..0dbaf1f2 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_health.yaml @@ -0,0 +1,15 @@ +identity: + name: tdai_health + author: tencentdb-agent-memory + label: + en_US: Check Gateway Health + zh_Hans: 检查 Gateway 健康状态 +description: + human: + en_US: Check whether TencentDB Agent Memory Gateway is reachable. + zh_Hans: 检查 TencentDB Agent Memory Gateway 是否可访问。 + llm: Check the local TencentDB Agent Memory Gateway health endpoint. +parameters: [] +extra: + python: + source: tools/tdai_health.py diff --git a/dify-plugin-tdai-memory/tools/tdai_memory_search.py b/dify-plugin-tdai-memory/tools/tdai_memory_search.py new file mode 100644 index 00000000..034d9979 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_memory_search.py @@ -0,0 +1,34 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload, normalize_limit, truncate_text + + +class TdaiMemorySearchTool(TdaiToolMixin, Tool): + """Search L1 structured memories through the Gateway.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().search_memories( + self._text(tool_parameters, "query"), + limit=normalize_limit(tool_parameters.get("limit")), + type_filter=self._text(tool_parameters, "type"), + scene=self._text(tool_parameters, "scene"), + ) + raw_results = result.get("results") + results = truncate_text( + str(raw_results) if raw_results is not None else "", + self._max_chars(tool_parameters), + ) + payload: dict[str, Any] = {"ok": True, "results": results} + for key in ("total", "strategy"): + if key in result: + payload[key] = result[key] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("memory_search", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_memory_search.yaml b/dify-plugin-tdai-memory/tools/tdai_memory_search.yaml new file mode 100644 index 00000000..0b42e7ef --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_memory_search.yaml @@ -0,0 +1,76 @@ +identity: + name: tdai_memory_search + author: tencentdb-agent-memory + label: + en_US: Search Structured Memories + zh_Hans: 搜索结构化记忆 +description: + human: + en_US: Search L1 structured memories. + zh_Hans: 搜索 L1 结构化记忆。 + llm: Search structured memory. +parameters: + - name: query + type: string + required: true + label: + en_US: Query + zh_Hans: 查询内容 + human_description: + en_US: Memory search query. + zh_Hans: 记忆搜索查询。 + llm_description: L1 memory query. + form: llm + - name: limit + type: number + required: false + min: 1 + max: 50 + default: 5 + label: + en_US: Limit + zh_Hans: 数量限制 + human_description: + en_US: Maximum number of results. Default 5, max 50. + zh_Hans: 最大返回数量。默认 5,最大 50。 + llm_description: L1 result count. + form: llm + - name: type + type: string + required: false + label: + en_US: Memory Type + zh_Hans: 记忆类型 + human_description: + en_US: Optional L1 memory type filter. + zh_Hans: 可选的 L1 记忆类型过滤。 + llm_description: Memory type filter. + form: llm + - name: scene + type: string + required: false + label: + en_US: Scene + zh_Hans: 场景 + human_description: + en_US: Optional scene filter. + zh_Hans: 可选场景过滤。 + llm_description: Scene filter. + form: llm + - name: max_chars + type: number + required: false + min: 0 + max: 20000 + default: 2000 + label: + en_US: Max Characters + zh_Hans: 最大字符数 + human_description: + en_US: Maximum result characters. 0 means unlimited. Default 2000. + zh_Hans: 返回结果的最大字符数,默认 2000。 + llm_description: Result char cap. + form: llm +extra: + python: + source: tools/tdai_memory_search.py diff --git a/dify-plugin-tdai-memory/tools/tdai_recall.py b/dify-plugin-tdai-memory/tools/tdai_recall.py new file mode 100644 index 00000000..504bc101 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_recall.py @@ -0,0 +1,34 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload, truncate_text + + +class TdaiRecallTool(TdaiToolMixin, Tool): + """Recall memory context before a Dify LLM node runs.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().recall( + self._text(tool_parameters, "query"), + self._text(tool_parameters, "session_key"), + user_id=self._text(tool_parameters, "user_id"), + ) + raw_context = result.get("context") + context = truncate_text( + str(raw_context) if raw_context is not None else "", + self._max_chars(tool_parameters), + ) + payload: dict[str, Any] = {"ok": True, "context": context} + if "strategy" in result: + payload["strategy"] = result["strategy"] + if "memory_count" in result: + payload["memory_count"] = result["memory_count"] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("recall", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_recall.yaml b/dify-plugin-tdai-memory/tools/tdai_recall.yaml new file mode 100644 index 00000000..74721fa0 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_recall.yaml @@ -0,0 +1,62 @@ +identity: + name: tdai_recall + author: tencentdb-agent-memory + label: + en_US: Recall Memory Context + zh_Hans: 召回记忆上下文 +description: + human: + en_US: Recall relevant memory context before an LLM node runs. + zh_Hans: 在 LLM 节点运行前召回相关记忆上下文。 + llm: Recall memory for this turn. +parameters: + - name: query + type: string + required: true + label: + en_US: Query + zh_Hans: 查询内容 + human_description: + en_US: Current user message or search query. + zh_Hans: 当前用户消息或搜索查询。 + llm_description: Message to recall from. + form: llm + - name: session_key + type: string + required: true + label: + en_US: Session Key + zh_Hans: 会话 Key + human_description: + en_US: Stable Dify conversation ID. + zh_Hans: 稳定的 Dify conversation_id。 + llm_description: Stable conversation ID. + form: llm + - name: user_id + type: string + required: false + label: + en_US: User ID + zh_Hans: 用户 ID + human_description: + en_US: Optional end-user identifier. + zh_Hans: 可选的终端用户标识。 + llm_description: Optional user ID. + form: llm + - name: max_chars + type: number + required: false + min: 0 + max: 20000 + default: 2000 + label: + en_US: Max Characters + zh_Hans: 最大字符数 + human_description: + en_US: Maximum context characters. 0 means unlimited. Default 2000. + zh_Hans: 返回上下文的最大字符数,默认 2000。 + llm_description: Context char cap. + form: llm +extra: + python: + source: tools/tdai_recall.py diff --git a/dify-plugin-tdai-memory/tools/tdai_session_end.py b/dify-plugin-tdai-memory/tools/tdai_session_end.py new file mode 100644 index 00000000..8ea4c45d --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_session_end.py @@ -0,0 +1,26 @@ +from __future__ import annotations + +from collections.abc import Generator +from typing import Any + +from dify_plugin import Tool +from dify_plugin.entities.tool import ToolInvokeMessage + +from tools.base import TdaiToolMixin, build_error_payload + + +class TdaiSessionEndTool(TdaiToolMixin, Tool): + """Flush a Dify conversation session without stopping the Gateway.""" + + def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage, None, None]: + try: + result = self._client().end_session( + self._text(tool_parameters, "session_key"), + user_id=self._text(tool_parameters, "user_id"), + ) + payload = {"ok": True} + if "flushed" in result: + payload["flushed"] = result["flushed"] + yield self.create_json_message(payload) + except Exception as exc: + yield self.create_json_message(build_error_payload("session_end", exc)) diff --git a/dify-plugin-tdai-memory/tools/tdai_session_end.yaml b/dify-plugin-tdai-memory/tools/tdai_session_end.yaml new file mode 100644 index 00000000..b5e253c3 --- /dev/null +++ b/dify-plugin-tdai-memory/tools/tdai_session_end.yaml @@ -0,0 +1,37 @@ +identity: + name: tdai_session_end + author: tencentdb-agent-memory + label: + en_US: End Memory Session + zh_Hans: 结束记忆会话 +description: + human: + en_US: Flush the current session buffer without stopping the Gateway. + zh_Hans: 刷新当前会话缓冲,不会停止 Gateway。 + llm: End a memory session and flush buffered pipeline work. +parameters: + - name: session_key + type: string + required: true + label: + en_US: Session Key + zh_Hans: 会话 Key + human_description: + en_US: Stable Dify conversation ID. + zh_Hans: 稳定的 Dify conversation_id。 + llm_description: Session ID to flush. + form: llm + - name: user_id + type: string + required: false + label: + en_US: User ID + zh_Hans: 用户 ID + human_description: + en_US: Optional end-user identifier. + zh_Hans: 可选的终端用户标识。 + llm_description: Optional user ID. + form: llm +extra: + python: + source: tools/tdai_session_end.py diff --git a/docs/cross-platform-comparison.md b/docs/cross-platform-comparison.md new file mode 100644 index 00000000..8da4ec5d --- /dev/null +++ b/docs/cross-platform-comparison.md @@ -0,0 +1,25 @@ +# Cross-Platform Adapter Comparison + +This comparison explains the platform-specific adapter work for OpenClaw, +Hermes, and Dify. The target architecture keeps memory behavior in `TdaiCore` +and limits each adapter to host integration concerns. + +The core differences are session management, LLM invocation, and tool registration. + +| Dimension | OpenClaw | Hermes | Dify | +| --- | --- | --- | --- | +| Adapter shape | Native OpenClaw plugin hooks in `index.ts` and `src/adapters/openclaw`. | Python provider plus Gateway sidecar under `hermes-plugin/memory/memory_tencentdb`. | Dify tool plugin under `dify-plugin-tdai-memory`. | +| Session management | Uses OpenClaw session context and host lifecycle hooks. | Uses Hermes conversation/session identifiers and forwards them to Gateway routes. | Uses Dify `conversation_id` as `session_key`; workflow authors must pass it to recall, capture, and session end. | +| LLM invocation | Can use OpenClaw host LLM APIs through the OpenClaw adapter. | Uses standalone Gateway/Hermes path with OpenAI-compatible configuration from environment or Gateway config. | Does not call an LLM. Dify owns the LLM node; the plugin only returns recalled context. | +| Tool registration | Registered through `openclaw.plugin.json` and OpenClaw extension loading. | Registered as a Hermes memory provider module. | Registered through Dify `manifest.yaml`, provider YAML, and tool YAML files. | +| Recall injection | Hook injects memory before prompt construction. | Provider/Gateway path supplies recall context to Hermes. | Workflow explicitly calls `tdai_recall` before the LLM node and injects returned `context`. | +| Capture timing | Hook captures completed turns from OpenClaw runtime events. | Provider sends completed conversation turns to Gateway. | Workflow explicitly calls `tdai_capture` after the assistant response exists. | +| Gateway lifecycle | OpenClaw can manage plugin/gateway lifecycle through host integration. | Hermes can auto-start or connect to a sidecar Gateway. | Dify plugin does not start Gateway; operator starts Gateway separately or uses the quickstart mock harness. | +| Auth | Host/plugin configuration plus optional Gateway Bearer auth. | `MEMORY_TENCENTDB_GATEWAY_API_KEY` or `TDAI_GATEWAY_API_KEY` for client-side Bearer auth. | Provider credential `gateway_api_key`; remote plain HTTP with API key is rejected by the adapter. | +| Failure mode | Host hook errors should degrade without breaking the conversation. | Provider degrades when Gateway is unavailable. | Tool calls return `{ "ok": false, ... }` so Dify workflows can continue without memory. | + +## Key Difference + +OpenClaw and Hermes can integrate at host lifecycle points. Dify workflows are +graph-based, so the user must place recall and capture tools in the graph. This +is why the Dify adapter favors explicit tools rather than hidden hooks. diff --git a/docs/dify-plugin-installation-guide.md b/docs/dify-plugin-installation-guide.md new file mode 100644 index 00000000..92ba657d --- /dev/null +++ b/docs/dify-plugin-installation-guide.md @@ -0,0 +1,338 @@ +# Dify Plugin Installation and Workflow Guide + +This guide shows how to package, install, configure, and verify the TencentDB +Agent Memory Dify tool plugin. + +The Dify adapter is intentionally thin: it does not reimplement the memory +engine. It forwards Dify tool calls to the TencentDB Agent Memory Gateway, +which owns recall, capture, search, and session-level memory processing. + +## Scope + +This guide covers: + +- Packaging the Dify tool plugin into a `.difypkg` file. +- Installing the package into a Dify workspace. +- Configuring a Gateway endpoint reachable from the Dify plugin runtime. +- Wiring recall and capture tools into a Dify workflow. +- Verifying basic memory write and read behavior. + +## Data Flow + +```mermaid +flowchart LR + User["User message"] --> Dify["Dify workflow"] + Dify --> Recall["tdai_recall"] + Recall --> Gateway["TencentDB Agent Memory Gateway"] + Gateway --> Core["TdaiCore"] + Core --> Gateway + Gateway --> Recall + Recall --> LLM["LLM node"] + LLM --> Assistant["Assistant response"] + Assistant --> Capture["tdai_capture"] + Capture --> Gateway + Gateway --> Core +``` + +## Prerequisites + +Prepare: + +1. A running Dify instance with plugin support enabled. +2. The Dify plugin CLI. +3. Python 3.12 for the plugin runtime. +4. A running TencentDB Agent Memory Gateway. +5. A Gateway URL reachable from the Dify plugin runtime. + +Gateway URL examples: + +```text +# Dify runs in Docker and Gateway runs on the host: +http://host.docker.internal:8420 + +# Gateway is exposed on the LAN: +http://192.168.x.x:8420 + +# Gateway is exposed through HTTPS: +https://memory-gateway.example.com +``` + +Do not assume that `127.0.0.1` inside the Dify plugin runtime points to the +host machine. In Docker deployments, it usually points to the container itself. + +## Package the Plugin + +From the repository root: + +```bash +dify plugin package ./dify-plugin-tdai-memory +``` + +Expected output: + +```text +plugin packaged successfully +``` + +If packaging fails, check: + +- `manifest.yaml` syntax. +- Provider YAML path. +- Tool YAML paths. +- Python source paths. +- Icon path. Dify plugin YAML should reference `icon.svg`; the file lives + under `_assets/icon.svg` in the plugin package layout. +- Files excluded by `.difyignore`. + +## Install in Dify + +Open Dify and use: + +```text +Workspace -> Plugins -> Install Plugin -> Local Package +``` + +Upload the generated `.difypkg`. After installation, the plugin should appear +as: + +```text +TencentDB Agent Memory +``` + +For local unsigned packages, Dify plugin daemon may require local development +configuration: + +```text +FORCE_VERIFYING_SIGNATURE=false +``` + +Keep signature verification enabled for production plugin distribution unless +the package is signed and trusted by the target environment. + +## Configure Provider Credentials + +Configure the installed provider: + +| Field | Example | Notes | +| --- | --- | --- | +| Gateway URL | `http://host.docker.internal:8420` | Endpoint reachable from the Dify plugin runtime. | +| Gateway API Key | optional | Must match `TDAI_GATEWAY_API_KEY` when Gateway auth is enabled. | +| Timeout Seconds | `10` | Request timeout for plugin-to-Gateway calls. | + +Do not expose real API keys in screenshots, logs, or committed configuration. + +## Tool Mapping + +| Tool | Gateway endpoint | Typical workflow position | +| --- | --- | --- | +| `tdai_health` | `GET /health` | Setup validation | +| `tdai_recall` | `POST /recall` | Before the LLM node | +| `tdai_capture` | `POST /capture` | After the LLM node | +| `tdai_memory_search` | `POST /search/memories` | Debug/admin workflow | +| `tdai_conversation_search` | `POST /search/conversations` | Debug/admin workflow | +| `tdai_session_end` | `POST /session/end` | End of workflow/session | + +Use Dify `conversation_id` as `session_key`. A workflow run id changes on every +execution, but `conversation_id` stays stable across turns in the same +conversation. + +## Minimal Workflow + +Recommended node order: + +```text +Start -> tdai_recall -> LLM -> tdai_capture -> End +``` + +Start inputs: + +```text +user_id: string +conversation_id: string +query: string +``` + +Recall tool input: + +```json +{ + "user_id": "{{start.user_id}}", + "session_key": "{{start.conversation_id}}", + "query": "{{start.query}}", + "max_chars": 2000 +} +``` + +LLM prompt example: + +```text +Relevant memory: +{{tdai_recall.context}} + +User question: +{{start.query}} +``` + +Capture tool input: + +```json +{ + "user_id": "{{start.user_id}}", + "session_key": "{{start.conversation_id}}", + "user_content": "{{start.query}}", + "assistant_content": "{{llm.text}}" +} +``` + +Optional session flush: + +```json +{ + "user_id": "{{start.user_id}}", + "session_key": "{{start.conversation_id}}" +} +``` + +## Verification Scenario + +Two-turn check: + +1. Capture memory: + + ```text + user_id = demo-user + conversation_id = demo-conversation + query = My preferred coding language is Go, and I usually work on Kubernetes projects. + ``` + + Expected: + + ```text + tdai_capture returns ok=true. + Gateway stores the raw conversation turn. + ``` + +2. Recall or search memory: + + ```text + user_id = demo-user + conversation_id = demo-conversation + query = What programming language do I usually prefer? + ``` + + Expected: + + ```text + tdai_recall calls the Gateway successfully. + tdai_conversation_search can read the captured raw conversation immediately. + Structured L1 recall depends on the Gateway/Core consolidation pipeline. + ``` + +## Local Validation Log + +Environment: + +```text +Dify: langgenius/dify-api:1.15.0 +Dify plugin daemon: langgenius/dify-plugin-daemon:0.6.3-local +Gateway URL used by Dify plugin runtime: http://host.docker.internal:8420 +Gateway health from Dify API container: status=ok, vectorStore=true, embeddingService=false +``` + +Quickstart e2e: + +```text +[1/4] Starting or reusing TencentDB Agent Memory Gateway at http://172.23.224.1:8420 +Gateway already healthy +[2/4] Starting mock Dify plugin server at http://127.0.0.1:18420 +[3/4] Capturing a completed Dify turn through tdai_capture +{"l0_recorded": 2, "ok": true, "scheduler_notified": true} +[4/4] Recalling memory through tdai_recall +{"context": "", "memory_count": 0, "ok": true} +Quickstart e2e succeeded: Gateway -> mock Dify server -> capture -> recall +``` + +Dify package/install: + +```text +$ dify plugin package ./dify-plugin-tdai-memory +2026/07/04 16:54:02 INFO plugin packaged successfully output_path=/tmp/tdai_memory_0.0.1.difypkg +package size: 19227 bytes + +upload unique_identifier: +tencentdb-agent-memory/tdai_memory:0.0.1@83c5476576a25586fc2223f0cd9cadf7331cfc3b52719b983039528a9ebab872 + +install task: +status=success +message=installed +plugin_id=tencentdb-agent-memory/tdai_memory +``` + +Dify plugin invocation through `PluginToolManager`: + +```text +provider=tencentdb-agent-memory/tdai_memory/tdai_memory +tools=tdai_health, tdai_recall, tdai_capture, tdai_memory_search, tdai_conversation_search, tdai_session_end + +tdai_health: +{"ok": true, "status": "ok", "stores": {"embeddingService": false, "vectorStore": true}, "version": "0.1.0"} + +tdai_capture: +{"l0_recorded": 2, "ok": true, "scheduler_notified": true} + +tdai_conversation_search: +{"ok": true, "total": 2, "results": "Found 2 matching message(s) ... TDAI_DIFY_E2E_TOKEN_20260704_SKYBLUE ..."} + +tdai_recall: +{"context": "", "memory_count": 0, "ok": true} +``` + +The invocation path above is: + +```text +Dify API -> plugin_daemon -> tdai_memory Python plugin -> TencentDB Agent Memory Gateway +``` + +## Troubleshooting + +### Plugin cannot connect to Gateway + +Check the Gateway URL from inside the Dify plugin runtime. For Docker-based +Dify, prefer: + +```text +http://host.docker.internal:8420 +``` + +### Provider credentials fail to save + +Check: + +- Gateway URL format. +- Timeout value. +- Gateway reachability. +- Whether Gateway auth is enabled and the API key is correct. + +### Recall returns empty memory + +Check: + +- Whether `tdai_capture` ran first. +- Whether `session_key` is stable. +- Whether the Gateway uses the expected data directory. +- Whether the L1/L2/L3 consolidation pipeline has run. + +Use `tdai_conversation_search` for immediate L0 read-after-write validation. + +### Capture succeeds but later recall/search misses data + +Avoid using a transient workflow run id as `session_key`. Prefer Dify +`conversation_id`. + +## Security Notes + +Private or local Gateway URLs are allowed because self-hosted memory +infrastructure is expected. + +Do not send Bearer credentials over non-local plain HTTP. Use HTTPS for remote +Gateway deployments. diff --git a/docs/dify-workflow-diagram.md b/docs/dify-workflow-diagram.md new file mode 100644 index 00000000..ec3a8100 --- /dev/null +++ b/docs/dify-workflow-diagram.md @@ -0,0 +1,39 @@ +# Dify Workflow Diagram + +This diagram covers the recommended Dify workflow for TencentDB Agent Memory. +The adapter keeps Dify-specific wiring at the edge and reuses the existing +Gateway and `TdaiCore` pipeline. + +```mermaid +flowchart TD + User["End user"] --> DifyStart["Dify workflow start"] + DifyStart --> RecallTool["tdai_recall tool"] + RecallTool --> ClientRecall["TdaiGatewayClient"] + ClientRecall --> GatewayRecall["Gateway POST /recall"] + GatewayRecall --> CoreRecall["TdaiCore handleBeforeRecall"] + CoreRecall --> StoresRead["Memory stores and embedding search"] + StoresRead --> CoreRecall + CoreRecall --> GatewayRecall + GatewayRecall --> RecallTool + RecallTool --> Prompt["Inject returned context into LLM prompt"] + Prompt --> LLM["Dify LLM node"] + LLM --> CaptureTool["tdai_capture tool"] + CaptureTool --> ClientCapture["TdaiGatewayClient"] + ClientCapture --> GatewayCapture["Gateway POST /capture"] + GatewayCapture --> CoreCapture["TdaiCore handleTurnCommitted"] + CoreCapture --> L0["L0 raw conversation"] + CoreCapture --> Scheduler["Progressive memory scheduler"] + Scheduler --> L1L3["L1/L2/L3 memories"] + LLM --> User + DifyStart --> EndTool["tdai_session_end at workflow end"] + EndTool --> GatewayEnd["Gateway POST /session/end"] +``` + +Recommended identifiers: + +| Dify value | TDAI parameter | Reason | +| --- | --- | --- | +| `conversation_id` | `session_key` | Stable across turns in the same Dify conversation. | +| End-user id | `user_id` | Optional metadata; current Gateway/Core behavior still relies primarily on `session_key`. | +| Current user message | `query` for recall | Drives memory retrieval before generation. | +| User plus assistant turn | `capture` body | Records completed conversation turns after generation. |