Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/ISSUE_TEMPLATE/bug-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ Use the component checklist below and labels instead.

Where is the issue happening?

- [ ] Indexer (`packages/indexer`)
- [ ] MCP server (`packages/mcp`)
- [ ] Python backend/API (`apps/api`)
- [ ] MCP service (`apps/api`)
- [ ] Indexer CLI (`apps/api`)
- [ ] Obsidian plugin (`packages/obsidian-plugin`)
- [ ] Core/shared (`packages/core`)
- [ ] Docs
Expand Down
9 changes: 5 additions & 4 deletions .github/ISSUE_TEMPLATE/bug-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ body:
label: Component
description: Where is the issue happening?
options:
- Indexer (packages/indexer)
- MCP server (packages/mcp)
- Python backend/API (apps/api)
- MCP service (apps/api)
- Indexer CLI (apps/api)
- Obsidian plugin (packages/obsidian-plugin)
- Core/shared (packages/core)
- Docs
Expand All @@ -31,7 +32,7 @@ body:
attributes:
label: What happened?
description: What did you do, what happened, and what did you expect?
placeholder: "Indexer crashes with ... / MCP tool returns ..."
placeholder: "Python backend returns 503 with ... / MCP service startup fails with ..."
validations:
required: true

Expand All @@ -42,7 +43,7 @@ body:
description: Minimal steps so we can reproduce
placeholder: |
1. Set OPENAI_EMBEDDING_MODEL=...
2. Run pnpm -C packages/indexer start -- ...
2. Run uv run --directory apps/api ailss-indexer --vault "$AILSS_VAULT_PATH"
3. Observe ...
validations:
required: true
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/refactor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ body:
attributes:
label: Validation
description: Which checks/tests will confirm behavior is preserved?
placeholder: "- pnpm --filter @ailss/core test\n- pnpm --filter @ailss/mcp test"
placeholder: "- pnpm --filter @ailss/core test\n- pnpm py:test"
validations:
required: true

Expand Down
5 changes: 3 additions & 2 deletions .github/ISSUE_TEMPLATE/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ Use labels for component tagging instead.

Which area should be covered?

- [ ] Indexer (`packages/indexer`)
- [ ] MCP server (`packages/mcp`)
- [ ] Python backend/API (`apps/api`)
- [ ] MCP service (`apps/api`)
- [ ] Indexer CLI (`apps/api`)
- [ ] Obsidian plugin (`packages/obsidian-plugin`)
- [ ] Core/shared (`packages/core`)
- [ ] Docs
Expand Down
5 changes: 3 additions & 2 deletions .github/ISSUE_TEMPLATE/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ body:
label: Component
description: Which area should be covered?
options:
- Indexer (packages/indexer)
- MCP server (packages/mcp)
- Python backend/API (apps/api)
- MCP service (apps/api)
- Indexer CLI (apps/api)
- Obsidian plugin (packages/obsidian-plugin)
- Core/shared (packages/core)
- Docs
Expand Down
13 changes: 11 additions & 2 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,21 @@ plugin:
mcp:
- changed-files:
- any-glob-to-any-file:
- packages/mcp/**
- apps/api/src/ailss_api/mcp*.py
- apps/api/tests/test_mcp_runtime.py
- docs/reference/mcp-tools.md
- packages/obsidian-plugin/src/mcp/**
- packages/obsidian-plugin/src/settingsSections/mcpServiceSection.ts
- packages/obsidian-plugin/test/mcp/**

indexer:
- changed-files:
- any-glob-to-any-file:
- packages/indexer/**
- apps/api/src/ailss_api/indexer*.py
- packages/obsidian-plugin/src/indexer/**
- packages/obsidian-plugin/src/settingsSections/indexMaintenanceSection.ts
- packages/obsidian-plugin/src/ui/indexDbResetFlow.ts
- packages/obsidian-plugin/src/ui/indexer*.ts

core:
- changed-files:
Expand Down
6 changes: 4 additions & 2 deletions apps/api/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# AILSS API

Local FastAPI app for the Python-first AILSS backend baseline.
Python service app for the AILSS local runtime.

## Commands

```bash
uv sync --directory apps/api
uv sync --directory apps/api --locked
uv run --directory apps/api ailss-api --host 127.0.0.1 --port 8000
uv run --directory apps/api ailss-mcp-http
uv run --directory apps/api ailss-indexer --help
uv run --directory apps/api pytest
```

Expand Down
2 changes: 1 addition & 1 deletion apps/api/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "ailss-api"
version = "0.1.0"
description = "Python-first local agent backend baseline for AILSS"
description = "Python service app for the AILSS local runtime"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/ailss_api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def create_app(settings: Settings | None = None) -> FastAPI:
app = FastAPI(
title="AILSS API",
version="0.1.0-dev",
summary="Python-first local agent backend baseline",
summary="Python-owned local service runtime",
)

@app.get("/health", response_model=HealthResponse)
Expand Down
37 changes: 20 additions & 17 deletions docs/01-overview.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
# System overview

This document describes the current AILSS runtime and the transition baseline established by
issue #175.
This document describes the current AILSS runtime established through issues #175, #179,
and #181.

Today, the implemented system is split into **three parts**. The next baseline keeps those
parts in place while adding a Python-first local backend surface for retrieval, agent
orchestration, evaluation, and lightweight observability.
Today, the implemented system is split into **three current runtime parts** backed by the
Python service app in `apps/api`: the indexer, the localhost MCP service, and the
Obsidian plugin shell. `packages/core` remains a shared TypeScript utility package, not a
launched runtime service.

## 1) Indexer

Expand Down Expand Up @@ -71,7 +72,8 @@ Transport / client integration:
- Drop cached session ID
- Send a fresh `initialize` request to obtain a new `Mcp-Session-Id`
- Retry the original request once with the new session ID
- Local dev still supports running the MCP server over stdio (CLI).
- Local dev can run the Python MCP service directly from the CLI
(`uv run --directory apps/api ailss-mcp-http`).
- Optional shutdown endpoint (disabled by default):
- If `AILSS_MCP_HTTP_SHUTDOWN_TOKEN` is set (or `startAilssMcpHttpServer({ shutdown: { token } })` is used), the server exposes `POST /__ailss/shutdown`.
- This endpoint requires the shutdown token (separate from the normal MCP request token) and shuts down the HTTP server + all MCP sessions.
Expand Down Expand Up @@ -110,11 +112,11 @@ Write tools are gated and not exposed by default:
Responsibilities:

- Display recommendations in a UI.
- Center the local runtime around the Python backend for health, retrieval, agent, eval,
and status flows.
- Center the local runtime around the Python backend, Python MCP service, and Python
indexer in `apps/api`.
- Keep the local index DB up to date (manual reindex and optional debounced auto-index).
- Treat the MCP transport and existing indexer as transition components while the current
migration remains in progress.
- Keep explicit launch overrides available for users who need custom `uv run --directory
... apps/api` paths.
- Only perform vault writes when explicitly requested (for example an MCP write tool call with `apply=true`).
- Applying changes can be implemented either via the Obsidian Vault API or via direct filesystem writes (but must remain gated and auditable).

Expand All @@ -124,19 +126,20 @@ Responsibilities:
- Recommendation = DB read
- Apply = file write; requires an explicit action (Obsidian UI or MCP write tool with `apply=true`, including `capture_note`/`edit_note`).

## Transition baseline (issue #175)
## Current runtime baseline

- Obsidian plugin remains the local UX shell and launcher.
- Existing Node/TypeScript packages remain the transition baseline for indexing, MCP
transport, and gated vault writes.
- The Python-first backend surface now owns the local API contracts for retrieval, agent
- `apps/api` owns the shipped local runtime for indexing, MCP transport, retrieval, agent
orchestration, evaluation, and lightweight observability.
- Plugin startup, status, and troubleshooting flows should center on that Python backend
surface while the Node-owned components remain explicit transition layers.
- `packages/core` remains the shared TypeScript utility/schema package, not an
independently launched service.
- Plugin startup, status, and troubleshooting flows center on the Python-owned runtime.
- The current backend contract is `GET /health`, `POST /retrieve`, `POST /agent/run`, and
`POST /eval/run`, with plugin-managed lifecycle plus a guarded shutdown path for stale
local processes.
- Local-first, single-user scope remains the project boundary for this phase.

See `docs/architecture/python-first-local-agent-backend.md` for the service boundaries,
contract details, and acceptance criteria.
contract details, and acceptance criteria. See
`docs/architecture/legacy-node-typescript-runtime-removal.md` for the retirement record of
the removed Node runtime path.
56 changes: 30 additions & 26 deletions docs/03-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
This document lists an implementation sequence that starts small and then expands.
It also records a few **hard decisions** so code and docs stay consistent.

> Status note (2026-03-22): the migration to a Python-owned local runtime is complete.
> `apps/api` now owns the indexer CLI, localhost MCP service, retrieval, agent flow, and
> eval path. Historical sections below preserve the staged implementation record and may
> mention retired Node runtime paths; use `Current status`, `docs/01-overview.md`, and
> `docs/architecture/python-first-local-agent-backend.md` for the live architecture.

## 0) Confirm assumptions / decisions

- Primary goal: **JD-aligned portfolio strengthening** for backend/agent roles (closest to the Sendbird/Socra profile), not a cloud SaaS launch.
Expand All @@ -16,48 +22,46 @@ It also records a few **hard decisions** so code and docs stay consistent.
- Codex can trigger explicit write tools over MCP (no per-edit UI flow) once enabled.
- Connection is configured globally once (e.g. `~/.codex/config.toml`) via a URL + token.
- Vault path: the vault is **external** and provided via configuration (e.g., `AILSS_VAULT_PATH`).
- Architecture direction for the next phase:
- Architecture direction for the current phase:
- Keep the Obsidian plugin as the local UX shell / launcher.
- Keep the current Node/TypeScript packages as the working baseline during transition.
- Add a **Python-first backend surface** for retrieval, agent orchestration, and evaluation.
- `apps/api` owns the local runtime for indexing, MCP, retrieval, agent orchestration,
and evaluation.
- `packages/core` remains the shared TypeScript utility/schema package.
- Non-goals for this phase:
- multi-user SaaS / remote hosting
- heavy cloud-first infra for its own sake
- Samsung-specific manufacturing, optimization, or embedded scope

## Current status

- Indexer MVP exists (`packages/indexer`)
- Supports full-vault indexing and path-scoped updates (`--paths`)
- Supports explicit DB reset (`--reset-db`) when switching embedding models
- Validates DB embedding identity (model/dimension) and fails fast on mismatch
- Full-vault runs prune DB entries for deleted files
- Has a deterministic wrapper test (stubbed embeddings; no network)
- MCP server MVP exists (`packages/mcp`)
- Read-first tools: `get_context`, `expand_typed_links_outgoing`, `find_typed_links_incoming`, `resolve_note`, `read_note`, `get_vault_tree`, `frontmatter_validate`, `find_broken_links`, `search_notes`, `list_tags`, `list_keywords`, `list_typed_link_rels`, `get_tool_failure_report`
- Explicit write tools (gated; `AILSS_ENABLE_WRITE_TOOLS=1`): `capture_note`, `canonicalize_typed_links`, `edit_note`, `improve_frontmatter`, `relocate_note`
- Transport: stdio + streamable HTTP (`/mcp` on localhost; supports multiple concurrent sessions)
- Python runtime now owns the shipped local services in `apps/api`
- `ailss-indexer` supports full-vault indexing, path-scoped updates (`--paths`), and
explicit DB reset (`--reset-db`) when switching embedding models
- `ailss-mcp-http` is the active localhost MCP surface
- Read-first tools: `get_context`, `expand_typed_links_outgoing`, `find_typed_links_incoming`, `resolve_note`, `read_note`, `get_vault_tree`, `frontmatter_validate`, `find_broken_links`, `search_notes`, `list_tags`, `list_keywords`, `list_typed_link_rels`, `get_tool_failure_report`
- Explicit write tools (gated; `AILSS_ENABLE_WRITE_TOOLS=1`): `capture_note`, `canonicalize_typed_links`, `edit_note`, `improve_frontmatter`, `relocate_note`
- Obsidian plugin MVP exists (`packages/obsidian-plugin`)
- UI: status modals for indexing and the localhost MCP service
- Indexing: `AILSS: Reindex vault` command + optional auto-index on file changes (debounced; spawns the indexer process)
- Indexing: `AILSS: Reindex vault` command + optional auto-index on file changes (debounced; spawns the Python indexer process)
- MCP service: optional localhost MCP server for Codex (URL + token; can expose gated write tools)
- Supports multiple concurrent MCP sessions (multiple Codex processes)
- Python backend baseline exists (`apps/api`)
- Python backend exists in `apps/api`
- FastAPI app with `GET /health`, `POST /retrieve`, `POST /agent/run`, and `POST /eval/run`
- Semantic retrieval over the existing SQLite + `sqlite-vec` index, plus explicit lexical baseline mode
- LangGraph workflow for `retrieve -> decide -> read -> answer -> validate`
- Eval dataset runner + local run/eval artifacts
- Obsidian plugin now manages the local Python backend
- Lifecycle: start, stop, restart, readiness wait, guarded shutdown reclaim
- Commands: backend health, retrieval, grounded agent run, eval run
- Shared TypeScript utilities remain in `packages/core`
- Gap relative to the portfolio goal:
- strong local MCP + retrieval + Obsidian integration already exist
- Python backend baseline, explicit workflow orchestration, and reproducible eval now exist
- remaining gaps are full LLM reasoning quality, deeper lifecycle recovery, richer eval/cost trending, and transition cleanup
- transition cleanup is now split explicitly across:
- issue #182 parity verification for the current MCP tool surface
- issue #180 plugin runtime re-centering around the Python backend
- issue #181 staged removal planning for the legacy Node/TypeScript path
- Python runtime, explicit workflow orchestration, and reproducible eval now exist
- remaining gaps are full LLM reasoning quality, deeper lifecycle recovery, and richer eval/cost trending
- legacy runtime cleanup completed through issues #179 / #181; historical notes remain below for traceability

Historical note:

- Sections below preserve the staged implementation record. When they mention retired Node
packages or transition-only runtime ownership, treat those references as historical
context rather than the current architecture.

## 1) Design the index schema

Expand Down Expand Up @@ -245,7 +249,7 @@ Plan:
- DX: make the MCP server runnable like `npx … --vault <path>` (CLI args + published package/wrapper)
- Upgrades: document schema/model change behavior (when a full reindex is required)

## 10) Codex integration via plugin-hosted MCP service (localhost)
## 10) Codex integration via plugin-hosted MCP service (historical implementation record)

Goal:

Expand Down Expand Up @@ -392,7 +396,7 @@ Fallback approach (acceptable, higher overhead): per-session subprocess
- Pros: strongest isolation, simplest correctness story.
- Cons: more processes, more ports, more restart surface.

## 11) Portfolio-aligned transition plan (Python-first, local-first)
## 11) Portfolio-aligned transition plan (historical record)

Goal:

Expand Down
4 changes: 2 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ This folder is organized so you can read the AILSS system docs in order: **Conte
## Architecture

- Package structure: [architecture/packages.md](./architecture/packages.md)
- Python-first local agent baseline: [architecture/python-first-local-agent-backend.md](./architecture/python-first-local-agent-backend.md)
- Python-owned local runtime architecture: [architecture/python-first-local-agent-backend.md](./architecture/python-first-local-agent-backend.md)
- Python parity for MCP tool surface: [architecture/python-mcp-parity.md](./architecture/python-mcp-parity.md)
- Staged removal of the legacy Node/TypeScript runtime path: [architecture/legacy-node-typescript-runtime-removal.md](./architecture/legacy-node-typescript-runtime-removal.md)
- Legacy Node/TypeScript runtime removal record: [architecture/legacy-node-typescript-runtime-removal.md](./architecture/legacy-node-typescript-runtime-removal.md)
- Data & database: [architecture/data-db.md](./architecture/data-db.md)

## Ops
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ export function renderAdvancedSection(
});
});

advancedContainer.createEl("h4", { text: "MCP service (transition layer)" });
advancedContainer.createEl("h4", { text: "MCP service (localhost)" });

new Setting(advancedContainer)
.setName("Command")
Expand Down
3 changes: 2 additions & 1 deletion pnpm-workspace.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
packages:
- "packages/*"
- "packages/core"
- "packages/obsidian-plugin"

# pnpm v10 supply-chain security: only allow dependencies that need build scripts
onlyBuiltDependencies:
Expand Down
Loading