codex 5.3-xhigh: Implement support for SQLite

SQLITE_MIGRATIONS.md

@@ -0,0 +1,136 @@
+# SQLite Migrations (Prisma Next)
+
+This document describes the current SQLite migration scope in Prisma Next and a future roadmap for broader migration support.
+
+It is intentionally **target-owned**: the implementation lives in `@prisma-next/target-sqlite` and must not leak dialect-specific logic into SQL family lanes/runtime.
+
+## Current Scope (MVP)
+
+SQLite migrations are currently **additive-only** and optimized for the primary MVP flow:
+
+- `prisma-next db init` on an empty SQLite database file
+- contract marker + ledger tables written in the same DB
+
+The SQLite migration planner/runner live in:
+
+- `packages/3-targets/3-targets/sqlite/src/core/migrations/planner.ts`
+- `packages/3-targets/3-targets/sqlite/src/core/migrations/runner.ts`
+
+### Supported Operations
+
+- Create missing tables
+- Create missing columns (nullable only)
+- Create indexes / unique indexes (where supported by SQLite)
+- Create foreign keys as part of **new table creation**
+- Create and maintain target-owned control tables:
+  - `prisma_contract_marker`
+  - `prisma_contract_ledger`
+
+### Explicitly Unsupported (For Now)
+
+These changes require a table rebuild strategy and are expected to fail fast with actionable errors:
+
+- Dropping columns
+- Changing column types or nullability
+- Changing default expressions
+- Adding/removing/changing foreign keys on existing tables
+- Renaming tables/columns (unless represented as drop+add with rebuild)
+- Altering primary keys
+
+## Why This Is Hard In SQLite
+
+SQLite has limited `ALTER TABLE` support. Many schema changes require:
+
+- Creating a new table with the desired schema
+- Copying data over
+- Dropping the old table
+- Renaming the new table
+- Recreating indexes and constraints
+
+This is feasible, but it has non-trivial edge cases and needs a careful, deterministic planner so `db init` and `db verify` remain reliable.
+
+## Roadmap: Full Diff Migrations (Plan B)
+
+This section describes a concrete future approach to support broader diffs while preserving Prisma Next architectural boundaries.
+
+### 1) Detect Which Tables Need Rebuild
+
+During planning, classify per-table diffs into:
+
+- **Additive** (can be done with CREATE TABLE / ADD COLUMN / CREATE INDEX)
+- **Rebuild-required** (anything that SQLite cannot express as a safe ALTER)
+
+Examples of rebuild-required diffs:
+
+- Drop/rename column
+- Type change
+- Not-null change
+- PK change
+- FK change on existing table
+
+### 2) Rebuild Algorithm (Single Table)
+
+For each table `T` that must be rebuilt:
+
+1. Compute `T_new` schema from the **desired contract**.
+1. Create a temp table, e.g. `__prisma_next_new_T`:
+   - Use the desired column list, PK, FKs, and constraints.
+1. Copy data:
+   - `INSERT INTO __prisma_next_new_T(colA, colB, ...) SELECT oldA, oldB, ... FROM T;`
+   - For dropped/renamed columns, omit or map them.
+   - For new NOT NULL columns, require a default or fail the plan.
+1. Drop old table `T`.
+1. Rename temp table to `T`.
+1. Recreate indexes (and any triggers/views if Prisma Next ever owns them).
+
+Transaction strategy:
+
+- Prefer one transaction per rebuild wave.
+- Use `BEGIN IMMEDIATE` to avoid mid-migration write races.
+- Temporarily disable FK enforcement if required for the drop/rename steps:
+  - `PRAGMA foreign_keys = OFF` (then re-enable and validate at the end).
+
+### 3) Dependency Ordering (Multiple Tables)
+
+When multiple tables require rebuild, plan in waves:
+
+- Rebuild tables without inbound FKs first (or disable FKs temporarily).
+- Rebuild referenced tables before referencing tables if enforcing FKs during copy.
+
+If FK disable is used, do a final validation phase:
+
+- `PRAGMA foreign_key_check`
+- Fail with a structured error that includes the violating row/table.
+
+### 4) Data Safety and Explicit Failures
+
+A rebuild plan must fail fast when it cannot guarantee correctness, for example:
+
+- Dropping a column would lose required data and there is no explicit mapping.
+- Type conversion is lossy or invalid for existing values.
+- New NOT NULL column has no default and no mapping.
+
+These failures must be reported as stable errors with:
+
+- the table/column involved
+- what diff triggered the rebuild
+- the required user action (e.g. "provide a default" or "write a manual migration")
+
+### 5) How This Fits Prisma Next Boundaries
+
+- The **planner/runner** remain fully within `@prisma-next/target-sqlite`.
+- SQL family runtime/lane remains dialect-agnostic (no branching on `sqlite`).
+- Introspection stays adapter-owned (`@prisma-next/adapter-sqlite`) and should not embed migration logic.
+- Marker and ledger schema remain target-owned and must be excluded from strict schema verification.
+
+## Testing Strategy
+
+When implementing Plan B, add targeted tests under `@prisma-next/target-sqlite`:
+
+- Unit tests for diff classification (additive vs rebuild)
+- Golden tests for planned SQL statements
+- Integration tests with a real temp DB file:
+  - seed schema v1 + data
+  - migrate to v2 via rebuild
+  - verify data preservation and FK correctness
+

architecture.config.json

@@ -108,6 +108,18 @@
       "layer": "targets",
       "plane": "runtime"
     },
+    {
+      "glob": "packages/3-targets/3-targets/sqlite/src/exports/control.ts",
+      "domain": "extensions",
+      "layer": "targets",
+      "plane": "migration"
+    },
+    {
+      "glob": "packages/3-targets/3-targets/sqlite/src/exports/runtime.ts",
+      "domain": "extensions",
+      "layer": "targets",
+      "plane": "runtime"
+    },
     {
       "glob": "packages/3-targets/6-adapters/postgres/src/core/**",
       "domain": "targets",
@@ -126,6 +138,24 @@
       "layer": "adapters",
       "plane": "runtime"
     },
+    {
+      "glob": "packages/3-targets/6-adapters/sqlite/src/core/**",
+      "domain": "targets",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-targets/6-adapters/sqlite/src/exports/control.ts",
+      "domain": "targets",
+      "layer": "adapters",
+      "plane": "migration"
+    },
+    {
+      "glob": "packages/3-targets/6-adapters/sqlite/src/exports/runtime.ts",
+      "domain": "targets",
+      "layer": "adapters",
+      "plane": "runtime"
+    },
     {
       "glob": "packages/3-targets/7-drivers/postgres/src/exports/control.ts",
       "domain": "targets",
@@ -138,6 +168,18 @@
       "layer": "drivers",
       "plane": "runtime"
     },
+    {
+      "glob": "packages/3-targets/7-drivers/sqlite/src/exports/control.ts",
+      "domain": "targets",
+      "layer": "drivers",
+      "plane": "migration"
+    },
+    {
+      "glob": "packages/3-targets/7-drivers/sqlite/src/exports/runtime.ts",
+      "domain": "targets",
+      "layer": "drivers",
+      "plane": "runtime"
+    },
     {
       "glob": "packages/3-extensions/compat-prisma/**",
       "domain": "extensions",
@@ -179,6 +221,54 @@
       "domain": "extensions",
       "layer": "adapters",
       "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/core/**",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/types/**",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/control.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "migration"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/runtime.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "runtime"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/codec-types.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/operation-types.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/column-types.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
+    },
+    {
+      "glob": "packages/3-extensions/sqlite-vector/src/exports/pack.ts",
+      "domain": "extensions",
+      "layer": "adapters",
+      "plane": "shared"
     }
   ],
   "rules": {

docs/reference/capabilities.md

@@ -4,6 +4,26 @@ This document defines the canonical capability keys and reserved namespaces used
 
 Capabilities describe **what the database environment can do**. Adapters report capabilities at connect time, and the runtime negotiates them with extension packs. The contract only **declares requirements** (`contract.capabilities`) and pins the resulting `profileHash`; it does not define capabilities.
 
+## Implementation Note (Current Prototype)
+
+The current Prisma Next implementation stores contract capability requirements **per target**:
+
+- Lanes gate features via `contract.capabilities[contract.target]` (keys like `returning`, `lateral`, `jsonAgg`).
+- Extension packs typically declare required flags under the same target key (e.g. `sqlitevector/cosine` under `sqlite`).
+
+The `sql.*` namespace model described below is the long-term intended shape for adapter capability advertisement and negotiation. It is not yet the shape used by lane gating.
+
+Example contract capabilities (today):
+
+```json
+{
+  "capabilities": {
+    "postgres": { "returning": true, "lateral": true, "jsonAgg": true, "pgvector/cosine": true },
+    "sqlite": { "returning": true, "lateral": true, "jsonAgg": true, "sqlitevector/cosine": true }
+  }
+}
+```
+
 ## Adapter (database) capabilities
 
 Adapter-reported features of the database runtime. These are not contract-owned; they are discovered and negotiated.