feat: database with configuration

.github/workflows/database-e2e.yml

@@ -43,10 +43,10 @@ jobs:
       # GHA `services:` blocks can't pass `-c wal_level=logical` to postgres,
       # which the row-change tests require. Reuse the same docker-compose
       # stack the harness uses locally for dev/CI parity.
-      - name: Install iii engine (latest from main)
+      - name: Install iii engine (next)
         run: |
           curl -fsSL --retry 3 --retry-connrefused --retry-delay 5 \
-            https://install.iii.dev/iii/main/install.sh | sh
+            https://install.iii.dev/iii/main/install.sh | sh -s -- --next
           echo "$HOME/.local/bin" >> "$GITHUB_PATH"
 
       - name: Verify engine

database/README.md

@@ -17,26 +17,64 @@ iii worker add database@1.0.0
 
 ## Configure
 
-Add a single `databases` block to your `config.yaml`. SQLite is the recommended starting point — no server, just a file:
+Runtime settings live in the **`configuration` worker** under id **`database`**. The worker registers its JSON Schema at startup, reads the live value via `configuration::get`, and hot-reloads connection pools when the value changes.
+
+Persisted values default to `./data/configuration/database.yaml` (fs adapter). Edit that file directly or call `configuration::set` — both propagate without a worker restart.
+
+### Zero-config default
+
+With no seed file and no stored configuration value, the worker uses a built-in default:
 
 ```yaml
-workers:
-  - name: database
-    config:
-      databases:
-        primary:
-          url: sqlite:./data/iii.db
-          pool:
-            max: 10
-            idle_timeout_ms: 30000
-            acquire_timeout_ms: 5000
-        analytics:
-          url: ${ANALYTICS_URL}    # postgres:// or mysql://
-          pool: { max: 5 }
+databases:
+  primary:
+    url: sqlite:./data/iii.db
+    pool:
+      max: 10
+      idle_timeout_ms: 30000
+      acquire_timeout_ms: 5000
 ```
 
+This is seeded into the `configuration` worker on first register and used as a runtime fallback when the stored value is `null`.
+
+### Optional seed file
+
+Pass `--config <path>` to supply a YAML seed file. When present, its `databases` block is passed as `initial_value` on `configuration::register` (overriding the built-in default for first-time registration). See [`config.yaml.example`](config.yaml.example).
+
+Engine-managed deployments can inline config under the worker entry; the engine delivers it via `--config` as before.
+
+### Value shape
+
+SQLite is the recommended starting point — no server, just a file:
+
+```yaml
+databases:
+  primary:
+    url: sqlite:./data/iii.db
+    pool:
+      max: 10
+      idle_timeout_ms: 30000
+      acquire_timeout_ms: 5000
+  analytics:
+    url: ${ANALYTICS_URL:postgres://localhost/analytics}
+    pool: { max: 5 }
+```
+
+Set or replace the whole value:
+
+```bash
+iii trigger configuration::get id=database
+iii trigger configuration::set id=database value='{"databases":{"primary":{"url":"sqlite:./data/iii.db"}}}'
+```
+
+Env placeholders use **`${VAR:default}`** syntax. The configuration worker expands them on every `configuration::get` call, so env changes propagate without a restart.
+
 URL scheme picks the driver: `sqlite:`, `postgres://`, `postgresql://`, `mysql://`.
 
+### Hot reload
+
+When configuration changes (`configuration::set`, or an external edit to `./data/configuration/database.yaml`), the worker rebuilds connection pools in place. Invalid configs are rejected and the previous pools are kept. In-flight prepared-statement handles and open transactions continue on their original pool until they expire.
+
 ### TLS (postgres + mysql)
 
 Postgres and mysql connections default to **`tls.mode: require`** — TLS handshake required, certificate chain validated against the system trust store, hostname verification skipped (matches libpq's `sslmode=require`). Override per-database:

database/config.yaml.example

@@ -1,10 +1,18 @@
-workers:
-  - name: database
-    config:
-      databases:
-        primary:
-          url: sqlite:./data/iii.db
-          pool:
-            max: 10
-            idle_timeout_ms: 30000
-            acquire_timeout_ms: 5000
+# Optional seed file for first-time registration (--config ./config.yaml.example).
+# When omitted, the worker seeds the built-in default:
+#
+#   databases:
+#     primary:
+#       url: sqlite:./data/iii.db
+#       pool: { max: 10, idle_timeout_ms: 30000, acquire_timeout_ms: 5000 }
+#
+# After the first boot, the runtime source of truth is the `configuration`
+# worker entry `database`, persisted at ./data/configuration/database.yaml.
+
+databases:
+  primary:
+    url: sqlite:./data/iii.db
+    pool:
+      max: 10
+      idle_timeout_ms: 30000
+      acquire_timeout_ms: 5000

database/skills/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: database
+description: >-
+  Run SQL against PostgreSQL, MySQL, or SQLite from the iii engine — reads,
+  writes, transactions, and prepared statements over managed connection pools.
+---
+
+# database
+
+The database worker connects to PostgreSQL, MySQL, and SQLite through a
+managed per-database connection pool. Every callable surface lives under
+the `database::*` namespace. The driver is chosen from each database URL
+scheme (`sqlite:`, `postgres://`, `postgresql://`, `mysql://`).
+
+Runtime settings live in the `configuration` worker under id `database`;
+pools hot-reload when the value changes. SQLite is the recommended starting
+point. Placeholder syntax: `?` for SQLite and MySQL, `$1`/`$2`/… for Postgres.
+
+## When to Use
+
+- You need to read rows from a configured database (`database::query`).
+- You need to insert, update, delete, or run DDL and read affected-row
+  counts or autoincrement ids (`database::execute`).
+- Several statements must commit or roll back together as one unit
+  (`database::transaction` or the interactive transaction surface).
+- The same parameterized SQL will run many times and you want to skip
+  per-call parse/plan cost (`database::prepareStatement` +
+  `database::runStatement`).
+- You need read-your-writes across round-trips with logic between steps
+  (`database::beginTransaction` … `commitTransaction` / `rollbackTransaction`).
+- You want to react to Postgres row-level changes once logical replication
+  streaming ships (`database::row-change` trigger — see below).
+
+## Boundaries
+
+- Not a migration tool, ORM, or schema designer — pass raw SQL only.
+- Not a general pub/sub bus — use `database::row-change` only for Postgres
+  table change feeds, not for application events.
+- `database::query` is read-oriented; use `database::execute` for writes.
+  Running a SELECT through `execute` discards rows.
+- Prepared handles pin a pool connection until TTL expiry — not transactions.
+  Batch `database::transaction` needs every statement up front; use the
+  interactive surface when code must branch between steps.
+- MySQL ignores the `returning` option on `execute` (warn-once). SQLite
+  degrades `read_committed` / `repeatable_read` isolation to serializable.
+- For filesystem or shell operations, use the `shell` worker instead.
+
+## Functions
+
+- `database::query` — run read-only SQL and return rows, row count, and
+  column metadata.
+- `database::execute` — run write SQL (INSERT/UPDATE/DELETE/DDL) and
+  return affected rows, optional last insert id, and optional RETURNING rows.
+- `database::prepareStatement` — parse and plan SQL once; return a handle
+  that pins a pool connection until TTL expiry.
+- `database::runStatement` — re-execute a prepared handle with new bind
+  params; response shape matches `query`.
+- `database::transaction` — run an ordered batch of statements atomically;
+  rolls back on first failure and reports `failed_index`.
+- `database::beginTransaction` — open an interactive transaction and
+  return an id plus expiry deadline.
+- `database::transactionQuery` — read SQL inside an open interactive
+  transaction; same envelope as `query`.
+- `database::transactionExecute` — write SQL inside an open interactive
+  transaction; same envelope as `execute`. Rejects bare transaction-control
+  SQL — finalize via `commitTransaction` or `rollbackTransaction`.
+- `database::commitTransaction` — commit and finalize an interactive
+  transaction.
+- `database::rollbackTransaction` — roll back and finalize an interactive
+  transaction.
+
+Interactive transactions auto-roll back when `timeout_ms` elapses (default
+30 s, max 5 min). Prepared handles default to a 1 h TTL (max 24 h) with no
+explicit release call — let them expire or stop using them when done.
+
+## Reactive triggers
+
+Register a `database::row-change` trigger when a function should run
+automatically on Postgres INSERT/UPDATE/DELETE for specific tables — without
+polling with `database::query`.
+
+Reach for it when:
+
+- A downstream worker or workflow must react to row mutations in near real
+  time on Postgres.
+- You need decoded row payloads (old/new values) from logical replication
+  rather than polling an outbox table.
+
+Do not bind when:
+
+- The writer already has the new row in its `execute` or `transactionExecute`
+  return payload.
+- You are on SQLite or MySQL — this trigger type is Postgres-only.
+- You need events today — v1.0.0 returns `UNSUPPORTED` on `registerTrigger`
+  pending an upstream `tokio-postgres` replication API release.
+
+### How to bind
+
+1. Register a handler: `registerFunction('stream::on-row-change', handler)`.
+2. Register the trigger:
+
+```typescript
+iii.registerTrigger({
+  type: 'database::row-change',
+  function_id: 'stream::on-row-change',
+  config: {
+    db: 'primary',
+    schema: 'public',
+    tables: ['orders', 'payments'],
+    // optional: slot_name, publication_name — see get function info
+  },
+})
+```
+
+Config: `db`, `schema` (default `public`), `tables`. Slot/publication names
+derive from `trigger_id` unless overridden. For event payload shape, call
+`get function info` on the trigger type or handler function id.