From c12f5658d79bfe6f2033f6862ea0b301f4bd5e8c Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 14 Jun 2026 23:25:57 +0000 Subject: [PATCH 01/13] docs: add live interactivity concept (Lumens) --- docs/interactivity-concept.md | 441 ++++++++++++++++++++++++++++++++++ 1 file changed, 441 insertions(+) create mode 100644 docs/interactivity-concept.md diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md new file mode 100644 index 0000000..6a439c1 --- /dev/null +++ b/docs/interactivity-concept.md @@ -0,0 +1,441 @@ +# Omadia UI — Live Interactivity Concept (Lumens) + +> How Omadia UI gains rich, agent-generated, **Tier-1-fast** interactivity — +> Tetris, interactive data workflows, unusual visualisations (think the old +> HDD-defragmenter view), live maps — **without** giving up the whitelist / +> no-arbitrary-code security model that makes the canvas safe. +> +> This is the Omadia answer to Claude's Live Artifacts, deliberately +> re-aimed: more capable where Live Artifacts is uselessly limited, and +> *structurally* safer than "arbitrary code in a sandbox". +> +> Companion to `CONCEPT.md` (the *what* of the canvas), `visual-spec.md` +> (Lume), `docs/protocol/1.0.md` (the wire format) and `design-rationale.md` +> (the *why*). This document is **concept only** — no implementation, no PR +> plan. It extends, and stays inside, the architecture in `CONCEPT.md`. + +Version 0.1 — first draft. Introduces the **Lumen**: a self-contained, +declarative, deterministic interactive unit that runs in a bounded VM on +Tier 1, is generated and brokered agentically on Tiers 2/3, and is safe to +share and to save as a preset *because* it is data, not code. + +--- + +## 0. The problem with the two existing answers + +There are two ways the industry currently lets an LLM produce "an +interactive thing", and both are wrong for Omadia: + +| Approach | What it does | Why it fails for us | +|---|---|---| +| **Omadia today** (24 declarative primitives + whitelist parser) | Agent emits a JSON tree of fixed primitives; Tier 1 renders it. Safe, beautiful, instant. | Cannot express a *game loop*, *custom per-frame rendering*, *local rules/physics*, or *novel visualisations*. There is no primitive for "a falling tetromino", and there never should be — you cannot enumerate every interactive idea as a primitive. | +| **Claude Live Artifacts** (arbitrary React/JS in a sandboxed iframe) | Agent writes real code; a sandbox runs it. Maximally expressive *in compute*. | The sandbox is **all-or-nothing and near-empty**: no real data, no network, no persistence, no host integration. The simplest real use cases ("load map tiles", "save my high score", "act on my Jira data") are blocked — not by missing compute, but by missing **capabilities**. And the only thing standing between the code and the user's machine is that the sandbox boundary holds. | + +The key diagnosis — and the thesis of this document: + +> **Most of what Live Artifacts blocks is blocked by missing *capabilities*, +> not missing *compute*.** Tetris does not need network. A map needs *tiles*, +> not arbitrary JS. A data workflow needs *your data and a write-back path*, +> not a Turing-complete escape hatch. +> +> So the smart trade is the inverse of Live Artifacts: **constrain the +> computation** (make it declarative, bounded, deterministic, interpreted — +> not arbitrary code) and **open the capabilities** (real data, tools, even +> network) — but **mediated, declared, and gated** through the Tier-2/3 +> orchestration Omadia already has. + +That is the whole concept in one paragraph. Everything below is the +mechanics. + +--- + +## 1. The Lumen — definition + +A **Lumen** is a self-contained interactive unit on the canvas. It is the +omadia-native equivalent of a Live Artifact, but it is **declarative data**, +not code. A Lumen has exactly four declared parts, plus an optional fifth: + +```ts +type Lumen = { + state: StateSchema; // typed, bounded, serialisable state + transitions: Record; // pure (state, event) -> state, in Lume Expressions + view: ViewExpr; // pure state -> primitive/scene tree, in Lume Expressions + events: EventBinding[]; // declared inputs -> transitions (keys, pointer, tick, timer) + capabilities?: CapabilityRequest[]; // declared, default-deny doors to the outside world +}; +``` + +Plain-English mapping: + +- **`state`** — the Lumen's memory. For Tetris: the board grid, the active + piece, score, level, game-over flag. Typed and size-capped. +- **`transitions`** — the rules. Pure functions `(state, event) → newState`, + written in **Lume Expressions** (a small, total, sandboxed expression + language — §3). For Tetris: `tick` drops the piece, `moveLeft`, `rotate`, + `lockPiece`, `clearLines`. No I/O, no host access, deterministic. +- **`view`** — the look. A pure function `state → tree`, producing either + ordinary Omadia primitives (so a Lumen can be a live form/table/dashboard) + **or** a `scene` (the new immediate-mode draw surface, §4) for custom + visuals like a game board or a defrag grid. +- **`events`** — what wakes the Lumen: declared keys (`ArrowLeft`, `Space`), + pointer events on scene elements, a host **tick** at a declared, capped + rate (≤ 60 fps), or a timer. Input is whitelisted — a Lumen cannot listen + to keys it did not declare. +- **`capabilities`** — the *only* way a Lumen touches anything outside its + own state: persistence, data fetch, map tiles, clipboard, share. Default + deny. Each is effect-classified and brokered by Tier 2 (§5). + +Because all five parts are declarative data validated by the same kind of +whitelist parser Omadia already ships, a Lumen flows through the **existing** +`surface_snapshot` / `surface_patch` grammar as ordinary tree content. **No +new transport is required for the Lumen itself** — only the validator and the +Tier-1 runtime are new. + +> **Naming (light-vocabulary discipline, per `CONCEPT.md` §"Light +> Vocabulary").** "Lumen" — the SI unit of luminous flux — is *proposed*, not +> locked: a Lumen is a discrete, self-contained, portable quantum of Lume. +> The term labels something genuinely new (a shareable, self-running unit of +> the material) and makes it more thinkable. Per the discipline, it needs +> sign-off before it enters the locked vocabulary; until then the descriptive +> fallback is "live canvas unit". The reserved word **Spark** ("a discrete +> generative-initiation event") is *not* this — a Spark would be the *act* of +> the agent condensing a Lumen, distinct from the Lumen itself. + +--- + +## 2. Why this is structurally safer than a sandbox (the security thesis) + +The Omadia security moat today is the **whitelist parser**: the renderer +never executes agent input, it validates it against a closed schema and +renders only known primitives (CSP is `default-src 'self'`, no `unsafe-eval`; +the Ajv validators are pre-compiled at build time exactly so raw `Function()` +compilation never happens — see `app/src/renderer/src/validate/`). A Lumen +must not punch a hole in that moat. It does not, because: + +1. **No arbitrary code, ever.** A Lumen carries *data* (state schema, + expression ASTs, view template, event bindings). The Tier-1 runtime is an + **interpreter shipped inside the Host App**, not `eval`. Lume Expressions + are parsed to a validated AST and walked by a deterministic evaluator. CSP + stays `default-src 'self'`, no `unsafe-eval`, no iframe-with-script. The + whitelist philosophy is *extended*, not *abandoned*: unknown expression + node → reject; unknown event type → reject; unknown capability → reject. +2. **Bounded computation — cannot hang, cannot DoS the host.** Every + transition and every view evaluation runs under a **gas budget** + (instruction count) and a **wall-clock ceiling** per frame. Loops are + bounded comprehensions (`map`/`fold` over *declared* collections) only — + no open `while`, no unbounded/​unguarded recursion. State has a size cap. + Frame rate is capped by the host. A pathological Lumen hits its budget and + is **halted with a `surface_error`**; it can never freeze the canvas or + starve other Lumens. (Contrast: a sandboxed `while(true)` can wedge a + worker.) +3. **Determinism by construction.** All non-determinism is host-supplied and + seeded: `random` is a seeded PRNG handed in by the host; `now`/tick + timestamps come from the host clock. Given `(state, event)` the next state + is identical on every machine. This single property buys four things at + once: **replay** (Trace can replay a session), **undo/redo** (navigate the + state history), **safe sharing** (a recipient runs the *identical* + behaviour), and **v2 multiplayer** (deterministic ops are lockstep- and + CRDT-friendly — it rides the `treeRevision`-as-opaque-id and shared-canvas + hooks already reserved in `CONCEPT.md`). +4. **Capability default-deny, effect-classified, agent-brokered.** A Lumen + reaches the outside world only through declared capabilities, and each + capability call reuses the existing `local` / `internal` / `external-effect` + classification and the **confirmation-modal contract** from `CONCEPT.md` + §"Security Surface". The Lumen never talks to the network, the filesystem, + or a tool directly — it *requests*, Tier 2 *brokers*. (Contrast: Live + Artifacts' iframe either has a capability or does not, with no agent in the + loop to scope, confirm, or audit it.) +5. **Origin & trust for imported Lumens.** A shared or preset Lumen carries a + **capability manifest** that is surfaced to the receiving user *before + first run* ("This shared canvas wants to: save high scores · load map + tiles from OpenStreetMap. Allow?"). Capabilities are HMAC-scoped exactly + like `dataRef` (`HMAC(serverSecret, tenant ‖ user ‖ canvasSession ‖ …)`), + and external capabilities carry a provider allowlist. An imported Lumen + with an un-grantable capability is rendered inert, never silently + escalated. + +The net: a Lumen is **more capable** than a Live Artifact (it can reach real +data, tools, and — when declared and granted — the network) while being +**less dangerous**, because the dangerous axis (arbitrary code execution) is +removed entirely and the powerful axis (capabilities) is mediated by the +agent and the user instead of by a sandbox boundary alone. + +| | Claude Live Artifacts | Omadia Lumens | +|---|---|---| +| Compute model | arbitrary JS/React | declarative + bounded interpreter (no `eval`) | +| Security boundary | iframe sandbox must hold | nothing to escape — no code runs; capabilities gated | +| Real data access | essentially none | yes, via brokered capabilities + existing DataRef/Class-D | +| Network | blanket-denied | declared, provider-allowlisted, brokered (e.g. map tiles) | +| Persistence | none | `persist` capability → `memoryStore` | +| Rendering | foreign iframe, off-theme | native Lume, on-theme, 60 fps | +| Agent can read/modify it live | no | yes — state is structured, agent can introspect & live-patch | +| Safe to share / preset | not really (it's code) | yes — it's validated, deterministic data | +| Can hang the host | yes (`while(true)`) | no (gas + frame ceiling → halt) | + +--- + +## 3. Lume Expressions — the bounded language + +The transition and view functions are written in **Lume Expressions** (LX): +a small, **pure, total** expression language. It is deliberately *not* +Turing-complete in the dangerous direction. + +- **Pure & total.** Every expression is a function of its inputs with no side + effects. No statements, no mutation, no exceptions-as-control-flow. +- **Values.** numbers, booleans, strings, lists, records, and typed `state` + references. No closures over host objects, no `this`, no prototypes. +- **Operators & built-ins.** arithmetic, comparison, boolean logic, + `if`/`match`, record/list construction, and a fixed standard library + (`map`, `filter`, `fold`, `range`, `min`, `max`, `len`, string ops, a small + math set). The standard library is a **whitelist** — same discipline as the + primitive vocabulary. +- **Bounded iteration only.** `map`/`fold`/`range` over collections whose size + is bounded by `state` (itself size-capped). No `while`, no general + recursion. This is what makes the gas bound a *static* guarantee rather than + a hope. +- **Host-provided non-determinism.** `random()` and `now()` are not free + functions; they read host-seeded values passed into the evaluation context, + preserving determinism/replay. +- **Validated AST.** LX is delivered as a JSON AST (not source text), so there + is no parser-injection surface and the validator is the same shape as the + primitive whitelist: walk the tree, reject any unknown node type. + +LX is **versioned** alongside `omadia-canvas-protocol` and negotiated at the +boot handshake (the client declares the LX version and gas limits it +supports, exactly as it already declares `localOperations`). + +> **Why not WASM / QuickJS?** A hardened WASM or embedded-JS sandbox is the +> obvious "powerful escape hatch" and is noted here as a possible **v2+** +> capability for the rare Lumen that genuinely needs arbitrary compute. It is +> *not* the v1 answer because it re-introduces exactly the "a boundary must +> hold" risk this concept is built to avoid, and it breaks determinism/replay/ +> shareability. v1 deliberately ships the declarative model first and measures +> whether real use cases actually exceed it. (Same "radical-restraint-first, +> measure, then add" discipline `CONCEPT.md` uses for the prompt bar.) + +--- + +## 4. The `scene` primitive — custom visuals without custom code + +The 24 primitives cover data/UI; they do not cover "draw me a board of +coloured cells at 60 fps". One new editor-class primitive closes that gap: + +**`scene`** — a declarative, immediate-mode draw surface. The Lumen's `view` +function emits, each frame, a **draw-list** from a whitelisted shape +vocabulary: + +| Shape | For | +|---|---| +| `rect` / `roundRect` | tetromino cells, defrag blocks, bars, map markers | +| `line` / `polyline` / `path` | grids, connections, routes (reuses `vector-path` geometry) | +| `circle` / `ellipse` | nodes, dots, map pins | +| `sprite` (a `DataRef` image) | tiles, icons, game art | +| `text` | scores, labels (rendered in Lume type registers) | +| `group` / `transform` | layers, camera pan/zoom (reuses canvas-region zoom/pan affordances) | + +Properties are restricted to **theme tokens + Lume palette** (so a Lumen is +always on-theme — a game still looks like Omadia, never like a foreign +website) plus geometry. The draw-list is **data**, validated by the whitelist +parser — there is no canvas `2d`/`webgl` script handed to the agent. Tier 1 +rasterises the draw-list to canvas/WebGL natively at 60 fps from local state; +this is pure **Class A** interaction — *zero* server contact for the frame +loop (`CONCEPT.md` §"Latency paths"). + +`scene` is distinct from the existing `canvas-region` (which is a *pixel +editor buffer* for brush/blur ops). `scene` is a *retained/immediate hybrid +render target driven by Lumen state*. Both can coexist. Pointer events on +scene elements bind to Lumen events via stable element ids, so the existing +`TargetRef` and beam model work *inside* a Lumen (you can beam the agent about +a specific marker or cell). + +--- + +## 5. Capabilities — the mediated doors + +Capabilities are the heart of "better than Live Artifacts". Each is declared +in the Lumen, effect-classified, and brokered by Tier 2. Default deny. + +| Capability | Effect class | Broker path | Example | +|---|---|---|---| +| `persist(key, value)` | `internal` | Tier 2 → `memoryStore@1` under a Lumen-scoped namespace | Tetris high score; map's last viewport | +| `loadData(dataRef)` | `internal` | Tier 2 hands the Lumen a **read-only, size-capped projection** of an existing `DataRef` | drive a defrag-style viz from a real dataset; bind a map to a places table | +| `writeData(target, value)` | `internal` / `external-effect` | reuses the **Class-D mutation contract** + `writeCapabilities` manifest | an interactive triage workflow that commits status changes back to Jira | +| `tiles(provider, z/x/y)` | `internal` | Tier 2/3 fetches from a **provider-allowlisted** endpoint, returns sprite `DataRef`s | OpenStreetMap / Mapbox map tiles | +| `fetch(declaredEndpoint)` | `internal` / `external-effect` | Tier 3 tool call against an **allowlisted, agent-approved** endpoint only | a live feed into a visualisation | +| `clipboard(text)` | `external-effect` | confirmation-modal gate | "copy result" | +| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §6 | share Tetris with a colleague; save as a gallery preset | + +Mechanics: a capability call from the running Lumen is *not* a direct call. It +emits a capability-request action (effect-classified) up through the channel; +Tier 2 validates it against the Lumen's granted manifest, brokers it +(memoryStore / Tier-3 tool / allowlisted endpoint), and patches the result +back — or, for `external-effect`, raises the standard confirmation modal +first. **The Lumen's deterministic local loop keeps running** while a +capability call is in flight (async-by-default, `CONCEPT.md` +§"Async Architecture") — a map pans smoothly while new tiles stream in; a +workflow stays interactive while a write-back resolves. + +This is the precise inversion of Live Artifacts: the network is *not* banned, +it is **named, allowlisted, agent-brokered, user-confirmed, and audited in +Trace**. + +--- + +## 6. Sharing & presets — safe because it's data + +Two user-facing features the user explicitly asked for, both essentially free +once a Lumen is declarative+deterministic+capability-manifested: + +**Share a canvas / Lumen with selected users.** A Lumen serialises cleanly +(it is validated data + a capability manifest). Sharing rides the +forward-compat hooks already in `CONCEPT.md` §"Forward Compatibility": extend +`canvasOwnership` from `{kind:"single-user"}` to `{kind:"group", members}`, +and the channel plugin (already the designated **fan-out point**) multicasts +surface events to connected members. The recipient's Tier 1 **re-validates** +the Lumen and shows its **capability manifest for consent before first run**. +Because behaviour is deterministic, every member sees the identical Lumen; +because capabilities are per-user-granted, a shared Tetris can save *your* +high score without touching *mine*. Real-time multiplayer (two people in one +Tetris) is a v2 topic but is *unblocked* by determinism — deterministic ops +are exactly what lockstep/CRDT need. + +**Save as a preset.** A Lumen can be named, optionally **parameterised** +(declare which parts of `state` are preset inputs), and stored in a preset +store (`memoryStore@1` namespace, e.g. `lumen-presets//`). A +**Lumen gallery** lets the user re-instantiate "my defrag-style project +viewer" or "the team standup board" on any canvas, with fresh data bound via +`loadData`. Presets are the natural unit for an eventual community/library of +Omadia interactive templates — and they are safe to distribute precisely +because they are validated, deterministic, capability-declared data, not code. + +--- + +## 7. The agent relationship — generation *and* live introspection + +Lumens are generated agentically (Tier 2 composes the Lumen the way it +composes primitive trees today; heavy generation or data binding can recruit +Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: + +- **Agentic generation.** "Build me a Tetris" → Tier 2 emits a Lumen + (state/transitions/view/events) via `surface_snapshot`. "Make it faster as + the score climbs" → `surface_patch` adjusting the `tick` transition. The + agent refines the *running* behaviour conversationally. +- **Live introspection.** Because `state` is structured and readable, the + agent can answer "what's my high score?" or "summarise the current board" + by reading Lumen state from canvas-state — a Live Artifact is an opaque box + the model cannot see into. +- **Beam into a Lumen.** A user can beam a scene element ("why is this block + red?") or a region; the existing `TargetRef`/beam machinery resolves inside + the Lumen. +- **Composability on one canvas.** A Lumen is ordinary tree content, so a map + Lumen sits next to a Jira `table` and they can be wired (select a marker → + filter the table) through normal canvas mechanics. Live Artifacts are + isolated islands; Lumens are first-class canvas citizens. + +--- + +## 8. The user's four use cases, mapped + +| Use case | state | view | events | capabilities | Tier split | +|---|---|---|---|---|---| +| **Tetris** (build · play · share) | board grid, active piece, score, level | `scene` cell grid | declared keys + host `tick` (≤60 fps) | `persist` (high score), `share` | loop is pure **Class A** on Tier 1; generation/share on Tier 2 | +| **Interactive data workflow** | working set, step, selection, edits | primitives (table/form) or `scene` | pointer/submit | `loadData` (real data in), `writeData` (commit back via Class-D + `writeCapabilities`) | Tier 1 interaction; Tier 2 brokers writes; Tier 3 owns the system of record | +| **Defrag-style / unusual viz** | dataset projection, animation cursor | `scene` coloured-cell grid | host `tick` for animation | `loadData` | data fetch via Tier 2/3; animation pure Tier 1 | +| **Interactive map** | viewport, markers, selection | `scene` (sprites = tiles, markers) + pan/zoom | pointer (pan/zoom/click) | `tiles` (provider-allowlisted), `loadData` (places), `persist` (last viewport) | pan/zoom/render pure Tier 1; tiles brokered Tier 2/3 | + +Every one of these is impossible-or-crippled in both Omadia-today (no +behaviour model) and Live-Artifacts (no real capabilities). Each is natural +in the Lumen model. + +--- + +## 9. How it fits the existing architecture (deltas, not rewrites) + +Everything below is **additive** and stays inside the `CONCEPT.md` tier model, +authority split, and security surface. No wire-grammar rewrite. + +| Area | Delta | +|---|---| +| **Primitives** | add `scene` (editor-class) — protocol minor bump (`1.x`); draw-list is whitelisted shape data | +| **Tree content** | add the `behavior`/`lumen` section (state/transitions/view/events/capabilities) — validated by an **extended whitelist parser** (schema + LX-AST validator) | +| **Tier-1 client** | new **Lumen runtime**: deterministic LX evaluator, gas + frame ceiling, scene rasteriser, event dispatch, seeded `random`/clock. All Class-A; the frame loop never touches the server | +| **Tier-2 orchestrator** | composes/patches Lumens; **brokers capability calls**; grants/scopes capability manifests; persists Lumen state & presets; manages share/ownership | +| **Tier-3** | reached only via brokered capabilities (`tiles`, `fetch`, `writeData`, AI ops) — unchanged interface | +| **Transport** | Lumen rides existing `surface_snapshot`/`surface_patch`; capability calls reuse the effect-classified action path + `surface_action_result`/patch; **one** new optional event family `surface_capability_*` if streaming results need it | +| **Security** | extend whitelist parser to LX-AST + scene + capability manifest; reuse HMAC scoping for capability tokens; reuse `local`/`internal`/`external-effect` + confirmation modal | +| **Handshake** | client declares supported **LX version**, **gas limits**, **scene support**, and **granted capability classes** alongside `localOperations` | +| **Identity / sharing** | reuse `canvasOwnership` group extension + channel fan-out; new `lumen-presets/**` and `lumen-state/**` memory namespaces | +| **Versioning** | LX, scene vocabulary, and capability catalog versioned with `omadia-canvas-protocol`; capability catalog negotiated like the ops catalog | + +What classic channels see: **nothing** — same as `CONCEPT.md`. All additive, +engaged only behind the `canvas` capability. + +--- + +## 10. The sweet-spot dial (answering the user's central ask) + +The user's framing: Live Artifacts errs too far toward restriction; find the +better-tuned point between *possibility* and *safety*. The Lumen model tunes +**two independent dials** instead of the one coarse dial a sandbox gives you: + +1. **Compute dial — set permanently to "constrained".** Declarative, + bounded, deterministic, interpreted. This is not a per-Lumen choice; it is + the architecture. It removes the entire arbitrary-code threat class. (v2+ + may add an opt-in WASM dial for genuine outliers, gated hard.) +2. **Capability dial — set per-Lumen, per-user, per-call, openable far.** + This is where expressiveness lives. Because compute is safe, we can afford + to open capabilities *generously but explicitly*: real data, write-back, + persistence, even allowlisted network — each declared, brokered, confirmed, + audited. + +Live Artifacts collapses both dials into one ("how much of the sandbox do we +trust?") and is forced to keep it low, which is why "even the simplest use +cases are blocked". Splitting the dials is the smarter approach: **lock the +dangerous one, open the useful one.** + +--- + +## 11. Open questions for the spike (flagged, not answered) + +1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, + scene draw-list length, tick rate — measured against the four reference + Lumens (Tetris, workflow, defrag-viz, map). Spike-tunable, like the + `viewState` budget in `CONCEPT.md`. +2. **LX surface area.** Exactly which standard-library functions ship in + `LX/1.0`. Bias small; grow by minor bump. Risk: too small blocks real + Lumens; too large grows the audit surface. +3. **Scene performance ceiling.** Draw-list size at which Tier-1 raster drops + below 60 fps; whether WebGL is required for v1 or canvas-2d suffices for + the reference set. +4. **Capability granularity & consent fatigue.** How fine-grained the consent + prompt should be without nagging; defaults for trusted first-party + capabilities (`persist`) vs. always-confirm ones (`external-effect`). +5. **Determinism vs. real time.** Maps and live feeds are inherently + non-deterministic at the *data* edge; confirm that seeding the *compute* + while treating capability results as external inputs keeps replay coherent + (replay re-feeds recorded capability results, like a recorded test). +6. **LLM reliability emitting LX.** Can a fast (Haiku-class) model emit valid + LX ASTs reliably, or is Lumen generation a Sonnet/Opus job? Mirrors + Riskiest-Assumption #1 in `CONCEPT.md`. Likely: generation on a stronger + model, in-session tweaks cheaper. +7. **Preset trust & distribution.** Signing, capability-manifest review UX, + and whether a shared community gallery needs a moderation/attestation + layer (likely v2+). + +--- + +## 12. What this is not (scope discipline) + +- **Not arbitrary code execution.** No `eval`, no iframe-with-script, no WASM + in v1. If a use case truly needs Turing-complete compute, it is a flagged + v2+ escape hatch, gated hard — not the default. +- **Not a new design language.** Lumens render in Lume, in the active palette, + in the type registers. A game looks like Omadia. +- **Not a bypass of the authority/security model.** Lumens obey the same + authority split (agent owns structure, client owns view-state), the same + stable-ID discipline, the same effect classification and confirmation + contract, the same DataRef/HMAC scoping. +- **Not an implementation plan.** This document is concept only. Protocol + schema, the LX grammar spec, the scene vocabulary, the capability catalog, + reference-Lumen walkthroughs, and the PR sequence are spike deliverables, + authored after this concept is accepted. From 2679dfa6cf62aae9e835a175e800408049c7849d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 15:55:10 +0000 Subject: [PATCH 02/13] docs: add render cadence, motion & thin-client/kiosk envelope (Lumens v0.2) --- docs/interactivity-concept.md | 136 ++++++++++++++++++++++++++++++---- 1 file changed, 122 insertions(+), 14 deletions(-) diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index 6a439c1..df25f02 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -14,10 +14,18 @@ > (the *why*). This document is **concept only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. -Version 0.1 — first draft. Introduces the **Lumen**: a self-contained, -declarative, deterministic interactive unit that runs in a bounded VM on -Tier 1, is generated and brokered agentically on Tiers 2/3, and is safe to -share and to save as a preset *because* it is data, not code. +Version 0.2 — adds §5 "Render cadence, motion & the thin-client / kiosk +envelope": cadence is declared per region (`static` / `reactive` / `tick`), +**reactive-by-default (~0 % CPU at rest)** — 60 Hz never applies to the whole +tree; presentation motion is a **declarative, GPU-run animation layer** +(transitions, parallax, Ken-Burns, particles, glow), distinct from LX +simulation ticks; the "wow" comes from existing assets + generated layout + +native effects; plus the thin-client/kiosk capability ladder and a mapping to +the four priorities (security · performance · visuals · wow · generative). +v0.1 — first draft. Introduces the **Lumen**: a self-contained, declarative, +deterministic interactive unit that runs in a bounded interpreter on Tier 1, +is generated and brokered agentically on Tiers 2/3, and is safe to share and +to save as a preset *because* it is data, not code. --- @@ -83,7 +91,7 @@ Plain-English mapping: to keys it did not declare. - **`capabilities`** — the *only* way a Lumen touches anything outside its own state: persistence, data fetch, map tiles, clipboard, share. Default - deny. Each is effect-classified and brokered by Tier 2 (§5). + deny. Each is effect-classified and brokered by Tier 2 (§6). Because all five parts are declarative data validated by the same kind of whitelist parser Omadia already ships, a Lumen flows through the **existing** @@ -250,7 +258,107 @@ a specific marker or cell). --- -## 5. Capabilities — the mediated doors +## 5. Render cadence, motion & the thin-client / kiosk envelope + +Nothing about a Lumen forces 60 Hz. A blanket game-loop is the *wrong* default +for almost everything and ruinous for an always-on kiosk. This section pins +the rendering model and the realistic capability envelope on weak hardware. + +### Reactive by default — 60 Hz only where it earns it + +Cadence is declared **per node/region, not globally**, in three classes: + +| Cadence | When it runs | LX cost at rest | For | +|---|---|---|---| +| **`static`** | rendered once; redrawn only when a `surface_patch` changes it | **zero** | most kiosk/dashboard content: layout, imagery, copy, KPIs | +| **`reactive`** *(default)* | `view` re-evaluated only for the sub-tree whose `state` slice changed, on event/data | **zero** until something changes | forms, tables, controls, selection | +| **`{ tick: hz }`** | host clock drives an LX transition at a declared, capped rate, **scoped to that sub-tree only** | one bounded transition/frame for that region only | the falling tetromino, a live chart, a defrag animation | + +The runtime **dirty-tracks** which `state` slices changed and re-evaluates only +the dependent `view` branches (retained-mode + memoisation). +`requestAnimationFrame` is scheduled **only while a ticking/animating region is +live** and torn down when it settles. **At rest a Lumen costs ~0 % CPU** — a +kiosk showing a beautiful, mostly-static screen burns nothing until someone +touches it or a single badge pulses. One Lumen routinely mixes all three: a +Tetris scene ticks at 60 Hz, the score label beside it is `reactive`, the +surrounding chrome is `static`. Only the part that must move pays. + +### Motion comes from a declarative animation layer, not from LX + +"Animation" conflates two different things; separating them is what makes +*wow on weak hardware* possible: + +- **Simulation** — state genuinely evolves by rules each step (a piece falls, + cells flip). This is an LX **tick**, used sparingly. +- **Presentation motion** — a panel slides in, a glow pulses, a number counts + up, a camera eases, an image slowly pans (Ken Burns). This must **not** be an + LX tick recomputing state per frame. It is a **declarative animation** the + host runs on the compositor/GPU: + + ```json + { "animate": { "property": "opacity", "from": 0, "to": 1, "duration": 300, "easing": "ease-out" } } + ``` + +The agent *declares* enter/exit/change transitions, easing, pulses, parallax +layers, Ken-Burns pan-zoom, backdrop blur/gradient/glow, and (as a native +effect) particle emitters. The **host executes them natively on the GPU** — +zero LX per frame, 60 fps smoothness on a fanless thin client. The Lume visual +language (glow, frosted glass, surface luminosity) *is* exactly this class of +GPU-composited effect, so the wow is cheap and on-rails, never hand-coded +pixel math. + +### Where the "wow" actually comes from (generated, natively executed) + +A stunning generated kiosk screen = **existing assets + generated layout + +native Lume effects + a touch of declarative motion**: + +- **Existing image/video material** → `sprite`/`image`/`media` via `DataRef` + (brand imagery, product photos, loops). The agent *composes*; it does not + synthesise pixels on-device. +- **Native effect vocabulary** (whitelisted, GPU): glow, backdrop-blur, + gradient, shadow, parallax, Ken-Burns, particle emitter — *declared*, not + computed in LX. +- **Generative authorship, native execution**: the agent generates the + composition, the motion declarations, and the asset bindings as data; the + host runs them on native rails. The result looks like a hand-tuned demo, + but it was generated. + +### Thin-client / kiosk capability ladder + +Because the client only ever does *bounded interpretation* + *raster of a +draw-list it already has* (everything heavy is brokered to Tier 2/3), the +thin client is the design centre, not the stress test: + +- 🟢 **Flüssig:** puzzle/board/card games (Tetris, 2048, chess), interactive + dashboards & workflows, data-viz incl. defrag-style grids to ~5–10 k cells + (canvas2d; more on WebGL), maps (tiles are GPU-composited images fetched by + Tier 2/3 — the client doesn't compute the map). +- 🟡 **Mit Maßnahmen:** large cellular automata / sims over big grids (drop to + 30 Hz, smaller grid, or push the step to Tier 3), very large scenes (WebGL + rasteriser, same declarative draw-list). +- 🔴 **Nicht in reinem LX:** realtime 3D, thousand-body physics, 100 k-particle + systems, per-pixel image processing, heavy solvers/ML. → escape hatches: + **native local-ops catalog** (Class B, pixel work), **Tier 3** (heavy + compute returns a `DataRef` the Lumen merely visualises), **v2 WASM** (hard + gated, rare outliers). + +The decisive kiosk property no "arbitrary-code-in-sandbox" approach has: the +**gas budget guarantees a clean halt** — a badly generated Lumen never freezes +an always-on display, it is stopped with a `surface_error`. + +### Mapping to the four priorities + +| Priority | How the cadence/motion model delivers it | +|---|---| +| **Security** | host-owned capped clock; `static`/`reactive` branches execute *no logic* at rest; motion is declarative (no per-frame code); capability default-deny unchanged; gas guarantees a clean halt | +| **Performance** | ~0 % CPU at rest; only ticking/animating sub-trees cost frames; GPU does the pretty part — built for fanless always-on kiosks | +| **High-quality visuals** | native Lume effects + GPU compositing + real assets via `DataRef`; consistent, on-theme, never a foreign iframe | +| **Wow-effect** | declarative transitions, parallax, Ken-Burns, particles, glow — generated by the agent, run natively at 60 fps even on thin clients | +| **Generative** | the agent authors structure, motion and asset bindings as data; existing material is referenced, not regenerated | + +--- + +## 6. Capabilities — the mediated doors Capabilities are the heart of "better than Live Artifacts". Each is declared in the Lumen, effect-classified, and brokered by Tier 2. Default deny. @@ -263,7 +371,7 @@ in the Lumen, effect-classified, and brokered by Tier 2. Default deny. | `tiles(provider, z/x/y)` | `internal` | Tier 2/3 fetches from a **provider-allowlisted** endpoint, returns sprite `DataRef`s | OpenStreetMap / Mapbox map tiles | | `fetch(declaredEndpoint)` | `internal` / `external-effect` | Tier 3 tool call against an **allowlisted, agent-approved** endpoint only | a live feed into a visualisation | | `clipboard(text)` | `external-effect` | confirmation-modal gate | "copy result" | -| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §6 | share Tetris with a colleague; save as a gallery preset | +| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §7 | share Tetris with a colleague; save as a gallery preset | Mechanics: a capability call from the running Lumen is *not* a direct call. It emits a capability-request action (effect-classified) up through the channel; @@ -281,7 +389,7 @@ Trace**. --- -## 6. Sharing & presets — safe because it's data +## 7. Sharing & presets — safe because it's data Two user-facing features the user explicitly asked for, both essentially free once a Lumen is declarative+deterministic+capability-manifested: @@ -310,7 +418,7 @@ because they are validated, deterministic, capability-declared data, not code. --- -## 7. The agent relationship — generation *and* live introspection +## 8. The agent relationship — generation *and* live introspection Lumens are generated agentically (Tier 2 composes the Lumen the way it composes primitive trees today; heavy generation or data binding can recruit @@ -334,7 +442,7 @@ Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: --- -## 8. The user's four use cases, mapped +## 9. The user's four use cases, mapped | Use case | state | view | events | capabilities | Tier split | |---|---|---|---|---|---| @@ -349,7 +457,7 @@ in the Lumen model. --- -## 9. How it fits the existing architecture (deltas, not rewrites) +## 10. How it fits the existing architecture (deltas, not rewrites) Everything below is **additive** and stays inside the `CONCEPT.md` tier model, authority split, and security surface. No wire-grammar rewrite. @@ -372,7 +480,7 @@ engaged only behind the `canvas` capability. --- -## 10. The sweet-spot dial (answering the user's central ask) +## 11. The sweet-spot dial (answering the user's central ask) The user's framing: Live Artifacts errs too far toward restriction; find the better-tuned point between *possibility* and *safety*. The Lumen model tunes @@ -395,7 +503,7 @@ dangerous one, open the useful one.** --- -## 11. Open questions for the spike (flagged, not answered) +## 12. Open questions for the spike (flagged, not answered) 1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, scene draw-list length, tick rate — measured against the four reference @@ -424,7 +532,7 @@ dangerous one, open the useful one.** --- -## 12. What this is not (scope discipline) +## 13. What this is not (scope discipline) - **Not arbitrary code execution.** No `eval`, no iframe-with-script, no WASM in v1. If a use case truly needs Turing-complete compute, it is a flagged From 7bd4cd458342b21a31e5689722902c3a132e7528 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 16:37:32 +0000 Subject: [PATCH 03/13] docs: Lume-correctness fixes, touch input, asset transport/caching/generation; add kiosk Lumen mockup --- docs/interactivity-concept.md | 147 ++++++++++- docs/mockups/kiosk-lumen-aura.html | 393 +++++++++++++++++++++++++++++ 2 files changed, 531 insertions(+), 9 deletions(-) create mode 100644 docs/mockups/kiosk-lumen-aura.html diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index df25f02..fce2210 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -14,6 +14,19 @@ > (the *why*). This document is **concept only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. +Version 0.3 — review fixes. **Lume correctness**: the motion/effect vocabulary +is now exact to `visual-spec.md` — Lume is **light-as-material** (surface +luminosity, accent-as-illumination, directional borders, soft corners), *not* +glassmorphism; the erroneous "frosted glass / backdrop-blur" effects are +removed (the only blur is the transient 800 ms condensation). **Touch & pointer +input** added to §5 as first-class (tap/longPress/drag/pinch/swipe, 44 pt +hit-targets, host gesture arbitration, no hover dependency, input-modality +handshake) for kiosk/iPad. **Assets** §6.1 — transport + **content-addressed, +never-stale caching** (`id = kind-sha256(content)`, cache-busting by +construction, HMAC-signed fetch, explicit invalidation). **Generated material** +§6.2 — images/sounds/voice come from **omadia-core LLM connectors** (Tier 3), +the Lumen only requests (`generateAsset`) and renders; nothing generated on the +client. Version 0.2 — adds §5 "Render cadence, motion & the thin-client / kiosk envelope": cadence is declared per region (`static` / `reactive` / `tick`), **reactive-by-default (~0 % CPU at rest)** — 60 Hz never applies to the whole @@ -300,12 +313,18 @@ surrounding chrome is `static`. Only the part that must move pays. ``` The agent *declares* enter/exit/change transitions, easing, pulses, parallax -layers, Ken-Burns pan-zoom, backdrop blur/gradient/glow, and (as a native -effect) particle emitters. The **host executes them natively on the GPU** — -zero LX per frame, 60 fps smoothness on a fanless thin client. The Lume visual -language (glow, frosted glass, surface luminosity) *is* exactly this class of -GPU-composited effect, so the wow is cheap and on-rails, never hand-coded -pixel math. +layers, Ken-Burns pan-zoom on assets, **accent glow / halo / aura** (the +two-stop and donut glow recipes), **surface luminosity**, **directional +light**, the **patch-condensation** materialisation, and (as a native effect) +light-mote particle emitters. The **host executes them natively on the GPU** — +zero LX per frame, 60 fps smoothness on a fanless thin client. This *is* the +Lume material — **light-as-material**, condensed out of light: surface +luminosity, accent-as-illumination, directional borders, soft corners +(`visual-spec.md` §1.2). **Lume is explicitly *not* glassmorphism** — no +refraction, no blur-as-chrome, "solid light, not see-through plastic" +(`visual-spec.md` §1.3). The only blur is the transient 800 ms condensation +materialisation, never standing chrome. So the wow is cheap and on-rails, +never hand-coded pixel math. ### Where the "wow" actually comes from (generated, natively executed) @@ -315,14 +334,60 @@ native Lume effects + a touch of declarative motion**: - **Existing image/video material** → `sprite`/`image`/`media` via `DataRef` (brand imagery, product photos, loops). The agent *composes*; it does not synthesise pixels on-device. -- **Native effect vocabulary** (whitelisted, GPU): glow, backdrop-blur, - gradient, shadow, parallax, Ken-Burns, particle emitter — *declared*, not - computed in LX. +- **Native Lume effect vocabulary** (whitelisted, GPU): surface-luminosity + gradients, accent glow / halo / aura (two-stop + donut), `glow-core` inner + light, directional light, elevation, parallax depth, Ken-Burns pan-zoom, + light-mote particles — *declared*, not computed in LX. No glassmorphism, no + blur-as-chrome (`visual-spec.md` §1.3). - **Generative authorship, native execution**: the agent generates the composition, the motion declarations, and the asset bindings as data; the host runs them on native rails. The result looks like a hand-tuned demo, but it was generated. +### Touch & pointer input — first-class, not an afterthought (kiosk · iPad) + +Kiosks and iPads are touch-first, so touch is in the Lumen event model from +day one — it is **not** mouse events with a shim. A Lumen's `events` declare +**pointer-semantic** inputs that resolve identically across mouse, trackpad +and touch (the same abstraction `CONCEPT.md` already uses for *context-invoke* += long-press): + +| Declared event | Touch | Mouse/trackpad | Use | +|---|---|---|---| +| `tap` | tap | click | activate a control, place, select | +| `longPress` (~400 ms) | press-and-hold | right-click / hold | **context-invoke** → action panel + Beam (per `CONCEPT.md`) | +| `drag` | one-finger drag | press-move | move a piece, pan a board, reorder | +| `pinch` | two-finger pinch | ctrl+wheel / trackpad pinch | zoom a map / scene | +| `swipe` | flick | wheel / two-finger | next/prev, dismiss, scroll | +| `pointerMove` *(opt-in)* | finger track | hover | drawing, aiming — `continuous-input` | + +Rules that make it kiosk-grade: + +- **Hit-targets, not pixels.** A Lumen declares interactive scene elements + with a **minimum 44×44 pt hit area** (Apple HIG), independent of the drawn + glyph size. The runtime enforces the minimum and does hit-testing against + stable element ids → the existing `TargetRef`/beam model works by touch. +- **Gesture arbitration is the host's job**, reusing `CONCEPT.md`'s long-press + arbitration (move > 6 px before 400 ms ⇒ drag, else context-invoke). A Lumen + never re-implements gesture disambiguation. +- **No hover dependency.** Hover is dropped as a *required* affordance (not + touch-capable); any hover effect is pure decoration with a tap/long-press + equivalent. This matches `CONCEPT.md`'s interaction model. +- **Touch-tuned density.** `style: "spacious"` and larger Lume hit-areas are + the kiosk default; the agent is told (UI Skill) to compose touch-first when + the canvas is flagged as a kiosk/tablet surface. +- **On-screen input.** Text entry uses the platform soft-keyboard via the + `input` primitive; a kiosk with no keyboard still works. No raw key events + are *required* — declared keys (`ArrowLeft`, `Space`) are an *enhancement* + for hardware-keyboard hosts, and every key-driven action has a touch + equivalent (on-screen control) when the host reports no keyboard at + handshake. + +The handshake already carries client capabilities; it is extended to report +**input modalities** (`touch` / `mouse` / `keyboard` / `pen`) so Tier 2 +composes the right affordances — a kiosk Lumen ships on-screen controls, a +desktop Lumen may add keyboard shortcuts on top. + ### Thin-client / kiosk capability ladder Because the client only ever does *bounded interpretation* + *raster of a @@ -371,6 +436,7 @@ in the Lumen, effect-classified, and brokered by Tier 2. Default deny. | `tiles(provider, z/x/y)` | `internal` | Tier 2/3 fetches from a **provider-allowlisted** endpoint, returns sprite `DataRef`s | OpenStreetMap / Mapbox map tiles | | `fetch(declaredEndpoint)` | `internal` / `external-effect` | Tier 3 tool call against an **allowlisted, agent-approved** endpoint only | a live feed into a visualisation | | `clipboard(text)` | `external-effect` | confirmation-modal gate | "copy result" | +| `generateAsset(spec)` | `internal` / `external-effect` | Tier 2 → **omadia-core LLM connectors** (Tier 3); returns a `DataRef` | generate an image / sound / TTS voice line for the scene | | `share(lumen)` / `savePreset(lumen)` | `external-effect` | §7 | share Tetris with a colleague; save as a gallery preset | Mechanics: a capability call from the running Lumen is *not* a direct call. It @@ -387,6 +453,69 @@ This is the precise inversion of Live Artifacts: the network is *not* banned, it is **named, allowlisted, agent-brokered, user-confirmed, and audited in Trace**. +### 6.1 Assets — transport, content-addressed caching, never-stale + +Images, sounds, video, voice lines and any other binary an asset-bearing +Lumen references all travel as **`DataRef`s** — the canonical mechanism from +`CONCEPT.md` §"DataRef lifecycle". The key property the user asked for — +**no stale-cache problem** — falls out of `DataRef` being **content-addressed**: + +> `id = "-"` — `kind ∈ {pixel, audio, video, …}`. +> **Same bytes → same id. Different bytes → different id. Always.** + +This is **cache-busting by construction**, the structural fix for the +"annoying browser still shows the old image because it didn't notice it +changed" behaviour: the id *is* the content hash, so a changed asset is a +**different id** and there is no way to address new content with an old +reference. The client cache is keyed by that hash, so a cache hit is a +*provable* byte-identity, never a heuristic on a URL + `Cache-Control` guess. + +**Transport path (host → UI client):** + +1. **Origin.** A Tier-3 tool or an omadia-core LLM connector (see §6.2) + produces the binary and returns a `DataRef {id, signedToken, expiresAt}` — + the binary itself stays server-side until fetched. +2. **Announce.** Tier 2 emits `surface_data_ref_created {DataRef, schema, + sizeHint}` on the surface stream; the Lumen's `view` references the asset + by `dataRef` on a `sprite`/`image`/`media` node. +3. **Fetch.** The client fetches the bytes **once** from the channel endpoint + using the **HMAC-signed token** (scope = tenant ‖ user ‖ canvasSession ‖ + body ‖ expiry; re-validated server-side — `CONCEPT.md` §"Security Surface"). +4. **Cache.** The client stores the bytes in a local content-addressed store + keyed by `id`. Every later reference to the same `id` is an instant local + hit — across turns, across Lumens, across canvases. Dedup is automatic + (identical assets share one entry). +5. **Invalidate.** Two triggers, both explicit: `expiresAt` reached, or a + `surface_data_ref_invalidated {id, reason}` from Tier 2 when a durable op + replaces the buffer. There is no time-based "maybe it's stale" guesswork. +6. **GC.** The client drops a local buffer once no live primitive references + its `id` **and** its expiry has passed. + +Large client-authored buffers (a `canvas-region` the user painted) are +content-hashed **locally** and only uploaded if a Tier-3 op needs them — the +same content-addressing in the other direction. + +### 6.2 Generated material comes from omadia-core, not from the Lumen + +Generative assets — images, sounds, music, **synthesised voice** — are +produced by the **LLM connectors wired into the omadia host (omadia-core)**, +**never** by the Lumen or the Tier-1 client. The UI side only *supports* them: +it **requests** via the `generateAsset` capability and **renders** the +returned `DataRef`. The division of labour: + +| Layer | Role in asset generation | +|---|---| +| **Lumen / Tier 1** | declares the need (`generateAsset(spec)`), renders the resulting `sprite`/`media`, runs declarative motion (Ken-Burns, etc.) on it. Generates **nothing**. | +| **Tier 2** | validates the capability grant, shapes the request, brokers it to the right connector, caches the returned `DataRef`, patches it into the scene. | +| **Tier 3 / omadia-core connectors** | the actual image/audio/voice model. Owns the generation, the model choice, the cost, the rate limits. Returns a content-addressed `DataRef`. | + +This keeps the security model intact (no model keys or generation logic on the +client), keeps the protocol model-agnostic (swap connectors without touching +the Lumen), and means generated material flows through the **exact same +content-addressed, never-stale cache** as any other asset (§6.1). A +regenerated image is simply a **new `id`** — the scene updates, the old bytes +GC out, nothing stale lingers. + --- ## 7. Sharing & presets — safe because it's data diff --git a/docs/mockups/kiosk-lumen-aura.html b/docs/mockups/kiosk-lumen-aura.html new file mode 100644 index 0000000..4aaa043 --- /dev/null +++ b/docs/mockups/kiosk-lumen-aura.html @@ -0,0 +1,393 @@ + + + + + +Omadia UI — Kiosk Lumen mockup (Lume · Lagoon) + + + + + + + +
+
+ + +
+
+
+
+
+
+ +
+
+
+
New · 2026
+

AURA Halo

+

A speaker that lights the room it fills. Spatial sound, condensed out of light.

+
+
+ + +
+
+
AURA Studio
+
--:--:--
+
+ +

Tap a feature to explore. Everything you see was composed by the agent — layout, motion and copy — and rendered natively in Lume.

+ +
+ + + + +
+ +
+ + +
+
+
+ +
+ +
+ + + + From 770a6a11a729962459911769d5ab7c00a97701ae Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 17:27:57 +0000 Subject: [PATCH 04/13] =?UTF-8?q?docs:=20add=20Lumen=20lifecycle=20&=20reu?= =?UTF-8?q?se=20=E2=80=94=20author=20once,=20instantiate=20&=20patch=20(v0?= =?UTF-8?q?.4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/interactivity-concept.md | 147 ++++++++++++++++++++++++++++++++-- 1 file changed, 141 insertions(+), 6 deletions(-) diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index fce2210..86ea435 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -14,6 +14,15 @@ > (the *why*). This document is **concept only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. +Version 0.4 — adds §8 "Lumen lifecycle & reuse — author once, then instantiate +& patch (never rebuild)": the Omadia counterpart to Claude's artifact logic. +Author once (`surface_snapshot`), persist in canvas-state, edit by **targeted +`surface_patch`** (not regeneration, not string-replace); a **preset library** +(first-party / tenant / user / canvas scopes, content-addressed, versioned, +parameterised) so reuse is **instantiate-without-an-LLM**; a **two-speed** +model (rare cold authoring vs. constant warm reuse); **resolve-then-generate** +lookup before any build; **fork+patch** with lineage for variations; a +behaviour-idiom library so even cold builds assemble vetted fragments. Version 0.3 — review fixes. **Lume correctness**: the motion/effect vocabulary is now exact to `visual-spec.md` — Lume is **light-as-material** (surface luminosity, accent-as-illumination, directional borders, soft corners), *not* @@ -547,7 +556,132 @@ because they are validated, deterministic, capability-declared data, not code. --- -## 8. The agent relationship — generation *and* live introspection +## 8. Lumen lifecycle & reuse — author once, then instantiate & patch (never rebuild) + +Regenerating a kiosk surface or a Tetris from scratch on every turn would be +expensive (tokens), slow (a full generation), and inconsistent (two "builds" +drift). It must not happen, and the architecture already has the pieces to +stop it. This is the Omadia counterpart to Claude's artifact logic +(create-once, then `update`) — extended from "one artifact per conversation" +to a **persistent, versioned, shareable component library**. + +### 8.1 Author once, edit by patch — never regenerate + +A Lumen is generated **once** (`surface_snapshot`) and then **lives in +canvas-state** (`CONCEPT.md` §"State Model"), persisting across turns and Host +App restarts. The agent does **not** re-emit it to keep it alive. Subsequent +changes are **targeted `surface_patch`es** addressed by stable id/path, +bumping `treeRevision`: + +| User says | What the agent emits | Cost | +|---|---|---| +| "Build me Tetris" *(nothing reusable exists)* | one `surface_snapshot` with the full Lumen | one expensive turn (once) | +| "Make it faster as the score climbs" | `surface_patch` touching only the `tick` transition | a few tokens | +| "Bigger board" | `surface_patch` on one `state` field + the `view` grid bounds | a few tokens | +| "Change the accent to Atelier" | accent re-tint patch (existing palette mechanic) | trivial | + +This is exactly Claude's create-then-update model — but a patch targets a +**validated tree by stable id**, not a string-replace on source code, so there +is no resync/drift surface and no risk of an edit corrupting the program. The +*structure is preserved by construction* across edits → consistency for free. + +### 8.2 The reuse library — presets & templates (the real "not from scratch") + +The durable answer is the **preset library** (§7). A vetted Lumen is authored +**once** and saved as a preset: **named, versioned, content-addressed, +parameterised**. Thereafter the agent does not build it — it **instantiates** +it: + +> "Gib mir Tetris" → Tier 2 resolves the intent to the `tetris@2` preset, +> binds its parameters (board size, palette, data), and renders it. +> **Near-zero LLM, sub-second, byte-identical every time.** + +Library scopes, outermost to innermost (first match wins, like CSS cascade): + +| Scope | Namespace | Holds | +|---|---|---| +| **First-party** | shipped with the orchestrator | curated, audited Lumens (Tetris, map, defrag-viz, standup board) | +| **Tenant / org** | `lumen-presets//shared/**` | the organisation's vetted library | +| **User** | `lumen-presets///**` | "my defrag-style project viewer" | +| **Canvas-local** | canvas-state | the instance currently on screen | + +Because a preset is **content-addressed** (`preset-`) and +**deterministic**, every instantiation is identical — there is no +"two-divergent-Tetris" problem, and a shared preset replays the same on every +machine (§2 determinism, §6.1 content-addressing). + +### 8.3 Two-speed generation — the cost & consistency model + +| Mode | When | Model | Latency | Frequency | +|---|---|---|---|---| +| **Cold authoring** | nothing in the library fits | strong model (Sonnet/Opus) | seconds | **rare** — once per genuinely new idea | +| **Warm reuse** | a preset matches (exactly or after a small edit) | none, or a fast model for a parametric patch | sub-second | **the common case** | + +After a cold authoring, Tier 2 **offers to save the result as a preset** +("Save this Tetris to your gallery?"). The expensive build is paid **once**; +every later use is warm. This is the same economics that makes Claude +artifacts feel cheap after creation — generalised into a persistent library +instead of a per-conversation artifact. + +### 8.4 Resolve-then-generate — the lookup before the build + +Before generating anything, Tier 2 runs a **library lookup** — the *same +shape* as the existing "check the per-canvas data cache before calling Tier 3" +rule in `CONCEPT.md` §"Per-canvas data cache": + +``` +intent ──▶ library lookup (semantic + capability match across scopes) + ├─ exact hit ──▶ instantiate preset (no LLM) + ├─ near hit ──▶ fork preset + targeted patch (fast model) + └─ miss ──▶ cold-author + offer to save (strong model) +``` + +So "build me X" is, by default, a **retrieve-and-bind** operation, not a +generation. Generation is the fallback, not the first move. + +### 8.5 Fork & vary — copy-on-write, with lineage + +"Tetris, but hexagonal" does **not** rebuild Tetris. Tier 2 **forks** the +`tetris` preset (copy-on-write → new content-addressed id, parent id recorded +for provenance) and applies a **targeted patch** to the affected +transitions/view. Cheap, consistent with the original, and the lineage is +auditable — important for the trust model when forks are shared (§2 point 5). + +### 8.6 Compose from idioms, not from zero + +Even cold authoring is not truly from scratch. The UI Skill carries a +**behaviour-idiom library** — the behaviour-layer extension of the +composition-idiom library already in `CONCEPT.md` §"The UI Skill" (Norton +Commander → two panes, etc.). Vetted building blocks the agent assembles: + +- a **scene-grid** skeleton (board/cell-matrix render), +- a **tick-loop** skeleton (gravity / animation step), +- **input-binding sets** (WASD/arrows/touch-swipe → moves), +- **Lume effect bundles** (a glow-pulse, a Ken-Burns hero, a condensation-in). + +Cold builds therefore assemble audited LX fragments rather than inventing +control flow each time — cheaper to generate **and** more consistent across +builds. This is the single biggest lever on both cost and reliability of cold +authoring. + +### 8.7 vs. Claude artifact logic + +| | Claude artifacts | Omadia Lumens | +|---|---|---| +| Persistence | per-conversation | cross-session canvas-state + **cross-user preset library** | +| Edit mechanism | string `update` / `rewrite` on source | **structural patch on a validated tree by stable id** (no drift) | +| Reuse | copy text into a new chat | **named, versioned, parameterised preset** — instantiate without an LLM | +| Consistency | re-generation can drift | content-addressed + deterministic ⇒ byte-identical | +| Sharing | share the code | share a **vetted, capability-manifested** preset (§7) | +| Variation | re-prompt | **fork + patch** with tracked lineage | + +The thesis: **the agent's job is to author rarely and reuse constantly.** A +Lumen is a durable, versioned component — built once, instantiated and patched +forever after. + +--- + +## 9. The agent relationship — generation *and* live introspection Lumens are generated agentically (Tier 2 composes the Lumen the way it composes primitive trees today; heavy generation or data binding can recruit @@ -571,7 +705,7 @@ Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: --- -## 9. The user's four use cases, mapped +## 10. The user's four use cases, mapped | Use case | state | view | events | capabilities | Tier split | |---|---|---|---|---|---| @@ -586,7 +720,7 @@ in the Lumen model. --- -## 10. How it fits the existing architecture (deltas, not rewrites) +## 11. How it fits the existing architecture (deltas, not rewrites) Everything below is **additive** and stays inside the `CONCEPT.md` tier model, authority split, and security surface. No wire-grammar rewrite. @@ -597,6 +731,7 @@ authority split, and security surface. No wire-grammar rewrite. | **Tree content** | add the `behavior`/`lumen` section (state/transitions/view/events/capabilities) — validated by an **extended whitelist parser** (schema + LX-AST validator) | | **Tier-1 client** | new **Lumen runtime**: deterministic LX evaluator, gas + frame ceiling, scene rasteriser, event dispatch, seeded `random`/clock. All Class-A; the frame loop never touches the server | | **Tier-2 orchestrator** | composes/patches Lumens; **brokers capability calls**; grants/scopes capability manifests; persists Lumen state & presets; manages share/ownership | +| **Lifecycle / reuse** (§8) | **resolve-then-generate**: library lookup before any build (same shape as the existing pre-Tier-3 data-cache check); preset **instantiate** (no LLM) / **fork+patch** (fast model) / **cold-author** (strong model); content-addressed, versioned, parameterised presets; behaviour-idiom library in the UI Skill | | **Tier-3** | reached only via brokered capabilities (`tiles`, `fetch`, `writeData`, AI ops) — unchanged interface | | **Transport** | Lumen rides existing `surface_snapshot`/`surface_patch`; capability calls reuse the effect-classified action path + `surface_action_result`/patch; **one** new optional event family `surface_capability_*` if streaming results need it | | **Security** | extend whitelist parser to LX-AST + scene + capability manifest; reuse HMAC scoping for capability tokens; reuse `local`/`internal`/`external-effect` + confirmation modal | @@ -609,7 +744,7 @@ engaged only behind the `canvas` capability. --- -## 11. The sweet-spot dial (answering the user's central ask) +## 12. The sweet-spot dial (answering the user's central ask) The user's framing: Live Artifacts errs too far toward restriction; find the better-tuned point between *possibility* and *safety*. The Lumen model tunes @@ -632,7 +767,7 @@ dangerous one, open the useful one.** --- -## 12. Open questions for the spike (flagged, not answered) +## 13. Open questions for the spike (flagged, not answered) 1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, scene draw-list length, tick rate — measured against the four reference @@ -661,7 +796,7 @@ dangerous one, open the useful one.** --- -## 13. What this is not (scope discipline) +## 14. What this is not (scope discipline) - **Not arbitrary code execution.** No `eval`, no iframe-with-script, no WASM in v1. If a use case truly needs Turing-complete compute, it is a flagged From fde404df78aa26cce989f0a82302ef9cc67cf368 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 17:35:04 +0000 Subject: [PATCH 05/13] =?UTF-8?q?docs:=20composition=20&=20cross-element?= =?UTF-8?q?=20interaction=20=E2=80=94=20ports=20&=20wires=20(v0.5)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/interactivity-concept.md | 70 +++++++++++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index 86ea435..839b6d9 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -14,6 +14,13 @@ > (the *why*). This document is **concept only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. +Version 0.5 — expands §9 with §9.1 "Composition & cross-element interaction": +a Lumen is just a node in the same primitive tree (composes in +container/pane/grid/tabs; a Lumen's `view` can hold primitives and vice +versa). Elements interact **bidirectionally and deterministically on Tier 1** +via shared `viewState.selection` by stable id and declarative **ports & wires** +(typed inputs/outputs, `from`→`to` bindings), with a least-privilege boundary +(a node reads only what is wired to it). Authority split unchanged. Version 0.4 — adds §8 "Lumen lifecycle & reuse — author once, then instantiate & patch (never rebuild)": the Omadia counterpart to Claude's artifact logic. Author once (`surface_snapshot`), persist in canvas-state, edit by **targeted @@ -698,10 +705,64 @@ Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: - **Beam into a Lumen.** A user can beam a scene element ("why is this block red?") or a region; the existing `TargetRef`/beam machinery resolves inside the Lumen. -- **Composability on one canvas.** A Lumen is ordinary tree content, so a map - Lumen sits next to a Jira `table` and they can be wired (select a marker → - filter the table) through normal canvas mechanics. Live Artifacts are - isolated islands; Lumens are first-class canvas citizens. +- **Composability on one canvas.** A Lumen is ordinary tree content (see §9.1), + so a map Lumen sits next to a Jira `table` and they interact through normal + canvas mechanics. Live Artifacts are isolated islands; Lumens are first-class + canvas citizens. + +### 9.1 Composition & cross-element interaction (it is just one tree) + +A Lumen is a node in the same primitive tree as every other primitive. There +is **no separate surface, no iframe, no island**. Two consequences: + +**(a) It composes like any primitive.** A Lumen lives inside `container` / +`pane` / `grid` / `tabs`, obeys the same layout, the same surface-nesting +ladder, the same Lume material, the same validator and the same render pass. +The agent places a map Lumen in one pane, a Jira `table` in another, a `chart` +below — one canvas, one tree. The boundary is porous *both ways*: a Lumen's +`view` itself emits ordinary primitives (a `button`, a `table`, a `status`), +so a Lumen can contain regular UI and regular UI can contain a Lumen. + +**(b) Elements interact — bidirectionally, deterministically, on Tier 1.** +The wiring reuses machinery that already exists, so it costs **no turn** and +runs at 60 fps: + +| Mechanism | How it wires | Direction | Tier | +|---|---|---|---| +| **Shared selection / view-state** | a `table` and a Lumen both reference the same `DataRef` + **stable IDs**; selection lives in `viewState.selection` keyed by those IDs (`CONCEPT.md` Authority Model) | UI ⇄ Lumen | **1 (Class A)** — no server | +| **Declarative ports & wires** | a node declares typed **inputs** and **outputs**; the agent declares **wires** (`from` → `to`) by stable id. The host routes values at Tier 1 | UI ⇄ Lumen, Lumen ⇄ Lumen | **1 (Class A)** | +| **Lumen capability → patch** | a Lumen output bound to `writeData`/`generateAsset`/etc. patches another container via Tier 2 | Lumen → UI (semantic) | 2 / 3 | +| **Beam / agent** | user beams across both ("compare these rows with the map"); agent reasons over the wired set | either, semantic | 2 | + +Concrete: + +- **Map Lumen ⇄ Jira `table`.** Select rows in the table → the bound markers + on the map glow (shared selection by `rowKey`). Tap a marker → the table + filters/scrolls to that `rowKey` (Lumen output wired to the table's + view-state). Both directions are pure Tier 1. +- **`form`/slider → simulation Lumen.** A `choice`/slider primitive is wired to + a Lumen `state` input (gravity, speed, grid size). Dragging the slider + retunes the running sim live — declarative, no turn. +- **Lumen output → primitive.** Tetris `game-over` (a Lumen output) wired to a + `status` primitive ("New high score!") and to a `writeData` capability that + persists the score back through Tier 3. +- **List ⇄ defrag-viz Lumen.** Hover/select a file in a `list` → its blocks + light up in the scene; click a block cluster → the list scrolls to that file. + +**The boundary that keeps it safe.** Cross-element wiring is **declared data** +(validated by the whitelist parser), and a node can only read what is **wired +to it** — it cannot reach arbitrary other elements' internals. This is +least-privilege by construction: the same port/wire boundary that makes a +Lumen safe to share makes cross-element interaction auditable and +shared-canvas-safe (wires resolve by stable id, deterministically, so they +replay and multicast unchanged). The **authority split is unchanged**: the +agent owns *which elements and wires exist* (structure); the client owns *the +values currently flowing through them* (view-state). + +> **SDK delta:** declarative **`ports`** (typed inputs/outputs) on primitives +> and Lumens, and **`wires`** (a `[{from: TargetRef+port, to: TargetRef+port}]` +> list) at the container/canvas level — additive tree content, Tier-1-resolved, +> whitelist-validated. No new transport. --- @@ -729,6 +790,7 @@ authority split, and security surface. No wire-grammar rewrite. |---|---| | **Primitives** | add `scene` (editor-class) — protocol minor bump (`1.x`); draw-list is whitelisted shape data | | **Tree content** | add the `behavior`/`lumen` section (state/transitions/view/events/capabilities) — validated by an **extended whitelist parser** (schema + LX-AST validator) | +| **Composition / interaction** (§9.1) | declarative **`ports`** (typed inputs/outputs) on primitives & Lumens + **`wires`** (`from`→`to` by `TargetRef`) at container/canvas level; Tier-1-resolved by stable id, whitelist-validated; reuses shared `viewState.selection`. No new transport | | **Tier-1 client** | new **Lumen runtime**: deterministic LX evaluator, gas + frame ceiling, scene rasteriser, event dispatch, seeded `random`/clock. All Class-A; the frame loop never touches the server | | **Tier-2 orchestrator** | composes/patches Lumens; **brokers capability calls**; grants/scopes capability manifests; persists Lumen state & presets; manages share/ownership | | **Lifecycle / reuse** (§8) | **resolve-then-generate**: library lookup before any build (same shape as the existing pre-Tier-3 data-cache check); preset **instantiate** (no LLM) / **fork+patch** (fast model) / **cold-author** (strong model); content-addressed, versioned, parameterised presets; behaviour-idiom library in the UI Skill | From e36d1eb8868a56c45acd5635e329b4cab3d1b9d8 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 17:43:55 +0000 Subject: [PATCH 06/13] docs: add normative Lumens definition spec; integrate into CONCEPT, visual-spec, implementation-plan, README --- CONCEPT.md | 34 +++ README.md | 11 +- docs/implementation-plan.md | 27 ++ docs/interactivity-concept.md | 9 +- docs/lumens-spec.md | 486 ++++++++++++++++++++++++++++++++++ docs/visual-spec.md | 48 ++++ 6 files changed, 611 insertions(+), 4 deletions(-) create mode 100644 docs/lumens-spec.md diff --git a/CONCEPT.md b/CONCEPT.md index 67172ea..7f3dc6d 100644 --- a/CONCEPT.md +++ b/CONCEPT.md @@ -721,6 +721,7 @@ The system uses a small, disciplined set of light-domain names because Lume is t | **Trace** | the command history / audit log of a canvas — a "trace of light" through what the user asked and the agent answered | introduced v0.12 | | **Flare** | reserved — for agent-initiated attention signals (e.g. notifications, "needs your input") | reserved, may activate if and when we introduce notification concepts | | **Spark** | reserved — for a discrete generative-initiation event distinguishable from a beam | reserved, may activate if a use case clearly differs from Beam | +| **Lumen** | a self-contained, declarative, deterministic interactive unit — UI condensed into a portable, shareable quantum of light (the Live-Interactivity extension) | **proposed**, pending vocabulary sign-off — see [`docs/interactivity-concept.md`](docs/interactivity-concept.md) + [`docs/lumens-spec.md`](docs/lumens-spec.md) | Anything else from the light domain (Photon, Shadow, Beacon, Cast, Lens, Prism, Reflection, Ray, Shine, Glimmer, …) does **not** enter the vocabulary unless it earns its slot by labelling something that has no good name yet. The default answer to "should we call this X?" is no. @@ -1200,6 +1201,39 @@ This section becomes the input for the v2 design phase. It is not a v1 deliverab --- +## Extension — Live Interactivity (Lumens) + +A planned **additive** extension brings rich, agent-generated, **Tier-1-fast** +interactivity to the canvas — games (Tetris), interactive data workflows, +unusual visualisations (defrag-style), live maps — the Omadia answer to +sandbox-style "live artifacts", deliberately re-aimed. Its thesis: most of what +arbitrary-code sandboxes block is blocked by missing **capabilities**, not +missing **compute** — so it **constrains computation** (a declarative, bounded, +deterministic, interpreted behaviour model — no arbitrary code, the whitelist +parser extends to it) and **opens capabilities** (real data, write-back, +allowlisted network, generated assets) **mediated** through the existing Tier-2/3 +orchestration and effect classification. + +The unit is a **Lumen**: `state + transitions + view + events + capabilities`, +plus a new `scene` primitive (a declarative draw surface), declarative +ports/wires for cross-element interaction, per-region render cadence, and a +preset library so the agent **authors once and reuses constantly** (not rebuilt +per turn). It is **forward-compatible** — Lumens ride the same `surface_*` +grammar, the same DataRef/HMAC scoping, the same authority split, the same +shared-canvas hooks; the deltas are additive (a `behavior` tree section, the +`scene` primitive → `omadia-canvas-protocol/1.1`, one optional +`surface_capability_*` event family). + +- **Rationale / concept:** [`docs/interactivity-concept.md`](docs/interactivity-concept.md) +- **Normative definition / spec:** [`docs/lumens-spec.md`](docs/lumens-spec.md) +- **Lume visual treatment:** [`docs/visual-spec.md`](docs/visual-spec.md) §4.13 +- **Reference mockup:** [`docs/mockups/kiosk-lumen-aura.html`](docs/mockups/kiosk-lumen-aura.html) + +Not a v1 deliverable of this base concept; tracked as its own implementation +workstream. + +--- + ## Riskiest Assumptions 1. **Top-tier and fast LLMs can reliably emit valid primitive trees as tool-use JSON.** Likely yes for Sonnet/Opus; unproven for Haiku-class on UI-tree synthesis (omadia uses Haiku only as classifier today). Mitigation: `ui_orchestrator_model` configurable; spike with Haiku first, fall back to Sonnet if reliability below threshold. diff --git a/README.md b/README.md index 9032bbd..58abe75 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,13 @@ Omadia UI is the next layer — a desktop application where the agent - 🗂️ **A real workspace.** Multiple live canvases with their own server sessions, tiling split panes, named desktops, a canvas library — all synced through the omadia registry and restored on restart. +- 🎮 **Live, safe interactivity (Lumens).** A planned additive extension lets + the agent generate self-contained interactive units — a game, an interactive + workflow, an unusual visualisation, a live map — as **declarative, + deterministic data** run by a bounded Tier-1 interpreter (no arbitrary code), + with capabilities mediated through Tiers 2/3. Shareable, presettable. See + [`docs/interactivity-concept.md`](docs/interactivity-concept.md) + + [`docs/lumens-spec.md`](docs/lumens-spec.md). ## ⚡ Quickstart @@ -82,7 +89,9 @@ refactor). | File | Purpose | |---|---| | [`CONCEPT.md`](CONCEPT.md) | Architecture, primitives, protocol, security, identity, SDK extension plan | -| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.4) | +| [`docs/interactivity-concept.md`](docs/interactivity-concept.md) | **Live Interactivity (Lumens)** — concept/rationale: Tier-1-fast, agent-generated, safe interactivity (games, workflows, maps) | +| [`docs/lumens-spec.md`](docs/lumens-spec.md) | **Lumens** — normative definition: LX, `scene`, events/touch, capabilities, ports/wires, presets (`omadia-canvas-protocol/1.1` draft) | +| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.5) | | [`docs/protocol/1.0.md`](docs/protocol/1.0.md) | Protocol specification + machine-validatable JSON Schemas | | [`docs/walkthroughs.md`](docs/walkthroughs.md) | Use-case walkthroughs — multi-source comparison + editor micro-task | | [`docs/tech-stack.md`](docs/tech-stack.md) | Tech-stack decision for the host app (Electron), with reasoning | diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 446d92a..cb16d8f 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -699,6 +699,33 @@ tied to the phase that exposes each one. --- +## 10. Lumens (Live Interactivity) extension — implementation outline + +A separable, **additive** workstream on top of the v1 baseline above (it +depends on the surface event family, the validator, and the orchestrator +already landing). Concept: [`interactivity-concept.md`](interactivity-concept.md); +normative definition: [`lumens-spec.md`](lumens-spec.md). Tracked for the +maintainer as a GitHub issue. Suggested phasing, smallest mergeable units first: + +| Phase | Deliverable | Where | +|---|---|---| +| **L0 — schemas** | JSON Schemas for the Lumen `behavior` section, the LX-AST node set, `scene`, `ports`/`wires`, the capability manifest; accept/reject fixtures | `docs/protocol/schema/` (this repo) → `omadia-canvas-protocol/1.1` | +| **L1 — LX interpreter (Tier 1)** | deterministic AST evaluator with gas + frame ceiling, bounded iteration, seeded `random`/`now`; the extended whitelist validator | host app `app/src/renderer/` | +| **L2 — `scene` primitive** | draw-list rasteriser (canvas2d first, WebGL behind it), token-only styling, buffer-native hit-testing → `TargetRef` | host app renderer | +| **L3 — cadence & animation** | per-region `static`/`reactive`/`{tick}` dirty-tracking + rAF scheduling; declarative `animate` layer on the Lume effect vocabulary; reduced-motion | host app renderer | +| **L4 — events & touch** | `tap`/`longPress`/`drag`/`pinch`/`swipe`/`key`/`tick`; 44 pt hit-targets; host gesture arbitration; input-modality handshake fields | host app + channel | +| **L5 — capabilities broker (Tier 2)** | `persist`/`loadData`/`writeData`/`tiles`/`fetch`/`generateAsset`/`clipboard`; effect-classified brokering + confirmation gate; `surface_capability_*` events; asset transport + content-addressed cache | `omadia-ui-orchestrator` + `omadia-ui-channel` + `byte5ai/omadia` core connectors for `generateAsset` | +| **L6 — ports & wires** | typed ports on primitives/Lumens, Tier-1 wire resolution, shared `viewState.selection` cross-element | host app + orchestrator | +| **L7 — lifecycle & presets** | author-once/patch; `lumen-presets/**` + `lumen-state/**` stores; resolve-then-generate lookup; fork+patch; behaviour-idiom library in the UI Skill | `omadia-ui-orchestrator` | +| **L8 — sharing** | `canvasOwnership` group extension + channel fan-out + import consent (rides the v2 shared-canvas hooks) | channel + orchestrator | +| **L9 — reference Lumens** | Tetris · interactive workflow · defrag-viz · map, traced end-to-end like `walkthroughs.md`; conformance fixtures | this repo | + +**Riskiest items** (mirror `interactivity-concept.md` §13): LLM reliability +emitting valid LX (likely a strong-model authoring job, fast-model patching); +gas/scene-perf calibration on the four reference Lumens; capability-consent UX. + +--- + ## Appendix — corrected critical-file map (omadia core @ `83ef79b`) The concept's "Critical files" section with line numbers refreshed against live diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index 839b6d9..a3d8ca7 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -9,9 +9,12 @@ > re-aimed: more capable where Live Artifacts is uselessly limited, and > *structurally* safer than "arbitrary code in a sandbox". > -> Companion to `CONCEPT.md` (the *what* of the canvas), `visual-spec.md` -> (Lume), `docs/protocol/1.0.md` (the wire format) and `design-rationale.md` -> (the *why*). This document is **concept only** — no implementation, no PR +> This is the **rationale** (the *why*, narrative); its **normative +> definition** (the *what* — types, grammar, contracts) is the companion +> [`lumens-spec.md`](lumens-spec.md). Further companions: `CONCEPT.md` (the +> canvas), `visual-spec.md` §4.13 (Lume rendering of Lumens), +> `docs/protocol/1.0.md` (the base wire format). This document is **concept +> only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. Version 0.5 — expands §9 with §9.1 "Composition & cross-element interaction": diff --git a/docs/lumens-spec.md b/docs/lumens-spec.md new file mode 100644 index 0000000..fbe2113 --- /dev/null +++ b/docs/lumens-spec.md @@ -0,0 +1,486 @@ +# omadia-canvas-protocol — Lumens (Live Interactivity) · draft 1.1 + +> **The normative definition** of the Live-Interactivity extension. Where +> [`interactivity-concept.md`](interactivity-concept.md) holds the *rationale* +> (the *why*, narrative), this document holds the *definition* (the *what*, +> normative) — the types, the grammar, the contracts a renderer and an +> orchestrator must implement. It is the Lumen counterpart to +> [`protocol/1.0.md`](protocol/1.0.md) and a companion to +> [`visual-spec.md`](visual-spec.md) (Lume) and [`../CONCEPT.md`](../CONCEPT.md) +> (the canvas architecture). + +**Status:** `draft 1.1`. Additive, **minor** bump over +`omadia-canvas-protocol/1.0` — old clients ignore unknown primitive types, +tree sections, events and capabilities (`protocol/1.0.md` §0). Nothing here +breaks the 1.0 wire grammar. The machine-validatable truth will live in +`schema/` (Lumen, LX-AST, scene, ports/wires, capability manifest) and is a +spike deliverable; where prose and schema disagree, the **schema wins**. + +A Lumen is the Omadia answer to "an interactive artifact": **declarative data, +not code**, run by a small deterministic interpreter on Tier 1, generated and +brokered agentically on Tiers 2/3, safe to share and to save as a preset +*because* it is data. See `interactivity-concept.md` §0 for the thesis (most of +what sandbox-artifacts block is blocked by missing *capabilities*, not missing +*compute* — so we constrain compute and open capabilities, mediated). + +--- + +## 0. Conventions & non-negotiable constraints + +Inherited from `protocol/1.0.md` §0 (two-axis versioning, opaque `RevisionId`, +stable IDs as the lingua franca, one JSON value per frame) plus: + +1. **No arbitrary code, ever.** A Lumen carries *data* — a typed state schema, + validated expression ASTs, a view template, event bindings, a capability + manifest. The Tier-1 runtime is an **interpreter shipped in the Host App**, + never `eval`/`Function()`. CSP stays `default-src 'self'`, no `unsafe-eval`. + The whitelist-parser discipline of 1.0 **extends** to LX-AST, scene shapes, + ports and capabilities: any unknown node/type → hard reject (`surface_error`). +2. **Bounded & total.** Every transition / view evaluation runs under a gas + budget and a wall-clock ceiling; iteration is bounded (no open `while`, no + general recursion); state is size-capped. A Lumen can never hang the host — + exceeding budget halts it with `surface_error`. +3. **Deterministic.** All non-determinism is host-seeded (`random`, `now`, + tick). `(state, event) → state` is identical on every machine — the basis + for replay, undo, safe sharing and v2 multi-user. +4. **Default-deny capabilities.** A Lumen reaches nothing outside its own state + except through declared, granted, effect-classified capabilities brokered by + Tier 2 (`CONCEPT.md` §"Security Surface" effect classes). +5. **Authority split unchanged.** The agent owns *structure* (which Lumens, + elements, wires, capability grants exist). The client owns *view-state* (the + values flowing through them, current selection, scroll). Stable IDs bind the + two (`CONCEPT.md` §"Authority Model"). +6. **Lume is the material.** Lumens render in Lume — light-as-material, **not** + glassmorphism (`visual-spec.md` §1.3). §10. + +--- + +## 1. Definition — the Lumen + +A **Lumen** is a self-contained interactive unit on the canvas. It is delivered +as **tree content** (a node, or a `behavior` section attached to a `container`) +inside an ordinary `surface_snapshot` / `surface_patch` — **no new transport**. + +```ts +type Lumen = { + id: string; // stable; patches/wires/beam target it + state: StateSchema; // §1.1 — typed, bounded, serialisable + transitions: Record; // §2 — pure (state,event)->state + view: LXNode; // §2,§3 — pure state -> primitive/scene tree + events: EventBinding[]; // §4 — declared inputs -> transitions + cadence?: CadenceSpec; // §5 — default "reactive" + capabilities?: CapabilityRequest[]; // §6 — default-deny doors out + ports?: PortSpec[]; // §7 — typed inputs/outputs for wiring + preset?: PresetRef; // §8 — provenance if instantiated/forked +}; +``` + +A Lumen is valid iff: its `state` conforms to §1.1, every `LXNode` in +`transitions`/`view` passes the §2 AST whitelist + static bounds check, every +`EventBinding` names a declared transition and a §4 event, every +`CapabilityRequest` names a §6 catalog capability, and every `PortSpec` is +§7-typed. Any failure → the Lumen is rejected wholesale with `surface_error` +(scope = the Lumen `id`); it never partially renders. + +### 1.1 State schema + +`state` is a typed, **closed** record. Every leaf declares a type from the LX +value set (§2.1) and bounds: + +```ts +type StateSchema = { + [key: string]: + | { type: 'int' | 'number', min?: number, max?: number, init: number } + | { type: 'bool', init: boolean } + | { type: 'string', maxLength: number, init: string } + | { type: 'enum', values: string[], init: string } + | { type: 'list', of: StateLeaf, maxLen: number, init: unknown[] } + | { type: 'record', fields: StateSchema, init: object } + | { type: 'grid', w: number, h: number, of: StateLeaf, init?: unknown } // bounded 2D — boards, defrag cells + | { type: 'dataRef', init?: DataRef }; // §6.1 read-only projection handle +}; +``` + +Total serialised `state` size is capped (initial default **256 KB**, +spike-tunable). `state` persists in canvas-state (`CONCEPT.md` §"State Model"); +it is the *only* memory a Lumen has. + +--- + +## 2. Lume Expressions (LX) + +LX is the **pure, total** expression language of `transitions` and `view`. It is +delivered as a **JSON AST**, never as source text (no parser-injection surface; +the validator is a tree-walk, exactly like the primitive whitelist). + +### 2.1 Values + +`int`, `number`, `bool`, `string`, `list`, `record{…}`, plus the read-only +`state` and `event` bindings in scope. No closures over host objects, no `this`, +no prototypes, no functions-as-values beyond the named std-lib. + +### 2.2 AST node catalog (whitelist) + +| Node | Form | Meaning | +|---|---|---| +| `lit` | `{lit: value}` | literal | +| `state` | `{state: path}` | read a `state` slice (dotted path; `grid` via `{state, at:[x,y]}`) | +| `event` | `{event: field}` | read a field of the triggering event | +| `var` / `let` | `{let:{name:expr}, in:expr}` | bind a local; lexically scoped, immutable | +| arithmetic | `{"+":[a,b]}` `-` `*` `/` `mod` | numeric | +| comparison | `{">":[a,b]}` `>=` `<` `<=` `==` `!=` | boolean | +| logic | `{and:[…]}` `or` `not` | boolean | +| `if` | `{if:c, then:a, else:b}` | total conditional (both branches required) | +| `match` | `{match:expr, cases:[{when,then}], else}` | total switch | +| record/list ctor | `{record:{…}}` `{list:[…]}` | construction | +| `set` | `{set:{path: expr}}` | **functional** update → returns a new state (no mutation) | +| std-lib call | `{call:name, args:[…]}` | from the §2.3 whitelist only | + +### 2.3 Standard library (whitelist, bounded) + +`map` `filter` `fold` `range` `len` `min` `max` `clamp` `abs` `floor` `round` +`concat` `slice` `contains` `indexOf` `keys` `values` string ops (`upper` +`lower` `pad` `fmt`) and a small math set. **`map`/`filter`/`fold`/`range` +iterate only over collections bounded by `state`** (which is size-capped) — this +is what makes the gas bound a *static* property. **No `while`, no general +recursion.** `random()` and `now()` read host-seeded context values (§0.3). + +### 2.4 Gas & determinism contract + +- Each `transition`/`view` evaluation is metered (instruction count). Initial + default **50 000 gas / evaluation**, spike-tunable. Over budget → + `surface_error`, the Lumen is halted (not the canvas). +- A wall-clock ceiling per frame is a secondary guard. +- Given identical `(state, event, seed)` the result is byte-identical + everywhere. Renderers MUST NOT introduce ambient non-determinism. + +### 2.5 Validation + +A Lumen's LX is accepted iff every node is in §2.2, every `call` target is in +§2.3, every `state`/`event` path resolves against the declared schema, and a +static pass proves iteration bounds and a gas ceiling. `view` MUST return a +valid primitive/scene tree (§3); `transitions` MUST return a value conforming to +`state`. Anything else → reject. + +--- + +## 3. The `scene` primitive (editor-class, 1.1) + +`scene` is a declarative immediate-mode draw surface — the 25th primitive, an +editor-class addition (`protocol/1.0.md` §2). The Lumen `view` emits, per +render, a **draw-list** from a closed shape vocabulary. There is **no** canvas +`2d`/`webgl` script exposed to the agent. + +```ts +type Scene = { + type: 'scene', + id: string, + width: int, height: int, // buffer-native coordinate space + camera?: { x:number, y:number, zoom:number }, // pan/zoom; buffer-native + draw: SceneNode[], +}; + +type SceneNode = + | { kind:'rect', x,y,w,h, r?, fill?, stroke?, strokeW?, id? } + | { kind:'circle', cx,cy,r, fill?, stroke?, strokeW?, id? } + | { kind:'line', x1,y1,x2,y2, stroke, strokeW?, id? } + | { kind:'path', points:[number,number][], closed?, fill?, stroke?, id? } // reuses vector-path geometry + | { kind:'sprite', x,y,w,h, dataRef: DataRef, id? } // §6.1 — images, tiles, glyphs + | { kind:'text', x,y, text, size?, weight?, register?, fill?, id? } // Lume type registers + | { kind:'group', transform?, children: SceneNode[], id? }; +``` + +- **Colours/styles are theme tokens + the active Lume palette only** (`accent`, + `accent.glow*`, surface/text tokens, semantic state tokens). A `scene` is + always on-theme — a game still looks like Omadia. Free-form colours are + clipped by the Tier-1 normaliser. +- **Coordinates are buffer-native**, independent of zoom/pan (`CONCEPT.md` + `bufferRegion`). Hit-testing maps pointer → buffer coords → the `id` of the + topmost hit node. +- A `SceneNode.id` is a **stable element id** → it is a `TargetRef` + (`{kind:'element', elementId}`) for beams, events (§4) and wires (§7). +- Tier 1 rasterises the draw-list natively (canvas2d or WebGL) at up to 60 fps + from local state — pure **Class A**, zero server contact per frame. + +`scene` coexists with `canvas-region` (the *pixel-editor buffer* for +brush/blur Class-B ops); `scene` is a *state-driven render target*, not an +editable pixel buffer. + +--- + +## 4. Events & input (touch-first) + +`events` bind **pointer-semantic** inputs to transitions. They resolve +identically across mouse / trackpad / touch / pen — the same abstraction +`CONCEPT.md` uses for context-invoke. + +```ts +type EventBinding = { + on: 'tap'|'longPress'|'drag'|'pinch'|'swipe'|'pointerMove'|'key'|'tick'|'timer'|'wire', + target?: TargetRef, // a scene-node id, a primitive, or the Lumen (default) + key?: string, // for 'key' — declared keys only (e.g. 'ArrowLeft','Space') + rate?: number, // for 'tick' — declared, capped (≤60) — see §5 + everyMs?: number, // for 'timer' + run: TransitionName, // the transition to evaluate +}; +``` + +Rules (normative): + +- **44×44 pt minimum hit-target.** Interactive scene nodes declare/inherit a + ≥44 pt hit area regardless of drawn glyph size (Apple HIG); the runtime + enforces it. +- **Host owns gesture arbitration**, reusing `CONCEPT.md` long-press arbitration + (move >6 px before 400 ms ⇒ drag, else context-invoke). A Lumen never + re-implements disambiguation. +- **No hover dependency.** Hover is decoration only; every hover affordance has a + tap/long-press equivalent. +- **Declared keys are an enhancement.** Every key-bound action has a touch + equivalent (on-screen control) when the host reports no keyboard at handshake. +- **`longPress` is reserved for context-invoke** (action panel + Beam) per + `CONCEPT.md`; a Lumen may *also* bind it but the host's context-invoke wins + unless the Lumen declares `captureLongPress: true` on the target. + +The handshake (§13) reports **input modalities** (`touch`/`mouse`/`keyboard`/ +`pen`) so Tier 2 composes the right affordances. + +--- + +## 5. Cadence & motion + +Cadence is declared **per node/region, not globally**. 60 Hz never applies to a +whole tree. + +```ts +type CadenceSpec = 'static' | 'reactive' | { tick: number /* Hz, ≤60 */ }; +``` + +| Cadence | Runs | Cost at rest | For | +|---|---|---|---| +| `static` | once; redraw only on patch | zero | layout, copy, imagery, KPIs | +| `reactive` *(default)* | the `view` sub-tree whose `state` slice changed | zero until change | controls, tables, selection | +| `{tick}` | host clock drives a transition at the capped rate, **scoped to that sub-tree** | one bounded transition/frame for that region | game loop, live chart, animation | + +The runtime **dirty-tracks** changed `state` slices and re-evaluates only +dependent `view` branches (retained-mode + memoisation); `requestAnimationFrame` +is scheduled only while a ticking/animating region is live. **At rest a Lumen +costs ~0 % CPU** (kiosk-critical). + +**Declarative animation ≠ LX tick.** *Presentation motion* (fade, glow-pulse, +count-up, camera ease, Ken-Burns, parallax) is a **declarative animation** the +host runs on the GPU — **zero LX per frame**: + +```ts +type Animate = { property, from, to, durationMs, easing, repeat?, delayMs? }; +``` + +Easing/durations come from the Lume motion tokens (`visual-spec.md` §2.11). +Only *simulation* (state evolving by rules) is an LX `tick`. Reduced-motion +(`prefers-reduced-motion`) collapses animations per `visual-spec.md` §2.11. + +--- + +## 6. Capabilities — the mediated doors + +Default-deny. Each capability is declared, effect-classified (`local` / +`internal` / `external-effect`, `CONCEPT.md` §"Security Surface"), granted by +Tier 2, and **brokered** — a Lumen never performs the effect directly. + +```ts +type CapabilityRequest = { cap: CapabilityName, scope?: object }; +``` + +| Capability | Effect | Broker | Notes | +|---|---|---|---| +| `persist` | internal | `memoryStore@1`, Lumen-scoped namespace | high scores, last viewport | +| `loadData` | internal | read-only, size-capped **projection** of a `DataRef` | data-driven viz/maps/workflows | +| `writeData` | internal / external-effect | **Class-D mutation contract** + `writeCapabilities` manifest | commit back (e.g. Jira) | +| `tiles` | internal | **provider-allowlisted** fetch → sprite `DataRef`s | map tiles (OSM/Mapbox) | +| `fetch` | internal / external-effect | allowlisted, agent-approved endpoint | live feed | +| `generateAsset` | internal / external-effect | **omadia-core LLM connectors** (Tier 3) → `DataRef` | image/sound/voice — §6.2 | +| `clipboard` | external-effect | confirmation-modal gate | copy | +| `share` / `savePreset` | external-effect | §8/§9 | publish a Lumen | + +**Mechanic.** A capability call emits an effect-classified action up the +channel; Tier 2 validates it against the granted manifest, brokers it, patches +the result back — or, for `external-effect`, raises the standard confirmation +modal first (`CONCEPT.md`). The deterministic local loop keeps running while a +call is in flight (async-by-default). Imported/shared Lumens surface their +**capability manifest for consent before first run**; capability tokens are +HMAC-scoped like `dataRef`. + +### 6.1 Asset transport & content-addressed caching (never-stale) + +All binaries (images, audio, video, tiles, voice) travel as **`DataRef`s** +(`CONCEPT.md` §"DataRef lifecycle"), which are **content-addressed**: + +> `id = "-"` — same bytes → same id; different +> bytes → different id. **Always.** + +This is **cache-busting by construction** — the structural fix for stale-cache +behaviour: the id *is* the content hash, so changed content is a *different id* +and old content can never be addressed by a new reference. Path: origin +(Tier-3 / connector) → `DataRef{id, signedToken, expiresAt}` → +`surface_data_ref_created` → client fetches **once** via HMAC token → local +content-addressed store keyed by `id` (instant hits across turns/Lumens/canvases, +automatic dedup) → invalidation **only** explicit (`expiresAt` or +`surface_data_ref_invalidated`) → GC when unreferenced and expired. No +time-based "maybe stale" guesswork. + +### 6.2 Generated material comes from omadia-core, not the Lumen + +Generative assets (images, sounds, music, **synthesised voice**) are produced by +the **LLM connectors wired into omadia-core**, never by the Lumen or the Tier-1 +client. The UI only *requests* (`generateAsset`) and *renders* the returned +`DataRef`. Division of labour: **Lumen/Tier 1** declares + renders + animates +(Ken-Burns etc.), generates nothing; **Tier 2** validates/brokers/caches/patches; +**Tier 3 / core connectors** own the model, choice, cost, rate limits, and +return a content-addressed `DataRef`. A regenerated asset is simply a new `id` — +the old bytes GC out, nothing stale lingers. + +--- + +## 7. Ports & wires — cross-element interaction + +A Lumen is a node in the same primitive tree; elements interact bidirectionally, +deterministically, on Tier 1 (`interactivity-concept.md` §9.1). + +```ts +type PortSpec = { name: string, dir: 'in'|'out', type: PortType }; // on primitives & Lumens +type Wire = { from: { ref: TargetRef, port: string }, to: { ref: TargetRef, port: string } }; // at container/canvas level +``` + +- **Shared selection / view-state.** A `table` and a Lumen referencing the same + `DataRef` + stable IDs share `viewState.selection` (keyed by id) — selecting in + one highlights in the other. Tier-1, no turn. +- **Wires** route a node's typed `out` port to another's `in` port by stable id; + the host resolves and propagates at Tier 1 (Class A). Examples: table + selection → map highlight; slider → sim `state` input; Tetris `game-over` → + `status` text + `writeData`. +- **Least-privilege.** A node reads **only** what is wired to it — it cannot + reach arbitrary other elements' internals. Wires are declared data, + whitelist-validated, resolve by stable id ⇒ deterministic, replayable, + shared-canvas-safe. Authority split unchanged (agent owns which wires exist; + client owns the values flowing). + +--- + +## 8. Lifecycle, presets & reuse + +The agent **authors rarely and reuses constantly** (`interactivity-concept.md` +§8). A Lumen is a durable, versioned component. + +- **Author once, patch after.** Created once via `surface_snapshot`; lives in + canvas-state; edited by **targeted `surface_patch`** addressed by stable + id/path (not regeneration, not string-replace). "Faster" = a one-line patch to + `tick`. +- **Presets.** A vetted Lumen is saved once: **named, versioned, + content-addressed (`preset-`), parameterised**. Scopes + (first match wins): first-party → tenant (`lumen-presets//shared/**`) + → user (`lumen-presets///**`) → canvas-local. Instantiation is + **deterministic, near-zero-LLM**. +- **Resolve-then-generate.** Before any build, Tier 2 runs a library lookup (same + shape as the pre-Tier-3 data-cache check): exact hit → instantiate; near hit → + fork + patch; miss → cold-author (strong model) + offer to save. +- **Fork & vary.** Copy-on-write → new content-addressed id, parent id recorded + for provenance; targeted patch for the variation. +- **Behaviour-idiom library.** The UI Skill carries vetted LX fragments + (scene-grid, tick-loop, input-binding sets, Lume effect bundles) so even cold + builds assemble audited pieces — cheaper and more consistent. + +--- + +## 9. Sharing + +A Lumen serialises cleanly (validated data + capability manifest). Sharing rides +the `CONCEPT.md` forward-compat hooks: `canvasOwnership` extends to +`{kind:"group", members}`; the channel plugin (the fan-out point) multicasts +surface events. The recipient's Tier 1 **re-validates** and shows the capability +manifest for **consent before first run**. Determinism ⇒ every member runs the +identical Lumen; per-user capability grants ⇒ a shared Tetris saves *your* score, +not *mine*. Real-time multiplayer is v2 but *unblocked* by determinism. + +--- + +## 10. Visual treatment (Lume) + +Lumens render in **Lume — light-as-material**, never glassmorphism +(`visual-spec.md` §1.3: "solid light, not see-through plastic"; no refraction, +no blur-as-chrome). The declarative animation layer (§5) *is* the Lume effect +vocabulary (`visual-spec.md` §3): surface-luminosity gradients, two-stop and +donut **glow**, `glow-core` inner light, directional borders, soft corners, +patch-condensation; Ken-Burns/parallax on assets; light-mote particles. `scene` +text uses the three Lume type registers (`visual-spec.md` §2.7). The only blur +is the transient 800 ms condensation (`visual-spec.md` §3.5). See +`visual-spec.md` §"Lumens & scene in Lume". + +--- + +## 11. Security model (summary) + +| Risk | Mitigation | +|---|---| +| Arbitrary code in the renderer | None runs — LX is a validated AST walked by a shipped interpreter; CSP `default-src 'self'`, no `unsafe-eval` | +| Runaway / DoS | Gas + frame ceiling + bounded iteration + state cap → halt with `surface_error`, never the canvas | +| Data exfiltration | Default-deny capabilities; Lumen reads only own state + wired-in ports; all egress brokered, allowlisted, confirmed, Trace-audited | +| Stale / poisoned assets | Content-addressed `DataRef` (id = content hash); HMAC-scoped fetch; explicit invalidation | +| Untrusted shared/imported Lumen | Re-validated on import; capability manifest consent before first run; HMAC scoping; fork lineage | +| Cross-element overreach | Port/wire least-privilege; declared, whitelist-validated, stable-id-resolved | +| Non-reversible effects | `external-effect` class → confirmation-modal gate before the real call | + +--- + +## 12. Wire & SDK deltas (additive over 1.0) + +- **Tree content:** `behavior`/`lumen` section (§1) and the `scene` primitive + (§3) — validated by the extended whitelist parser (schema + LX-AST). Carried + in existing `surface_snapshot` / `surface_patch`. +- **Ports & wires:** `ports` on primitives/Lumens, `wires` at container/canvas + level (§7) — additive tree content, Tier-1-resolved. +- **Cadence & animation:** `cadence` (§5) and `animate` descriptors — additive + trait content. +- **Events:** `surface_capability_request` (client→Tier 2) and + `surface_capability_result` (Tier 2→client) for §6 brokering; results may also + arrive as ordinary `surface_patch`. Reuses the effect-classified action path + and `surface_action_result`. One optional new event family. +- **Presets:** `lumen-presets/**` + `lumen-state/**` memoryStore namespaces (§8). +- **Handshake (§13):** client declares LX version, gas limits, scene support, + granted capability classes, and **input modalities** alongside + `localOperations`. + +Classic channels and 1.0-only clients see none of this — all additive, behind +the `canvas` capability, unknown types ignored (`protocol/1.0.md` §0). + +--- + +## 13. Versioning & negotiation + +LX, the scene vocabulary and the capability catalog are versioned with +`omadia-canvas-protocol` and negotiated at boot. The 1.0 `handshake_select` +(`protocol/1.0.md` §1) gains additive fields: + +```ts +handshake_select += { + lxVersion?: string, // e.g. "1.1" + lxGasLimit?: number, // client's per-eval gas ceiling + sceneSupport?: 'none'|'canvas2d'|'webgl', + capabilityClasses?: CapabilityName[], // what this client can broker/render + inputModalities?: ('touch'|'mouse'|'keyboard'|'pen')[], +} +``` + +A client may implement a subset (e.g. `scene` but not `tiles`); Tier 2 routes +accordingly and idioms degrade gracefully — the same principle as +`localOperations`. + +--- + +## 14. Conformance & open questions + +Conformance is the schema set in `schema/` (Lumen, LX-AST, scene, ports/wires, +capability manifest) + accept/reject fixtures, plus four reference Lumens +(Tetris, interactive workflow, defrag-viz, map) traced end-to-end like +`walkthroughs.md`. Open tuning items (gas/frame/state caps, LX std-lib surface, +scene perf ceiling, capability-consent granularity, determinism-vs-real-time, +LLM reliability emitting LX, preset trust/distribution) are enumerated in +`interactivity-concept.md` §13 — research items, not unspecified holes. diff --git a/docs/visual-spec.md b/docs/visual-spec.md index 00d0920..9e903e7 100644 --- a/docs/visual-spec.md +++ b/docs/visual-spec.md @@ -5,6 +5,14 @@ > Geist (structural) · Source Serif 4 (prose) · Geist Mono (data/code). > Codex-review-ready in the CONCEPT.md cadence. +Version 0.5 — **Lumens & `scene` visual treatment (§4.13).** Pins how the +Live-Interactivity extension renders in Lume: `scene` is editor-class +(`radius.0`); draw-list colours are tokens only (always on-theme); presentation +motion is the declarative Lume effect vocabulary (§3) on the GPU, not pixel +math and **not glass** (the §1.3 NOT-list holds for Lumens); cadence is +per-region. Companion to `../docs/interactivity-concept.md` (concept) and +`../docs/lumens-spec.md` (definition). + Version 0.4 — **Surface-nesting ladder & chrome budget.** Closes the two spec gaps that produced doubled chrome in the first shipped canvases: the spec never said which surface a nested container gets (§2.13 — the ladder: @@ -1265,6 +1273,46 @@ Tier-1 boundary where the agent's chrome ends and the user's raw work begins. --- +## 4.13 Lumens & `scene` — visual treatment + +The Live-Interactivity extension (`../docs/interactivity-concept.md` rationale, +`../docs/lumens-spec.md` definition) adds a 25th primitive, **`scene`** — a +declarative immediate-mode draw surface for games, custom visualisations and +maps — and **Lumens** (self-contained interactive units). Both render in Lume; +this section pins how. + +**`scene` is editor-class — `radius.0`, sharp edges.** Like `canvas-region` +and `timeline` (§2.9), a `scene` is where Lume material stops; the hard edge is +the Tier-1 boundary marker. Its `camera` pan/zoom reuses the canvas-region +affordances. + +**Draw-list colours are tokens only.** Every `scene` shape fills/strokes from +the design tokens + active palette (`accent`, `accent.glow*`, surface/text/ +semantic tokens) — never free-form colour. A game board, a defrag grid or a map +marker layer is therefore always on-theme: it looks like Omadia, not like a +foreign website. `scene` `text` nodes use the three type registers (§2.7). + +**Motion = the Lume effect vocabulary, declarative.** Presentation motion on a +Lumen (fade, glow-pulse, count-up, camera ease, Ken-Burns on a `sprite`, +parallax, light-mote particles) is a **declarative animation the host runs on +the GPU** — it composes the §3 primitives (two-stop glow §3.2, donut §3.3, +surface gradient §3.1, condensation §3.5) and the §2.11 motion tokens. It is +**not** per-frame pixel math, and it is emphatically **not glass** — the §1.3 +"NOT" list holds for Lumens too (no refraction, no blur-as-chrome, +glassmorphism stays out). The only blur is the transient condensation (§3.5). + +**Cadence is per-region (§ render-cadence in the concept).** `static` and +`reactive` regions cost ~0 % CPU at rest; only a `{tick}` region animates — a +kiosk Lumen is mostly static beautiful surface with a few lit, moving accents. +A reference mockup lives at +[`./mockups/kiosk-lumen-aura.html`](./mockups/kiosk-lumen-aura.html). + +**Touch-first.** Lumen hit-targets honour the 44 pt minimum; the material's +soft glow/halo affordances replace hover (which is dropped as a requirement) — +see `../docs/lumens-spec.md` §4. + +--- + ## 5. Composition idioms The five idioms from CONCEPT.md's Composition-Idiom Library, rendered in From 0017f0970309cfc9814fd533309fa38e769c2263 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 17:50:34 +0000 Subject: [PATCH 07/13] docs: remove Tetris example, use neutral game/business examples --- CONCEPT.md | 2 +- docs/implementation-plan.md | 2 +- docs/interactivity-concept.md | 46 +++++++++++++++++------------------ docs/lumens-spec.md | 6 ++--- 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/CONCEPT.md b/CONCEPT.md index 7f3dc6d..53f61c6 100644 --- a/CONCEPT.md +++ b/CONCEPT.md @@ -1204,7 +1204,7 @@ This section becomes the input for the v2 design phase. It is not a v1 deliverab ## Extension — Live Interactivity (Lumens) A planned **additive** extension brings rich, agent-generated, **Tier-1-fast** -interactivity to the canvas — games (Tetris), interactive data workflows, +interactivity to the canvas — small games, interactive data workflows, unusual visualisations (defrag-style), live maps — the Omadia answer to sandbox-style "live artifacts", deliberately re-aimed. Its thesis: most of what arbitrary-code sandboxes block is blocked by missing **capabilities**, not diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index cb16d8f..dee6b90 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -718,7 +718,7 @@ maintainer as a GitHub issue. Suggested phasing, smallest mergeable units first: | **L6 — ports & wires** | typed ports on primitives/Lumens, Tier-1 wire resolution, shared `viewState.selection` cross-element | host app + orchestrator | | **L7 — lifecycle & presets** | author-once/patch; `lumen-presets/**` + `lumen-state/**` stores; resolve-then-generate lookup; fork+patch; behaviour-idiom library in the UI Skill | `omadia-ui-orchestrator` | | **L8 — sharing** | `canvasOwnership` group extension + channel fan-out + import consent (rides the v2 shared-canvas hooks) | channel + orchestrator | -| **L9 — reference Lumens** | Tetris · interactive workflow · defrag-viz · map, traced end-to-end like `walkthroughs.md`; conformance fixtures | this repo | +| **L9 — reference Lumens** | an arcade game · interactive workflow · defrag-viz · map, traced end-to-end like `walkthroughs.md`; conformance fixtures | this repo | **Riskiest items** (mirror `interactivity-concept.md` §13): LLM reliability emitting valid LX (likely a strong-model authoring job, fast-model patching); diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index a3d8ca7..44ed904 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -1,7 +1,7 @@ # Omadia UI — Live Interactivity Concept (Lumens) > How Omadia UI gains rich, agent-generated, **Tier-1-fast** interactivity — -> Tetris, interactive data workflows, unusual visualisations (think the old +> small games, interactive data workflows, unusual visualisations (think the old > HDD-defragmenter view), live maps — **without** giving up the whitelist / > no-arbitrary-code security model that makes the canvas safe. > @@ -74,7 +74,7 @@ interactive thing", and both are wrong for Omadia: The key diagnosis — and the thesis of this document: > **Most of what Live Artifacts blocks is blocked by missing *capabilities*, -> not missing *compute*.** Tetris does not need network. A map needs *tiles*, +> not missing *compute*.** A game loop does not need network. A map needs *tiles*, > not arbitrary JS. A data workflow needs *your data and a write-back path*, > not a Turing-complete escape hatch. > @@ -107,11 +107,11 @@ type Lumen = { Plain-English mapping: -- **`state`** — the Lumen's memory. For Tetris: the board grid, the active - piece, score, level, game-over flag. Typed and size-capped. +- **`state`** — the Lumen's memory. For a game: the board, the moving + token(s), score, level, game-over flag. Typed and size-capped. - **`transitions`** — the rules. Pure functions `(state, event) → newState`, written in **Lume Expressions** (a small, total, sandboxed expression - language — §3). For Tetris: `tick` drops the piece, `moveLeft`, `rotate`, + language — §3). For a game: `tick` advances the board, `moveLeft`, `rotate`, `lockPiece`, `clearLines`. No I/O, no host access, deterministic. - **`view`** — the look. A pure function `state → tree`, producing either ordinary Omadia primitives (so a Lumen can be a live form/table/dashboard) @@ -312,7 +312,7 @@ the dependent `view` branches (retained-mode + memoisation). live** and torn down when it settles. **At rest a Lumen costs ~0 % CPU** — a kiosk showing a beautiful, mostly-static screen burns nothing until someone touches it or a single badge pulses. One Lumen routinely mixes all three: a -Tetris scene ticks at 60 Hz, the score label beside it is `reactive`, the +a game's scene ticks at 60 Hz, the score label beside it is `reactive`, the surrounding chrome is `static`. Only the part that must move pays. ### Motion comes from a declarative animation layer, not from LX @@ -413,7 +413,7 @@ Because the client only ever does *bounded interpretation* + *raster of a draw-list it already has* (everything heavy is brokered to Tier 2/3), the thin client is the design centre, not the stress test: -- 🟢 **Flüssig:** puzzle/board/card games (Tetris, 2048, chess), interactive +- 🟢 **Flüssig:** puzzle/board/card games (2048, chess, solitaire), interactive dashboards & workflows, data-viz incl. defrag-style grids to ~5–10 k cells (canvas2d; more on WebGL), maps (tiles are GPU-composited images fetched by Tier 2/3 — the client doesn't compute the map). @@ -449,14 +449,14 @@ in the Lumen, effect-classified, and brokered by Tier 2. Default deny. | Capability | Effect class | Broker path | Example | |---|---|---|---| -| `persist(key, value)` | `internal` | Tier 2 → `memoryStore@1` under a Lumen-scoped namespace | Tetris high score; map's last viewport | +| `persist(key, value)` | `internal` | Tier 2 → `memoryStore@1` under a Lumen-scoped namespace | a game's high score; map's last viewport | | `loadData(dataRef)` | `internal` | Tier 2 hands the Lumen a **read-only, size-capped projection** of an existing `DataRef` | drive a defrag-style viz from a real dataset; bind a map to a places table | | `writeData(target, value)` | `internal` / `external-effect` | reuses the **Class-D mutation contract** + `writeCapabilities` manifest | an interactive triage workflow that commits status changes back to Jira | | `tiles(provider, z/x/y)` | `internal` | Tier 2/3 fetches from a **provider-allowlisted** endpoint, returns sprite `DataRef`s | OpenStreetMap / Mapbox map tiles | | `fetch(declaredEndpoint)` | `internal` / `external-effect` | Tier 3 tool call against an **allowlisted, agent-approved** endpoint only | a live feed into a visualisation | | `clipboard(text)` | `external-effect` | confirmation-modal gate | "copy result" | | `generateAsset(spec)` | `internal` / `external-effect` | Tier 2 → **omadia-core LLM connectors** (Tier 3); returns a `DataRef` | generate an image / sound / TTS voice line for the scene | -| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §7 | share Tetris with a colleague; save as a gallery preset | +| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §7 | share a game with a colleague; save as a gallery preset | Mechanics: a capability call from the running Lumen is *not* a direct call. It emits a capability-request action (effect-classified) up through the channel; @@ -550,9 +550,9 @@ and the channel plugin (already the designated **fan-out point**) multicasts surface events to connected members. The recipient's Tier 1 **re-validates** the Lumen and shows its **capability manifest for consent before first run**. Because behaviour is deterministic, every member sees the identical Lumen; -because capabilities are per-user-granted, a shared Tetris can save *your* +because capabilities are per-user-granted, a shared game can save *your* high score without touching *mine*. Real-time multiplayer (two people in one -Tetris) is a v2 topic but is *unblocked* by determinism — deterministic ops +game) is a v2 topic but is *unblocked* by determinism — deterministic ops are exactly what lockstep/CRDT need. **Save as a preset.** A Lumen can be named, optionally **parameterised** @@ -568,7 +568,7 @@ because they are validated, deterministic, capability-declared data, not code. ## 8. Lumen lifecycle & reuse — author once, then instantiate & patch (never rebuild) -Regenerating a kiosk surface or a Tetris from scratch on every turn would be +Regenerating a kiosk surface or a game from scratch on every turn would be expensive (tokens), slow (a full generation), and inconsistent (two "builds" drift). It must not happen, and the architecture already has the pieces to stop it. This is the Omadia counterpart to Claude's artifact logic @@ -585,7 +585,7 @@ bumping `treeRevision`: | User says | What the agent emits | Cost | |---|---|---| -| "Build me Tetris" *(nothing reusable exists)* | one `surface_snapshot` with the full Lumen | one expensive turn (once) | +| "Build me a game" *(nothing reusable exists)* | one `surface_snapshot` with the full Lumen | one expensive turn (once) | | "Make it faster as the score climbs" | `surface_patch` touching only the `tick` transition | a few tokens | | "Bigger board" | `surface_patch` on one `state` field + the `view` grid bounds | a few tokens | | "Change the accent to Atelier" | accent re-tint patch (existing palette mechanic) | trivial | @@ -602,7 +602,7 @@ The durable answer is the **preset library** (§7). A vetted Lumen is authored parameterised**. Thereafter the agent does not build it — it **instantiates** it: -> "Gib mir Tetris" → Tier 2 resolves the intent to the `tetris@2` preset, +> "Instantiate the game" → Tier 2 resolves the intent to the `arcade@2` preset, > binds its parameters (board size, palette, data), and renders it. > **Near-zero LLM, sub-second, byte-identical every time.** @@ -610,14 +610,14 @@ Library scopes, outermost to innermost (first match wins, like CSS cascade): | Scope | Namespace | Holds | |---|---|---| -| **First-party** | shipped with the orchestrator | curated, audited Lumens (Tetris, map, defrag-viz, standup board) | +| **First-party** | shipped with the orchestrator | curated, audited Lumens (an arcade game, map, defrag-viz, standup board) | | **Tenant / org** | `lumen-presets//shared/**` | the organisation's vetted library | | **User** | `lumen-presets///**` | "my defrag-style project viewer" | | **Canvas-local** | canvas-state | the instance currently on screen | Because a preset is **content-addressed** (`preset-`) and **deterministic**, every instantiation is identical — there is no -"two-divergent-Tetris" problem, and a shared preset replays the same on every +"two-divergent-instances" problem, and a shared preset replays the same on every machine (§2 determinism, §6.1 content-addressing). ### 8.3 Two-speed generation — the cost & consistency model @@ -628,7 +628,7 @@ machine (§2 determinism, §6.1 content-addressing). | **Warm reuse** | a preset matches (exactly or after a small edit) | none, or a fast model for a parametric patch | sub-second | **the common case** | After a cold authoring, Tier 2 **offers to save the result as a preset** -("Save this Tetris to your gallery?"). The expensive build is paid **once**; +("Save this game to your gallery?"). The expensive build is paid **once**; every later use is warm. This is the same economics that makes Claude artifacts feel cheap after creation — generalised into a persistent library instead of a per-conversation artifact. @@ -651,8 +651,8 @@ generation. Generation is the fallback, not the first move. ### 8.5 Fork & vary — copy-on-write, with lineage -"Tetris, but hexagonal" does **not** rebuild Tetris. Tier 2 **forks** the -`tetris` preset (copy-on-write → new content-addressed id, parent id recorded +"the game, but on a hexagonal board" does **not** rebuild it. Tier 2 **forks** +the `arcade` preset (copy-on-write → new content-addressed id, parent id recorded for provenance) and applies a **targeted patch** to the affected transitions/view. Cheap, consistent with the original, and the lineage is auditable — important for the trust model when forks are shared (§2 point 5). @@ -697,7 +697,7 @@ Lumens are generated agentically (Tier 2 composes the Lumen the way it composes primitive trees today; heavy generation or data binding can recruit Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: -- **Agentic generation.** "Build me a Tetris" → Tier 2 emits a Lumen +- **Agentic generation.** "Build me a game" → Tier 2 emits a Lumen (state/transitions/view/events) via `surface_snapshot`. "Make it faster as the score climbs" → `surface_patch` adjusting the `tick` transition. The agent refines the *running* behaviour conversationally. @@ -746,7 +746,7 @@ Concrete: - **`form`/slider → simulation Lumen.** A `choice`/slider primitive is wired to a Lumen `state` input (gravity, speed, grid size). Dragging the slider retunes the running sim live — declarative, no turn. -- **Lumen output → primitive.** Tetris `game-over` (a Lumen output) wired to a +- **Lumen output → primitive.** A game's `game-over` (a Lumen output) wired to a `status` primitive ("New high score!") and to a `writeData` capability that persists the score back through Tier 3. - **List ⇄ defrag-viz Lumen.** Hover/select a file in a `list` → its blocks @@ -773,7 +773,7 @@ values currently flowing through them* (view-state). | Use case | state | view | events | capabilities | Tier split | |---|---|---|---|---|---| -| **Tetris** (build · play · share) | board grid, active piece, score, level | `scene` cell grid | declared keys + host `tick` (≤60 fps) | `persist` (high score), `share` | loop is pure **Class A** on Tier 1; generation/share on Tier 2 | +| **An arcade game** (build · play · share) | board, moving token(s), score, level | `scene` cell grid | declared keys + host `tick` (≤60 fps) | `persist` (high score), `share` | loop is pure **Class A** on Tier 1; generation/share on Tier 2 | | **Interactive data workflow** | working set, step, selection, edits | primitives (table/form) or `scene` | pointer/submit | `loadData` (real data in), `writeData` (commit back via Class-D + `writeCapabilities`) | Tier 1 interaction; Tier 2 brokers writes; Tier 3 owns the system of record | | **Defrag-style / unusual viz** | dataset projection, animation cursor | `scene` coloured-cell grid | host `tick` for animation | `loadData` | data fetch via Tier 2/3; animation pure Tier 1 | | **Interactive map** | viewport, markers, selection | `scene` (sprites = tiles, markers) + pan/zoom | pointer (pan/zoom/click) | `tiles` (provider-allowlisted), `loadData` (places), `persist` (last viewport) | pan/zoom/render pure Tier 1; tiles brokered Tier 2/3 | @@ -836,7 +836,7 @@ dangerous one, open the useful one.** 1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, scene draw-list length, tick rate — measured against the four reference - Lumens (Tetris, workflow, defrag-viz, map). Spike-tunable, like the + Lumens (an arcade game, workflow, defrag-viz, map). Spike-tunable, like the `viewState` budget in `CONCEPT.md`. 2. **LX surface area.** Exactly which standard-library functions ship in `LX/1.0`. Bias small; grow by minor bump. Risk: too small blocks real diff --git a/docs/lumens-spec.md b/docs/lumens-spec.md index fbe2113..912bc4c 100644 --- a/docs/lumens-spec.md +++ b/docs/lumens-spec.md @@ -355,7 +355,7 @@ type Wire = { from: { ref: TargetRef, port: string }, to: { ref: TargetRef, one highlights in the other. Tier-1, no turn. - **Wires** route a node's typed `out` port to another's `in` port by stable id; the host resolves and propagates at Tier 1 (Class A). Examples: table - selection → map highlight; slider → sim `state` input; Tetris `game-over` → + selection → map highlight; slider → sim `state` input; a game's `game-over` → `status` text + `writeData`. - **Least-privilege.** A node reads **only** what is wired to it — it cannot reach arbitrary other elements' internals. Wires are declared data, @@ -397,7 +397,7 @@ the `CONCEPT.md` forward-compat hooks: `canvasOwnership` extends to `{kind:"group", members}`; the channel plugin (the fan-out point) multicasts surface events. The recipient's Tier 1 **re-validates** and shows the capability manifest for **consent before first run**. Determinism ⇒ every member runs the -identical Lumen; per-user capability grants ⇒ a shared Tetris saves *your* score, +identical Lumen; per-user capability grants ⇒ a shared game saves *your* score, not *mine*. Real-time multiplayer is v2 but *unblocked* by determinism. --- @@ -479,7 +479,7 @@ accordingly and idioms degrade gracefully — the same principle as Conformance is the schema set in `schema/` (Lumen, LX-AST, scene, ports/wires, capability manifest) + accept/reject fixtures, plus four reference Lumens -(Tetris, interactive workflow, defrag-viz, map) traced end-to-end like +(an arcade game, interactive workflow, defrag-viz, map) traced end-to-end like `walkthroughs.md`. Open tuning items (gas/frame/state caps, LX std-lib surface, scene perf ceiling, capability-consent granularity, determinism-vs-real-time, LLM reliability emitting LX, preset trust/distribution) are enumerated in From 1e366887b18437770dca8f744fad006f44f1e20b Mon Sep 17 00:00:00 2001 From: Christian Wendler Date: Mon, 15 Jun 2026 20:40:03 +0200 Subject: [PATCH 08/13] docs: address Codex adversarial review of Lumens spec (1.1 rev 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Local Codex review (adversarial, vs origin/main) flagged six issues across the Live-Interactivity additions; tractable points fixed, two architectural points resolved with the maintainer: - Additivity made fail-closed: 1.1 content is negotiation-gated (a 1.0 client is never sent it; unknown types are hard-rejected by the whitelist, not silently "ignored"). Aligns lumens-spec §0/§12 with protocol/1.0 §2. - Capability authority: the agent owns capability *requests*, never *grants* (a grant is Tier-2 policy + user consent; no self-grant via patch). §0.5. - Bounded wakeups: `timer` is capped like `tick` via a combined per-Lumen wakeup budget, rejected at validation when exceeded. §0.2/§4/§11. - Capability-broker egress bounds (rate/quota/max-in-flight/idempotency/ backpressure) + state/DataRef-derived fetch/writeData classified external-effect unless pre-approved. §6/§11 + open-question §13.8. - Shared/preset assets travel by content id; Tier 2 re-mints DataRef tokens scoped to the recipient, inaccessible assets render inert. §9 / concept §7. - Cross-element reads: ambient-by-default selection replaced by a declared `expose` read-only interface (ambient-by-declaration) — un-exposed state stays private; imported Lumens see no ambient neighbour state. §7/§11/§9.1. Plus: LX versioned with the protocol (LX/1.1, handshake lxVersion "1.1"); mockup references visual-spec v0.5 §4.13. Version headers bumped (interactivity-concept 0.6; lumens-spec draft 1.1 rev 2). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/interactivity-concept.md | 79 +++++++++++----- docs/lumens-spec.md | 141 ++++++++++++++++++++++------- docs/mockups/kiosk-lumen-aura.html | 2 +- 3 files changed, 165 insertions(+), 57 deletions(-) diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md index 44ed904..a996f24 100644 --- a/docs/interactivity-concept.md +++ b/docs/interactivity-concept.md @@ -17,6 +17,22 @@ > only** — no implementation, no PR > plan. It extends, and stays inside, the architecture in `CONCEPT.md`. +Version 0.6 — **Codex-review fixes** (adversarial review of the 0.5 branch). +**Capability authority**: the agent owns capability *requests*, never *grants* +— a grant is Tier-2 policy + user consent, an agent patch can ask but not +self-grant (`lumens-spec.md` §0.5). **Additivity made fail-closed**: 1.1 content +is *negotiation-gated* (a 1.0 client is never sent it, and unknown types are +hard-rejected, not silently ignored — `lumens-spec.md` §0/§12). **Bounded +wakeups**: `timer` is capped like `tick` via a combined per-Lumen wakeup budget +(`lumens-spec.md` §0.2/§4). **Broker egress bounds**: per-capability rate/quota, +max-in-flight, idempotency and backpressure, plus state/`DataRef`-derived +`fetch`/`writeData` classified `external-effect` (§13.8; `lumens-spec.md` +§6/§11). **Shared assets**: travel by content `id`; Tier 2 re-mints `DataRef` +tokens for the recipient, inaccessible assets render inert (§7). **Cross-element +reads**: ambient-by-default selection replaced by a declared **`expose`** +read-only interface (ambient-*by-declaration*) — un-exposed state stays private, +imported Lumens observe no ambient neighbour state (§9.1). **LX versioning**: +LX ships as `LX/1.1`, declared at the handshake (§13.2). Version 0.5 — expands §9 with §9.1 "Composition & cross-element interaction": a Lumen is just a node in the same primitive tree (composes in container/pane/grid/tabs; a Lumen's `view` can hold primitives and vice @@ -555,6 +571,12 @@ high score without touching *mine*. Real-time multiplayer (two people in one game) is a v2 topic but is *unblocked* by determinism — deterministic ops are exactly what lockstep/CRDT need. +Asset references travel as content-addressed `DataRef` **ids** (or an asset +manifest), **never** the author's HMAC-scoped tokens (§6.1, scoped to the +author's `tenant ‖ user ‖ canvasSession`); on import Tier 2 **re-mints** each +token scoped to the recipient, and an asset the recipient may not access renders +**inert** rather than reusing a borrowed token. + **Save as a preset.** A Lumen can be named, optionally **parameterised** (declare which parts of `state` are preset inputs), and stored in a preset store (`memoryStore@1` namespace, e.g. `lumen-presets//`). A @@ -732,7 +754,7 @@ runs at 60 fps: | Mechanism | How it wires | Direction | Tier | |---|---|---|---| -| **Shared selection / view-state** | a `table` and a Lumen both reference the same `DataRef` + **stable IDs**; selection lives in `viewState.selection` keyed by those IDs (`CONCEPT.md` Authority Model) | UI ⇄ Lumen | **1 (Class A)** — no server | +| **Shared selection / view-state** | an element **publishes** a lightweight read-only interface (`expose`, e.g. `selection`); a neighbour referencing the same `DataRef` + **stable IDs** reads the *published* field by name — no explicit wire, and un-exposed state stays private (`CONCEPT.md` Authority Model) | UI ⇄ Lumen | **1 (Class A)** — no server | | **Declarative ports & wires** | a node declares typed **inputs** and **outputs**; the agent declares **wires** (`from` → `to`) by stable id. The host routes values at Tier 1 | UI ⇄ Lumen, Lumen ⇄ Lumen | **1 (Class A)** | | **Lumen capability → patch** | a Lumen output bound to `writeData`/`generateAsset`/etc. patches another container via Tier 2 | Lumen → UI (semantic) | 2 / 3 | | **Beam / agent** | user beams across both ("compare these rows with the map"); agent reasons over the wired set | either, semantic | 2 | @@ -752,20 +774,26 @@ Concrete: - **List ⇄ defrag-viz Lumen.** Hover/select a file in a `list` → its blocks light up in the scene; click a block cluster → the list scrolls to that file. -**The boundary that keeps it safe.** Cross-element wiring is **declared data** -(validated by the whitelist parser), and a node can only read what is **wired -to it** — it cannot reach arbitrary other elements' internals. This is -least-privilege by construction: the same port/wire boundary that makes a -Lumen safe to share makes cross-element interaction auditable and -shared-canvas-safe (wires resolve by stable id, deterministically, so they -replay and multicast unchanged). The **authority split is unchanged**: the -agent owns *which elements and wires exist* (structure); the client owns *the -values currently flowing through them* (view-state). - -> **SDK delta:** declarative **`ports`** (typed inputs/outputs) on primitives -> and Lumens, and **`wires`** (a `[{from: TargetRef+port, to: TargetRef+port}]` -> list) at the container/canvas level — additive tree content, Tier-1-resolved, -> whitelist-validated. No new transport. +**The boundary that keeps it safe.** Cross-element interaction is **declared +data** (validated by the whitelist parser), and a node can only read what is +**wired or published to it** — an explicit `wire`, or a neighbour's declared +**`expose`** interface (a lightweight, read-only set of view-state the neighbour +*chose* to offer, bound by shared id). It cannot reach arbitrary other elements' +internals, and **un-exposed state stays private** — so an imported or untrusted +Lumen observes no ambient neighbour state and itself leaks nothing it did not +publish. This is least-privilege by construction, *ambient-by-declaration* not +ambient-by-default: the same boundary that makes a Lumen safe to share makes +cross-element interaction auditable and shared-canvas-safe (wires and `expose` +resolve by stable id, deterministically, so they replay and multicast +unchanged). The **authority split is unchanged**: the agent owns *which +elements, wires and published interfaces exist* (structure); the client owns +*the values currently flowing through them* (view-state). + +> **SDK delta:** declarative **`ports`** (typed inputs/outputs) and **`expose`** +> (a published read-only view-state interface, bindable by shared id without a +> wire) on primitives and Lumens, and **`wires`** (a `[{from: TargetRef+port, +> to: TargetRef+port}]` list) at the container/canvas level — additive tree +> content, Tier-1-resolved, whitelist-validated. No new transport. --- @@ -793,7 +821,7 @@ authority split, and security surface. No wire-grammar rewrite. |---|---| | **Primitives** | add `scene` (editor-class) — protocol minor bump (`1.x`); draw-list is whitelisted shape data | | **Tree content** | add the `behavior`/`lumen` section (state/transitions/view/events/capabilities) — validated by an **extended whitelist parser** (schema + LX-AST validator) | -| **Composition / interaction** (§9.1) | declarative **`ports`** (typed inputs/outputs) on primitives & Lumens + **`wires`** (`from`→`to` by `TargetRef`) at container/canvas level; Tier-1-resolved by stable id, whitelist-validated; reuses shared `viewState.selection`. No new transport | +| **Composition / interaction** (§9.1) | declarative **`ports`** (typed inputs/outputs) + **`expose`** (published read-only interface) on primitives & Lumens + **`wires`** (`from`→`to` by `TargetRef`) at container/canvas level; Tier-1-resolved by stable id, whitelist-validated; shared `viewState.selection` read only via a published `expose` field (ambient-by-declaration). No new transport | | **Tier-1 client** | new **Lumen runtime**: deterministic LX evaluator, gas + frame ceiling, scene rasteriser, event dispatch, seeded `random`/clock. All Class-A; the frame loop never touches the server | | **Tier-2 orchestrator** | composes/patches Lumens; **brokers capability calls**; grants/scopes capability manifests; persists Lumen state & presets; manages share/ownership | | **Lifecycle / reuse** (§8) | **resolve-then-generate**: library lookup before any build (same shape as the existing pre-Tier-3 data-cache check); preset **instantiate** (no LLM) / **fork+patch** (fast model) / **cold-author** (strong model); content-addressed, versioned, parameterised presets; behaviour-idiom library in the UI Skill | @@ -835,12 +863,14 @@ dangerous one, open the useful one.** ## 13. Open questions for the spike (flagged, not answered) 1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, - scene draw-list length, tick rate — measured against the four reference - Lumens (an arcade game, workflow, defrag-viz, map). Spike-tunable, like the - `viewState` budget in `CONCEPT.md`. + scene draw-list length, tick rate, the `timer` minimum period and the + **combined `tick`+`timer` wakeup budget** per Lumen — measured against the + four reference Lumens (an arcade game, workflow, defrag-viz, map). + Spike-tunable, like the `viewState` budget in `CONCEPT.md`. 2. **LX surface area.** Exactly which standard-library functions ship in - `LX/1.0`. Bias small; grow by minor bump. Risk: too small blocks real - Lumens; too large grows the audit surface. + `LX/1.1` (LX is versioned with the protocol — the boot handshake declares + `lxVersion: "1.1"`, `lumens-spec.md` §13). Bias small; grow by minor bump. + Risk: too small blocks real Lumens; too large grows the audit surface. 3. **Scene performance ceiling.** Draw-list size at which Tier-1 raster drops below 60 fps; whether WebGL is required for v1 or canvas-2d suffices for the reference set. @@ -858,6 +888,13 @@ dangerous one, open the useful one.** 7. **Preset trust & distribution.** Signing, capability-manifest review UX, and whether a shared community gallery needs a moderation/attestation layer (likely v2+). +8. **Capability-broker egress bounds.** The exact anti-DoS / anti-cost contract + for the Tier-2 broker — per-capability rate/quota, max-in-flight, idempotency + keys and backpressure — so a `tick`/`timer`-driven `generateAsset`/`fetch`/ + `writeData` cannot push load or spend onto Tier 2/3, plus the rule that + state/`DataRef`-derived outbound requests are `external-effect` unless + pre-approved (`lumens-spec.md` §6, §11). Spike-tunable, measured against the + reference Lumens. --- diff --git a/docs/lumens-spec.md b/docs/lumens-spec.md index 912bc4c..22aebaf 100644 --- a/docs/lumens-spec.md +++ b/docs/lumens-spec.md @@ -10,11 +10,28 @@ > (the canvas architecture). **Status:** `draft 1.1`. Additive, **minor** bump over -`omadia-canvas-protocol/1.0` — old clients ignore unknown primitive types, -tree sections, events and capabilities (`protocol/1.0.md` §0). Nothing here -breaks the 1.0 wire grammar. The machine-validatable truth will live in -`schema/` (Lumen, LX-AST, scene, ports/wires, capability manifest) and is a -spike deliverable; where prose and schema disagree, the **schema wins**. +`omadia-canvas-protocol/1.0`, and **negotiation-gated**: all 1.1 content (the +`scene` primitive, the `behavior`/`lumen` tree section, ports/wires, the +`surface_capability_*` events) is emitted **only to clients that negotiated +support at the boot handshake** (§13). A 1.0-only client never negotiates it, so +Tier 2 never sends it; and if such content ever reached a 1.0 renderer it would +be **hard-rejected** by the whitelist parser (`protocol/1.0.md` §2), never +silently mis-rendered — additivity is enforced **fail-closed by negotiation, not +by clients silently ignoring unknown primitive types**. (Only additive *fields +on already-known types* ride 1.0's "ignore unknown fields" forward-compat, +`protocol/1.0.md` §0; unknown *types / tree sections* are gated, not ignored.) +Nothing here breaks the 1.0 wire grammar. The machine-validatable truth will +live in `schema/` (Lumen, LX-AST, scene, ports/wires, capability manifest) and +is a spike deliverable; where prose and schema disagree, the **schema wins**. + +> **Rev 2 (Codex review).** Clarifications within `draft 1.1` (no protocol-minor +> change): additivity is negotiation-gated / fail-closed (§0, §12); the agent +> owns capability *requests*, never *grants* (§0.5); `timer` is bounded like +> `tick` via a combined wakeup budget (§0.2, §4); capability-broker egress +> bounds + state/`DataRef`-derived `fetch`/`writeData` classification (§6, §11); +> shared/preset assets travel by content `id` with recipient-scoped token +> re-mint (§9); ambient cross-element reads replaced by a declared `expose` +> interface (§7, §11). A Lumen is the Omadia answer to "an interactive artifact": **declarative data, not code**, run by a small deterministic interpreter on Tier 1, generated and @@ -38,8 +55,9 @@ stable IDs as the lingua franca, one JSON value per frame) plus: ports and capabilities: any unknown node/type → hard reject (`surface_error`). 2. **Bounded & total.** Every transition / view evaluation runs under a gas budget and a wall-clock ceiling; iteration is bounded (no open `while`, no - general recursion); state is size-capped. A Lumen can never hang the host — - exceeding budget halts it with `surface_error`. + general recursion); state is size-capped; the **wakeup rate** (`tick` + + `timer` combined) is itself capped and count-limited per Lumen (§4). A Lumen + can never hang the host — exceeding any budget halts it with `surface_error`. 3. **Deterministic.** All non-determinism is host-seeded (`random`, `now`, tick). `(state, event) → state` is identical on every machine — the basis for replay, undo, safe sharing and v2 multi-user. @@ -47,9 +65,14 @@ stable IDs as the lingua franca, one JSON value per frame) plus: except through declared, granted, effect-classified capabilities brokered by Tier 2 (`CONCEPT.md` §"Security Surface" effect classes). 5. **Authority split unchanged.** The agent owns *structure* (which Lumens, - elements, wires, capability grants exist). The client owns *view-state* (the - values flowing through them, current selection, scroll). Stable IDs bind the - two (`CONCEPT.md` §"Authority Model"). + elements, wires, published `expose` interfaces, and capability **requests** + exist). The client owns *view-state* (the values flowing through them, + current selection, scroll). + Capability **grants** are *not* agent-owned: a request is granted only by + Tier-2 policy plus user consent where the effect class requires it (§6) — an + agent-authored patch can *ask* for a capability, never *self-grant* one + (no `fetch` / `writeData` / `share` escalation via a patch). Stable IDs bind + the two (`CONCEPT.md` §"Authority Model"). 6. **Lume is the material.** Lumens render in Lume — light-as-material, **not** glassmorphism (`visual-spec.md` §1.3). §10. @@ -70,7 +93,8 @@ type Lumen = { events: EventBinding[]; // §4 — declared inputs -> transitions cadence?: CadenceSpec; // §5 — default "reactive" capabilities?: CapabilityRequest[]; // §6 — default-deny doors out - ports?: PortSpec[]; // §7 — typed inputs/outputs for wiring + ports?: PortSpec[]; // §7 — typed inputs/outputs for explicit wiring + expose?: ExposeSpec[]; // §7 — published read-only view-state (the ambient-readable interface) preset?: PresetRef; // §8 — provenance if instantiated/forked }; ``` @@ -78,8 +102,8 @@ type Lumen = { A Lumen is valid iff: its `state` conforms to §1.1, every `LXNode` in `transitions`/`view` passes the §2 AST whitelist + static bounds check, every `EventBinding` names a declared transition and a §4 event, every -`CapabilityRequest` names a §6 catalog capability, and every `PortSpec` is -§7-typed. Any failure → the Lumen is rejected wholesale with `surface_error` +`CapabilityRequest` names a §6 catalog capability, and every `PortSpec` / +`ExposeSpec` is §7-typed. Any failure → the Lumen is rejected wholesale with `surface_error` (scope = the Lumen `id`); it never partially renders. ### 1.1 State schema @@ -240,6 +264,16 @@ Rules (normative): - **`longPress` is reserved for context-invoke** (action panel + Beam) per `CONCEPT.md`; a Lumen may *also* bind it but the host's context-invoke wins unless the Lumen declares `captureLongPress: true` on the target. +- **Bounded wakeups (timers are capped like ticks).** `tick` is rate-capped + (`rate` ≤ 60 Hz, §5). `timer` is bounded the *same* way: `everyMs` has an + enforced **minimum period**, the number of `tick` + `timer` bindings per + Lumen is **count-capped**, and their **combined wakeup rate shares one + per-Lumen budget**. A schedule that exceeds the budget is **rejected at + validation** (`surface_error`), never accepted-then-throttled. Caps are + spike-tunable initial defaults, like gas (§2.4); they keep the §0.2 + "cannot hang / cannot DoS the host" guarantee true for `timer`, not only for + `tick` (a swarm of 1 ms timers each individually under gas is still rejected + in aggregate). The handshake (§13) reports **input modalities** (`touch`/`mouse`/`keyboard`/ `pen`) so Tier 2 composes the right affordances. @@ -309,6 +343,18 @@ call is in flight (async-by-default). Imported/shared Lumens surface their **capability manifest for consent before first run**; capability tokens are HMAC-scoped like `dataRef`. +**Broker bounds (anti-DoS / anti-cost).** Because a capability call can be +emitted from a `tick`/`timer`, Tier 2 bounds **egress** the way Tier 1 bounds +compute (§0.2): per-capability **rate + quota**, a **max-in-flight** ceiling, +**idempotent de-duplication** of identical in-flight calls, and **backpressure** +when a broker saturates — so a ticking Lumen cannot move the DoS or cost problem +onto Tier 2/3. Caps are spike-tunable initial defaults (§14). Egress that +carries data **derived from Lumen state or a `DataRef`** (an outbound `fetch`, a +`writeData`) is treated as `external-effect` — per-call confirmation — *unless* +the endpoint **and** request shape were pre-approved at grant time; a bare +`internal` `fetch` may not smuggle state-derived data past the confirmation gate. +The exact quota/idempotency/backpressure contract is a spike deliverable (§14). + ### 6.1 Asset transport & content-addressed caching (never-stale) All binaries (images, audio, video, tiles, voice) travel as **`DataRef`s** @@ -346,22 +392,32 @@ A Lumen is a node in the same primitive tree; elements interact bidirectionally, deterministically, on Tier 1 (`interactivity-concept.md` §9.1). ```ts -type PortSpec = { name: string, dir: 'in'|'out', type: PortType }; // on primitives & Lumens -type Wire = { from: { ref: TargetRef, port: string }, to: { ref: TargetRef, port: string } }; // at container/canvas level +type PortSpec = { name: string, dir: 'in'|'out', type: PortType }; // on primitives & Lumens — explicit, wired +type ExposeSpec = { name: string, type: PortType }; // published read-only view-state, bindable by shared id WITHOUT a wire +type Wire = { from: { ref: TargetRef, port: string }, to: { ref: TargetRef, port: string } }; // at container/canvas level ``` -- **Shared selection / view-state.** A `table` and a Lumen referencing the same - `DataRef` + stable IDs share `viewState.selection` (keyed by id) — selecting in - one highlights in the other. Tier-1, no turn. +- **Shared selection / view-state — via a published interface, not ambient + reach.** An element declares a **lightweight published interface**: a small, + named, read-only set of view-state it *offers* to neighbours (`expose`, e.g. + `selection`, `viewport`). A neighbour referencing the same `DataRef` + stable + IDs may then read a **published** field by name — selecting in a `table` that + exposes `selection` highlights the bound markers in a Lumen, with **no explicit + wire and no turn** (Tier-1). The producer decides what is readable; + **un-exposed state stays private**, so an imported or untrusted element can + neither observe a neighbour's internals nor expose more than it declared. This + is *ambient-by-declaration*, not ambient-by-default. - **Wires** route a node's typed `out` port to another's `in` port by stable id; the host resolves and propagates at Tier 1 (Class A). Examples: table selection → map highlight; slider → sim `state` input; a game's `game-over` → `status` text + `writeData`. -- **Least-privilege.** A node reads **only** what is wired to it — it cannot - reach arbitrary other elements' internals. Wires are declared data, - whitelist-validated, resolve by stable id ⇒ deterministic, replayable, - shared-canvas-safe. Authority split unchanged (agent owns which wires exist; - client owns the values flowing). +- **Least-privilege.** A node reads **only** what is **wired or published** to it + (an explicit `wire`, or a neighbour's declared `expose` interface bound by + shared id) — it cannot reach arbitrary other elements' internals. Wires and + `expose` declarations are declared data, whitelist-validated, resolve by stable + id ⇒ deterministic, replayable, shared-canvas-safe. Authority split unchanged + (the agent owns which wires and published interfaces exist; the client owns the + values flowing). --- @@ -400,6 +456,15 @@ manifest for **consent before first run**. Determinism ⇒ every member runs the identical Lumen; per-user capability grants ⇒ a shared game saves *your* score, not *mine*. Real-time multiplayer is v2 but *unblocked* by determinism. +**Assets travel by content `id`, not by token.** A shared or preset Lumen +carries its asset references as content-addressed `DataRef` **ids** (or an asset +manifest) only — *never* the author's `signedToken`s, which are HMAC-scoped to +the author's `tenant ‖ user ‖ canvasSession` (§6.1) and would either fail for +the recipient or, if made reusable, break the isolation model. On +import/instantiate the recipient's Tier 2 **re-authorises and re-mints** each +`DataRef` token scoped to the *recipient*; an asset the recipient may not access +renders **inert**, never via a borrowed token. + --- ## 10. Visual treatment (Lume) @@ -421,11 +486,11 @@ is the transient 800 ms condensation (`visual-spec.md` §3.5). See | Risk | Mitigation | |---|---| | Arbitrary code in the renderer | None runs — LX is a validated AST walked by a shipped interpreter; CSP `default-src 'self'`, no `unsafe-eval` | -| Runaway / DoS | Gas + frame ceiling + bounded iteration + state cap → halt with `surface_error`, never the canvas | -| Data exfiltration | Default-deny capabilities; Lumen reads only own state + wired-in ports; all egress brokered, allowlisted, confirmed, Trace-audited | +| Runaway / DoS | Gas + frame ceiling + bounded iteration + state cap + **capped wakeup budget** (`tick` + `timer`, §4) → halt/reject with `surface_error`, never the canvas; capability **egress** is broker-bounded (rate/quota/max-in-flight/idempotent/backpressure, §6) so a tick-driven call cannot DoS Tier 2/3 | +| Data exfiltration | Default-deny capabilities; Lumen reads only own state + **wired/`expose`-published** ports; all egress brokered, allowlisted, confirmed, Trace-audited; **state/`DataRef`-derived** `fetch`/`writeData` classified `external-effect` (confirmed) unless pre-approved at grant (§6) | | Stale / poisoned assets | Content-addressed `DataRef` (id = content hash); HMAC-scoped fetch; explicit invalidation | | Untrusted shared/imported Lumen | Re-validated on import; capability manifest consent before first run; HMAC scoping; fork lineage | -| Cross-element overreach | Port/wire least-privilege; declared, whitelist-validated, stable-id-resolved | +| Cross-element overreach | Port/wire **+ published-interface** least-privilege — a node reads only what is **wired or `expose`-published** to it; un-exposed state is unreadable (an imported Lumen sees no ambient neighbour state); declared, whitelist-validated, stable-id-resolved | | Non-reversible effects | `external-effect` class → confirmation-modal gate before the real call | --- @@ -435,8 +500,9 @@ is the transient 800 ms condensation (`visual-spec.md` §3.5). See - **Tree content:** `behavior`/`lumen` section (§1) and the `scene` primitive (§3) — validated by the extended whitelist parser (schema + LX-AST). Carried in existing `surface_snapshot` / `surface_patch`. -- **Ports & wires:** `ports` on primitives/Lumens, `wires` at container/canvas - level (§7) — additive tree content, Tier-1-resolved. +- **Ports & wires:** `ports` and `expose` (published read-only interface) on + primitives/Lumens, `wires` at container/canvas level (§7) — additive tree + content, Tier-1-resolved. - **Cadence & animation:** `cadence` (§5) and `animate` descriptors — additive trait content. - **Events:** `surface_capability_request` (client→Tier 2) and @@ -449,7 +515,10 @@ is the transient 800 ms condensation (`visual-spec.md` §3.5). See `localOperations`. Classic channels and 1.0-only clients see none of this — all additive, behind -the `canvas` capability, unknown types ignored (`protocol/1.0.md` §0). +the `canvas` capability and **gated by boot negotiation** (§13): a client that +does not negotiate `scene` / LX / capability support is never sent 1.1 content, +and unknown *types / tree sections* are hard-rejected (not silently ignored) by +the 1.0 whitelist (`protocol/1.0.md` §2). --- @@ -477,10 +546,12 @@ accordingly and idioms degrade gracefully — the same principle as ## 14. Conformance & open questions -Conformance is the schema set in `schema/` (Lumen, LX-AST, scene, ports/wires, -capability manifest) + accept/reject fixtures, plus four reference Lumens -(an arcade game, interactive workflow, defrag-viz, map) traced end-to-end like -`walkthroughs.md`. Open tuning items (gas/frame/state caps, LX std-lib surface, -scene perf ceiling, capability-consent granularity, determinism-vs-real-time, -LLM reliability emitting LX, preset trust/distribution) are enumerated in +Conformance is the schema set in `schema/` (Lumen, LX-AST, scene, +ports/wires/`expose`, capability manifest) + accept/reject fixtures, plus four +reference Lumens (an arcade game, interactive workflow, defrag-viz, map) traced +end-to-end like `walkthroughs.md`. Open tuning items (gas/frame/state caps, +wakeup-budget caps for `tick`+`timer`, the capability-broker egress contract — +rate/quota/max-in-flight/idempotency/backpressure, LX std-lib surface, scene +perf ceiling, capability-consent granularity, determinism-vs-real-time, LLM +reliability emitting LX, preset trust/distribution) are enumerated in `interactivity-concept.md` §13 — research items, not unspecified holes. diff --git a/docs/mockups/kiosk-lumen-aura.html b/docs/mockups/kiosk-lumen-aura.html index 4aaa043..a1b0291 100644 --- a/docs/mockups/kiosk-lumen-aura.html +++ b/docs/mockups/kiosk-lumen-aura.html @@ -6,7 +6,7 @@ Omadia UI — Kiosk Lumen mockup (Lume · Lagoon)