diff --git a/CONCEPT.md b/CONCEPT.md index 67172ea..53f61c6 100644 --- a/CONCEPT.md +++ b/CONCEPT.md @@ -721,6 +721,7 @@ The system uses a small, disciplined set of light-domain names because Lume is t | **Trace** | the command history / audit log of a canvas — a "trace of light" through what the user asked and the agent answered | introduced v0.12 | | **Flare** | reserved — for agent-initiated attention signals (e.g. notifications, "needs your input") | reserved, may activate if and when we introduce notification concepts | | **Spark** | reserved — for a discrete generative-initiation event distinguishable from a beam | reserved, may activate if a use case clearly differs from Beam | +| **Lumen** | a self-contained, declarative, deterministic interactive unit — UI condensed into a portable, shareable quantum of light (the Live-Interactivity extension) | **proposed**, pending vocabulary sign-off — see [`docs/interactivity-concept.md`](docs/interactivity-concept.md) + [`docs/lumens-spec.md`](docs/lumens-spec.md) | Anything else from the light domain (Photon, Shadow, Beacon, Cast, Lens, Prism, Reflection, Ray, Shine, Glimmer, …) does **not** enter the vocabulary unless it earns its slot by labelling something that has no good name yet. The default answer to "should we call this X?" is no. @@ -1200,6 +1201,39 @@ This section becomes the input for the v2 design phase. It is not a v1 deliverab --- +## Extension — Live Interactivity (Lumens) + +A planned **additive** extension brings rich, agent-generated, **Tier-1-fast** +interactivity to the canvas — small games, interactive data workflows, +unusual visualisations (defrag-style), live maps — the Omadia answer to +sandbox-style "live artifacts", deliberately re-aimed. Its thesis: most of what +arbitrary-code sandboxes block is blocked by missing **capabilities**, not +missing **compute** — so it **constrains computation** (a declarative, bounded, +deterministic, interpreted behaviour model — no arbitrary code, the whitelist +parser extends to it) and **opens capabilities** (real data, write-back, +allowlisted network, generated assets) **mediated** through the existing Tier-2/3 +orchestration and effect classification. + +The unit is a **Lumen**: `state + transitions + view + events + capabilities`, +plus a new `scene` primitive (a declarative draw surface), declarative +ports/wires for cross-element interaction, per-region render cadence, and a +preset library so the agent **authors once and reuses constantly** (not rebuilt +per turn). It is **forward-compatible** — Lumens ride the same `surface_*` +grammar, the same DataRef/HMAC scoping, the same authority split, the same +shared-canvas hooks; the deltas are additive (a `behavior` tree section, the +`scene` primitive → `omadia-canvas-protocol/1.1`, one optional +`surface_capability_*` event family). + +- **Rationale / concept:** [`docs/interactivity-concept.md`](docs/interactivity-concept.md) +- **Normative definition / spec:** [`docs/lumens-spec.md`](docs/lumens-spec.md) +- **Lume visual treatment:** [`docs/visual-spec.md`](docs/visual-spec.md) §4.13 +- **Reference mockup:** [`docs/mockups/kiosk-lumen-aura.html`](docs/mockups/kiosk-lumen-aura.html) + +Not a v1 deliverable of this base concept; tracked as its own implementation +workstream. + +--- + ## Riskiest Assumptions 1. **Top-tier and fast LLMs can reliably emit valid primitive trees as tool-use JSON.** Likely yes for Sonnet/Opus; unproven for Haiku-class on UI-tree synthesis (omadia uses Haiku only as classifier today). Mitigation: `ui_orchestrator_model` configurable; spike with Haiku first, fall back to Sonnet if reliability below threshold. diff --git a/README.md b/README.md index 9032bbd..58abe75 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,13 @@ Omadia UI is the next layer — a desktop application where the agent - 🗂️ **A real workspace.** Multiple live canvases with their own server sessions, tiling split panes, named desktops, a canvas library — all synced through the omadia registry and restored on restart. +- 🎮 **Live, safe interactivity (Lumens).** A planned additive extension lets + the agent generate self-contained interactive units — a game, an interactive + workflow, an unusual visualisation, a live map — as **declarative, + deterministic data** run by a bounded Tier-1 interpreter (no arbitrary code), + with capabilities mediated through Tiers 2/3. Shareable, presettable. See + [`docs/interactivity-concept.md`](docs/interactivity-concept.md) + + [`docs/lumens-spec.md`](docs/lumens-spec.md). ## ⚡ Quickstart @@ -82,7 +89,9 @@ refactor). | File | Purpose | |---|---| | [`CONCEPT.md`](CONCEPT.md) | Architecture, primitives, protocol, security, identity, SDK extension plan | -| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.4) | +| [`docs/interactivity-concept.md`](docs/interactivity-concept.md) | **Live Interactivity (Lumens)** — concept/rationale: Tier-1-fast, agent-generated, safe interactivity (games, workflows, maps) | +| [`docs/lumens-spec.md`](docs/lumens-spec.md) | **Lumens** — normative definition: LX, `scene`, events/touch, capabilities, ports/wires, presets (`omadia-canvas-protocol/1.1` draft) | +| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.5) | | [`docs/protocol/1.0.md`](docs/protocol/1.0.md) | Protocol specification + machine-validatable JSON Schemas | | [`docs/walkthroughs.md`](docs/walkthroughs.md) | Use-case walkthroughs — multi-source comparison + editor micro-task | | [`docs/tech-stack.md`](docs/tech-stack.md) | Tech-stack decision for the host app (Electron), with reasoning | diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 446d92a..dee6b90 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -699,6 +699,33 @@ tied to the phase that exposes each one. --- +## 10. Lumens (Live Interactivity) extension — implementation outline + +A separable, **additive** workstream on top of the v1 baseline above (it +depends on the surface event family, the validator, and the orchestrator +already landing). Concept: [`interactivity-concept.md`](interactivity-concept.md); +normative definition: [`lumens-spec.md`](lumens-spec.md). Tracked for the +maintainer as a GitHub issue. Suggested phasing, smallest mergeable units first: + +| Phase | Deliverable | Where | +|---|---|---| +| **L0 — schemas** | JSON Schemas for the Lumen `behavior` section, the LX-AST node set, `scene`, `ports`/`wires`, the capability manifest; accept/reject fixtures | `docs/protocol/schema/` (this repo) → `omadia-canvas-protocol/1.1` | +| **L1 — LX interpreter (Tier 1)** | deterministic AST evaluator with gas + frame ceiling, bounded iteration, seeded `random`/`now`; the extended whitelist validator | host app `app/src/renderer/` | +| **L2 — `scene` primitive** | draw-list rasteriser (canvas2d first, WebGL behind it), token-only styling, buffer-native hit-testing → `TargetRef` | host app renderer | +| **L3 — cadence & animation** | per-region `static`/`reactive`/`{tick}` dirty-tracking + rAF scheduling; declarative `animate` layer on the Lume effect vocabulary; reduced-motion | host app renderer | +| **L4 — events & touch** | `tap`/`longPress`/`drag`/`pinch`/`swipe`/`key`/`tick`; 44 pt hit-targets; host gesture arbitration; input-modality handshake fields | host app + channel | +| **L5 — capabilities broker (Tier 2)** | `persist`/`loadData`/`writeData`/`tiles`/`fetch`/`generateAsset`/`clipboard`; effect-classified brokering + confirmation gate; `surface_capability_*` events; asset transport + content-addressed cache | `omadia-ui-orchestrator` + `omadia-ui-channel` + `byte5ai/omadia` core connectors for `generateAsset` | +| **L6 — ports & wires** | typed ports on primitives/Lumens, Tier-1 wire resolution, shared `viewState.selection` cross-element | host app + orchestrator | +| **L7 — lifecycle & presets** | author-once/patch; `lumen-presets/**` + `lumen-state/**` stores; resolve-then-generate lookup; fork+patch; behaviour-idiom library in the UI Skill | `omadia-ui-orchestrator` | +| **L8 — sharing** | `canvasOwnership` group extension + channel fan-out + import consent (rides the v2 shared-canvas hooks) | channel + orchestrator | +| **L9 — reference Lumens** | an arcade game · interactive workflow · defrag-viz · map, traced end-to-end like `walkthroughs.md`; conformance fixtures | this repo | + +**Riskiest items** (mirror `interactivity-concept.md` §13): LLM reliability +emitting valid LX (likely a strong-model authoring job, fast-model patching); +gas/scene-perf calibration on the four reference Lumens; capability-consent UX. + +--- + ## Appendix — corrected critical-file map (omadia core @ `83ef79b`) The concept's "Critical files" section with line numbers refreshed against live diff --git a/docs/interactivity-concept.md b/docs/interactivity-concept.md new file mode 100644 index 0000000..db3a263 --- /dev/null +++ b/docs/interactivity-concept.md @@ -0,0 +1,959 @@ +# Omadia UI — Live Interactivity Concept (Lumens) + +> How Omadia UI gains rich, agent-generated, **Tier-1-fast** interactivity — +> small games, interactive data workflows, unusual visualisations (think the old +> HDD-defragmenter view), live maps — **without** giving up the whitelist / +> no-arbitrary-code security model that makes the canvas safe. +> +> This is the Omadia answer to Claude's Live Artifacts, deliberately +> re-aimed: more capable where Live Artifacts is uselessly limited, and +> *structurally* safer than "arbitrary code in a sandbox". +> +> This is the **rationale** (the *why*, narrative); its **normative +> definition** (the *what* — types, grammar, contracts) is the companion +> [`lumens-spec.md`](lumens-spec.md). Further companions: `CONCEPT.md` (the +> canvas), `visual-spec.md` §4.13 (Lume rendering of Lumens), +> `docs/protocol/1.0.md` (the base wire format). This document is **concept +> only** — no implementation, no PR +> plan. It extends, and stays inside, the architecture in `CONCEPT.md`. + +Version 0.8 — **colour authority.** A Lumen's *own content* is **not** +palette-locked: the agent picks `colorMode: 'theme'|'brand'|'free'` (+ a declared +`palette`) from the **request + embedding context** (`lumens-spec.md` §3.1). +`theme` is only the *no-direction default* — justified by assuming the Lumen +embeds in an existing Lume UI, which is **not universal**; a user-built kiosk / +branded-ordering / product surface gets `brand`/`free` directly, using **any** +colour. Scoped to the Lumen's subtree: **Omadia chrome always stays Lume** (v1 +identity boundary, no host white-label). In brand/free the normaliser stops clipping and enforces no +contrast floor — accessibility of free-colour content is the author's +responsibility (44 pt hit-targets + reduced-motion still apply; interaction- +safety, not colour). Brand colour may still ride the Lume material (glow) or +render flat. +Version 0.7 — **expressiveness & practice-fit pass** (stress-tested by trying to +hand-write board-game-class Lumens — Tetris/Pacman as thought experiments — in +actual LX-AST). Closes three normative gaps the prose hid: the +`map`/`filter`/`fold` **binder forms** and computed-index **`at`/`setAt`** +(`lumens-spec.md` §2.2) — without them no iteration or board mutation is +expressible. Adds **native kernels** (§2.6): bounded, host-owned algorithms +(sort, group/aggregate, scale, layout, pathfind, …) that pure first-order LX +cannot express, exposed as pure calls — the *capability pattern applied to +compute*, so the no-arbitrary-code guarantee is untouched. Adds **declared +invariants** (§2.7, silent-wrong → loud-error) and a **golden-trace authoring +gate** (§14). Adds **transactional / high-frequency patterns** — +local-first/commit-once, session-scoped consent, optimistic+reconcile (§6.3) — +so kiosk/ordering flows fit without a modal per tap, plus a fifth (transactional) +reference Lumen. The takeaway: games stress the *compute* axis; ordering/kiosk +**transactions** stress the *capability* axis — different problems, so a working +game does not by itself validate a transactional business artifact. +Version 0.6 — **Codex-review fixes** (adversarial review of the 0.5 branch). +**Capability authority**: the agent owns capability *requests*, never *grants* +— a grant is Tier-2 policy + user consent, an agent patch can ask but not +self-grant (`lumens-spec.md` §0.5). **Additivity made fail-closed**: 1.1 content +is *negotiation-gated* (a 1.0 client is never sent it, and unknown types are +hard-rejected, not silently ignored — `lumens-spec.md` §0/§12). **Bounded +wakeups**: `timer` is capped like `tick` via a combined per-Lumen wakeup budget +(`lumens-spec.md` §0.2/§4). **Broker egress bounds**: per-capability rate/quota, +max-in-flight, idempotency and backpressure, plus state/`DataRef`-derived +`fetch`/`writeData` classified `external-effect` (§13.8; `lumens-spec.md` +§6/§11). **Shared assets**: travel by content `id`; Tier 2 re-mints `DataRef` +tokens for the recipient, inaccessible assets render inert (§7). **Cross-element +reads**: ambient-by-default selection replaced by a declared **`expose`** +read-only interface (ambient-*by-declaration*) — un-exposed state stays private, +imported Lumens observe no ambient neighbour state (§9.1). **LX versioning**: +LX ships as `LX/1.1`, declared at the handshake (§13.2). +Version 0.5 — expands §9 with §9.1 "Composition & cross-element interaction": +a Lumen is just a node in the same primitive tree (composes in +container/pane/grid/tabs; a Lumen's `view` can hold primitives and vice +versa). Elements interact **bidirectionally and deterministically on Tier 1** +via shared `viewState.selection` by stable id and declarative **ports & wires** +(typed inputs/outputs, `from`→`to` bindings), with a least-privilege boundary +(a node reads only what is wired to it). Authority split unchanged. +Version 0.4 — adds §8 "Lumen lifecycle & reuse — author once, then instantiate +& patch (never rebuild)": the Omadia counterpart to Claude's artifact logic. +Author once (`surface_snapshot`), persist in canvas-state, edit by **targeted +`surface_patch`** (not regeneration, not string-replace); a **preset library** +(first-party / tenant / user / canvas scopes, content-addressed, versioned, +parameterised) so reuse is **instantiate-without-an-LLM**; a **two-speed** +model (rare cold authoring vs. constant warm reuse); **resolve-then-generate** +lookup before any build; **fork+patch** with lineage for variations; a +behaviour-idiom library so even cold builds assemble vetted fragments. +Version 0.3 — review fixes. **Lume correctness**: the motion/effect vocabulary +is now exact to `visual-spec.md` — Lume is **light-as-material** (surface +luminosity, accent-as-illumination, directional borders, soft corners), *not* +glassmorphism; the erroneous "frosted glass / backdrop-blur" effects are +removed (the only blur is the transient 800 ms condensation). **Touch & pointer +input** added to §5 as first-class (tap/longPress/drag/pinch/swipe, 44 pt +hit-targets, host gesture arbitration, no hover dependency, input-modality +handshake) for kiosk/iPad. **Assets** §6.1 — transport + **content-addressed, +never-stale caching** (`id = kind-sha256(content)`, cache-busting by +construction, HMAC-signed fetch, explicit invalidation). **Generated material** +§6.2 — images/sounds/voice come from **omadia-core LLM connectors** (Tier 3), +the Lumen only requests (`generateAsset`) and renders; nothing generated on the +client. +Version 0.2 — adds §5 "Render cadence, motion & the thin-client / kiosk +envelope": cadence is declared per region (`static` / `reactive` / `tick`), +**reactive-by-default (~0 % CPU at rest)** — 60 Hz never applies to the whole +tree; presentation motion is a **declarative, GPU-run animation layer** +(transitions, parallax, Ken-Burns, particles, glow), distinct from LX +simulation ticks; the "wow" comes from existing assets + generated layout + +native effects; plus the thin-client/kiosk capability ladder and a mapping to +the four priorities (security · performance · visuals · wow · generative). +v0.1 — first draft. Introduces the **Lumen**: a self-contained, declarative, +deterministic interactive unit that runs in a bounded interpreter on Tier 1, +is generated and brokered agentically on Tiers 2/3, and is safe to share and +to save as a preset *because* it is data, not code. + +--- + +## 0. The problem with the two existing answers + +There are two ways the industry currently lets an LLM produce "an +interactive thing", and both are wrong for Omadia: + +| Approach | What it does | Why it fails for us | +|---|---|---| +| **Omadia today** (24 declarative primitives + whitelist parser) | Agent emits a JSON tree of fixed primitives; Tier 1 renders it. Safe, beautiful, instant. | Cannot express a *game loop*, *custom per-frame rendering*, *local rules/physics*, or *novel visualisations*. There is no primitive for "a falling tetromino", and there never should be — you cannot enumerate every interactive idea as a primitive. | +| **Claude Live Artifacts** (arbitrary React/JS in a sandboxed iframe) | Agent writes real code; a sandbox runs it. Maximally expressive *in compute*. | The sandbox is **all-or-nothing and near-empty**: no real data, no network, no persistence, no host integration. The simplest real use cases ("load map tiles", "save my high score", "act on my Jira data") are blocked — not by missing compute, but by missing **capabilities**. And the only thing standing between the code and the user's machine is that the sandbox boundary holds. | + +The key diagnosis — and the thesis of this document: + +> **Most of what Live Artifacts blocks is blocked by missing *capabilities*, +> not missing *compute*.** A game loop does not need network. A map needs *tiles*, +> not arbitrary JS. A data workflow needs *your data and a write-back path*, +> not a Turing-complete escape hatch. +> +> So the smart trade is the inverse of Live Artifacts: **constrain the +> computation** (make it declarative, bounded, deterministic, interpreted — +> not arbitrary code) and **open the capabilities** (real data, tools, even +> network) — but **mediated, declared, and gated** through the Tier-2/3 +> orchestration Omadia already has. + +That is the whole concept in one paragraph. Everything below is the +mechanics. + +--- + +## 1. The Lumen — definition + +A **Lumen** is a self-contained interactive unit on the canvas. It is the +omadia-native equivalent of a Live Artifact, but it is **declarative data**, +not code. A Lumen has exactly four declared parts, plus an optional fifth: + +```ts +type Lumen = { + state: StateSchema; // typed, bounded, serialisable state + transitions: Record; // pure (state, event) -> state, in Lume Expressions + view: ViewExpr; // pure state -> primitive/scene tree, in Lume Expressions + events: EventBinding[]; // declared inputs -> transitions (keys, pointer, tick, timer) + capabilities?: CapabilityRequest[]; // declared, default-deny doors to the outside world +}; +``` + +Plain-English mapping: + +- **`state`** — the Lumen's memory. For a game: the board, the moving + token(s), score, level, game-over flag. Typed and size-capped. +- **`transitions`** — the rules. Pure functions `(state, event) → newState`, + written in **Lume Expressions** (a small, total, sandboxed expression + language — §3). For a game: `tick` advances the board, `moveLeft`, `rotate`, + `lockPiece`, `clearLines`. No I/O, no host access, deterministic. +- **`view`** — the look. A pure function `state → tree`, producing either + ordinary Omadia primitives (so a Lumen can be a live form/table/dashboard) + **or** a `scene` (the new immediate-mode draw surface, §4) for custom + visuals like a game board or a defrag grid. +- **`events`** — what wakes the Lumen: declared keys (`ArrowLeft`, `Space`), + pointer events on scene elements, a host **tick** at a declared, capped + rate (≤ 60 fps), or a timer. Input is whitelisted — a Lumen cannot listen + to keys it did not declare. +- **`capabilities`** — the *only* way a Lumen touches anything outside its + own state: persistence, data fetch, map tiles, clipboard, share. Default + deny. Each is effect-classified and brokered by Tier 2 (§6). + +Because all five parts are declarative data validated by the same kind of +whitelist parser Omadia already ships, a Lumen flows through the **existing** +`surface_snapshot` / `surface_patch` grammar as ordinary tree content. **No +new transport is required for the Lumen itself** — only the validator and the +Tier-1 runtime are new. + +> **Naming (light-vocabulary discipline, per `CONCEPT.md` §"Light +> Vocabulary").** "Lumen" — the SI unit of luminous flux — is *proposed*, not +> locked: a Lumen is a discrete, self-contained, portable quantum of Lume. +> The term labels something genuinely new (a shareable, self-running unit of +> the material) and makes it more thinkable. Per the discipline, it needs +> sign-off before it enters the locked vocabulary; until then the descriptive +> fallback is "live canvas unit". The reserved word **Spark** ("a discrete +> generative-initiation event") is *not* this — a Spark would be the *act* of +> the agent condensing a Lumen, distinct from the Lumen itself. + +--- + +## 2. Why this is structurally safer than a sandbox (the security thesis) + +The Omadia security moat today is the **whitelist parser**: the renderer +never executes agent input, it validates it against a closed schema and +renders only known primitives (CSP is `default-src 'self'`, no `unsafe-eval`; +the Ajv validators are pre-compiled at build time exactly so raw `Function()` +compilation never happens — see `app/src/renderer/src/validate/`). A Lumen +must not punch a hole in that moat. It does not, because: + +1. **No arbitrary code, ever.** A Lumen carries *data* (state schema, + expression ASTs, view template, event bindings). The Tier-1 runtime is an + **interpreter shipped inside the Host App**, not `eval`. Lume Expressions + are parsed to a validated AST and walked by a deterministic evaluator. CSP + stays `default-src 'self'`, no `unsafe-eval`, no iframe-with-script. The + whitelist philosophy is *extended*, not *abandoned*: unknown expression + node → reject; unknown event type → reject; unknown capability → reject. +2. **Bounded computation — cannot hang, cannot DoS the host.** Every + transition and every view evaluation runs under a **gas budget** + (instruction count) and a **wall-clock ceiling** per frame. Loops are + bounded comprehensions (`map`/`fold` over *declared* collections) only — + no open `while`, no unbounded/​unguarded recursion. State has a size cap. + Frame rate is capped by the host. A pathological Lumen hits its budget and + is **halted with a `surface_error`**; it can never freeze the canvas or + starve other Lumens. (Contrast: a sandboxed `while(true)` can wedge a + worker.) +3. **Determinism by construction.** All non-determinism is host-supplied and + seeded: `random` is a seeded PRNG handed in by the host; `now`/tick + timestamps come from the host clock. Given `(state, event)` the next state + is identical on every machine. This single property buys four things at + once: **replay** (Trace can replay a session), **undo/redo** (navigate the + state history), **safe sharing** (a recipient runs the *identical* + behaviour), and **v2 multiplayer** (deterministic ops are lockstep- and + CRDT-friendly — it rides the `treeRevision`-as-opaque-id and shared-canvas + hooks already reserved in `CONCEPT.md`). +4. **Capability default-deny, effect-classified, agent-brokered.** A Lumen + reaches the outside world only through declared capabilities, and each + capability call reuses the existing `local` / `internal` / `external-effect` + classification and the **confirmation-modal contract** from `CONCEPT.md` + §"Security Surface". The Lumen never talks to the network, the filesystem, + or a tool directly — it *requests*, Tier 2 *brokers*. (Contrast: Live + Artifacts' iframe either has a capability or does not, with no agent in the + loop to scope, confirm, or audit it.) +5. **Origin & trust for imported Lumens.** A shared or preset Lumen carries a + **capability manifest** that is surfaced to the receiving user *before + first run* ("This shared canvas wants to: save high scores · load map + tiles from OpenStreetMap. Allow?"). Capabilities are HMAC-scoped exactly + like `dataRef` (`HMAC(serverSecret, tenant ‖ user ‖ canvasSession ‖ …)`), + and external capabilities carry a provider allowlist. An imported Lumen + with an un-grantable capability is rendered inert, never silently + escalated. + +The net: a Lumen is **more capable** than a Live Artifact (it can reach real +data, tools, and — when declared and granted — the network) while being +**less dangerous**, because the dangerous axis (arbitrary code execution) is +removed entirely and the powerful axis (capabilities) is mediated by the +agent and the user instead of by a sandbox boundary alone. + +| | Claude Live Artifacts | Omadia Lumens | +|---|---|---| +| Compute model | arbitrary JS/React | declarative + bounded interpreter (no `eval`) | +| Security boundary | iframe sandbox must hold | nothing to escape — no code runs; capabilities gated | +| Real data access | essentially none | yes, via brokered capabilities + existing DataRef/Class-D | +| Network | blanket-denied | declared, provider-allowlisted, brokered (e.g. map tiles) | +| Persistence | none | `persist` capability → `memoryStore` | +| Rendering | foreign iframe, off-theme | native Lume, on-theme, 60 fps | +| Agent can read/modify it live | no | yes — state is structured, agent can introspect & live-patch | +| Safe to share / preset | not really (it's code) | yes — it's validated, deterministic data | +| Can hang the host | yes (`while(true)`) | no (gas + frame ceiling → halt) | + +--- + +## 3. Lume Expressions — the bounded language + +The transition and view functions are written in **Lume Expressions** (LX): +a small, **pure, total** expression language. It is deliberately *not* +Turing-complete in the dangerous direction. + +- **Pure & total.** Every expression is a function of its inputs with no side + effects. No statements, no mutation, no exceptions-as-control-flow. +- **Values.** numbers, booleans, strings, lists, records, and typed `state` + references. No closures over host objects, no `this`, no prototypes. +- **Operators & built-ins.** arithmetic, comparison, boolean logic, + `if`/`match`, record/list construction, and a fixed standard library + (`map`, `filter`, `fold`, `range`, `min`, `max`, `len`, string ops, a small + math set). The standard library is a **whitelist** — same discipline as the + primitive vocabulary. +- **Bounded iteration only.** `map`/`fold`/`range` over collections whose size + is bounded by `state` (itself size-capped). No `while`, no general + recursion. This is what makes the gas bound a *static* guarantee rather than + a hope. +- **Host-provided non-determinism.** `random()` and `now()` are not free + functions; they read host-seeded values passed into the evaluation context, + preserving determinism/replay. +- **Validated AST.** LX is delivered as a JSON AST (not source text), so there + is no parser-injection surface and the validator is the same shape as the + primitive whitelist: walk the tree, reject any unknown node type. + +LX is **versioned** alongside `omadia-canvas-protocol` and negotiated at the +boot handshake (the client declares the LX version and gas limits it +supports, exactly as it already declares `localOperations`). + +> **Why not WASM / QuickJS?** A hardened WASM or embedded-JS sandbox is the +> obvious "powerful escape hatch" and is noted here as a possible **v2+** +> capability for the rare Lumen that genuinely needs arbitrary compute. It is +> *not* the v1 answer because it re-introduces exactly the "a boundary must +> hold" risk this concept is built to avoid, and it breaks determinism/replay/ +> shareability. v1 deliberately ships the declarative model first and measures +> whether real use cases actually exceed it. (Same "radical-restraint-first, +> measure, then add" discipline `CONCEPT.md` uses for the prompt bar.) + +--- + +## 4. The `scene` primitive — custom visuals without custom code + +The 24 primitives cover data/UI; they do not cover "draw me a board of +coloured cells at 60 fps". One new editor-class primitive closes that gap: + +**`scene`** — a declarative, immediate-mode draw surface. The Lumen's `view` +function emits, each frame, a **draw-list** from a whitelisted shape +vocabulary: + +| Shape | For | +|---|---| +| `rect` / `roundRect` | tetromino cells, defrag blocks, bars, map markers | +| `line` / `polyline` / `path` | grids, connections, routes (reuses `vector-path` geometry) | +| `circle` / `ellipse` | nodes, dots, map pins | +| `sprite` (a `DataRef` image) | tiles, icons, game art | +| `text` | scores, labels (rendered in Lume type registers) | +| `group` / `transform` | layers, camera pan/zoom (reuses canvas-region zoom/pan affordances) | + +Properties carry geometry plus a colour the agent chooses from intent. The +**no-direction default** is theme tokens + Lume palette — justified by the +assumption the Lumen embeds in an existing Lume UI (so a game looks like Omadia, +not a foreign website). That assumption is **not universal**: a user's kiosk or +product surface needs *their* colours, and the agent then picks a brand/free +palette for the Lumen's **own content** directly (`lumens-spec.md` §3.1) — Omadia +chrome always stays Lume (the host stays recognisable). The draw-list is +**data**, validated by the whitelist parser — there is no canvas `2d`/`webgl` +script handed to the agent. Tier 1 +rasterises the draw-list to canvas/WebGL natively at 60 fps from local state; +this is pure **Class A** interaction — *zero* server contact for the frame +loop (`CONCEPT.md` §"Latency paths"). + +`scene` is distinct from the existing `canvas-region` (which is a *pixel +editor buffer* for brush/blur ops). `scene` is a *retained/immediate hybrid +render target driven by Lumen state*. Both can coexist. Pointer events on +scene elements bind to Lumen events via stable element ids, so the existing +`TargetRef` and beam model work *inside* a Lumen (you can beam the agent about +a specific marker or cell). + +--- + +## 5. Render cadence, motion & the thin-client / kiosk envelope + +Nothing about a Lumen forces 60 Hz. A blanket game-loop is the *wrong* default +for almost everything and ruinous for an always-on kiosk. This section pins +the rendering model and the realistic capability envelope on weak hardware. + +### Reactive by default — 60 Hz only where it earns it + +Cadence is declared **per node/region, not globally**, in three classes: + +| Cadence | When it runs | LX cost at rest | For | +|---|---|---|---| +| **`static`** | rendered once; redrawn only when a `surface_patch` changes it | **zero** | most kiosk/dashboard content: layout, imagery, copy, KPIs | +| **`reactive`** *(default)* | `view` re-evaluated only for the sub-tree whose `state` slice changed, on event/data | **zero** until something changes | forms, tables, controls, selection | +| **`{ tick: hz }`** | host clock drives an LX transition at a declared, capped rate, **scoped to that sub-tree only** | one bounded transition/frame for that region only | the falling tetromino, a live chart, a defrag animation | + +The runtime **dirty-tracks** which `state` slices changed and re-evaluates only +the dependent `view` branches (retained-mode + memoisation). +`requestAnimationFrame` is scheduled **only while a ticking/animating region is +live** and torn down when it settles. **At rest a Lumen costs ~0 % CPU** — a +kiosk showing a beautiful, mostly-static screen burns nothing until someone +touches it or a single badge pulses. One Lumen routinely mixes all three: a +a game's scene ticks at 60 Hz, the score label beside it is `reactive`, the +surrounding chrome is `static`. Only the part that must move pays. + +### Motion comes from a declarative animation layer, not from LX + +"Animation" conflates two different things; separating them is what makes +*wow on weak hardware* possible: + +- **Simulation** — state genuinely evolves by rules each step (a piece falls, + cells flip). This is an LX **tick**, used sparingly. +- **Presentation motion** — a panel slides in, a glow pulses, a number counts + up, a camera eases, an image slowly pans (Ken Burns). This must **not** be an + LX tick recomputing state per frame. It is a **declarative animation** the + host runs on the compositor/GPU: + + ```json + { "animate": { "property": "opacity", "from": 0, "to": 1, "duration": 300, "easing": "ease-out" } } + ``` + +The agent *declares* enter/exit/change transitions, easing, pulses, parallax +layers, Ken-Burns pan-zoom on assets, **accent glow / halo / aura** (the +two-stop and donut glow recipes), **surface luminosity**, **directional +light**, the **patch-condensation** materialisation, and (as a native effect) +light-mote particle emitters. The **host executes them natively on the GPU** — +zero LX per frame, 60 fps smoothness on a fanless thin client. This *is* the +Lume material — **light-as-material**, condensed out of light: surface +luminosity, accent-as-illumination, directional borders, soft corners +(`visual-spec.md` §1.2). **Lume is explicitly *not* glassmorphism** — no +refraction, no blur-as-chrome, "solid light, not see-through plastic" +(`visual-spec.md` §1.3). The only blur is the transient 800 ms condensation +materialisation, never standing chrome. So the wow is cheap and on-rails, +never hand-coded pixel math. + +### Where the "wow" actually comes from (generated, natively executed) + +A stunning generated kiosk screen = **existing assets + generated layout + +native Lume effects + a touch of declarative motion**: + +- **Existing image/video material** → `sprite`/`image`/`media` via `DataRef` + (brand imagery, product photos, loops). The agent *composes*; it does not + synthesise pixels on-device. +- **Native Lume effect vocabulary** (whitelisted, GPU): surface-luminosity + gradients, accent glow / halo / aura (two-stop + donut), `glow-core` inner + light, directional light, elevation, parallax depth, Ken-Burns pan-zoom, + light-mote particles — *declared*, not computed in LX. No glassmorphism, no + blur-as-chrome (`visual-spec.md` §1.3). +- **Generative authorship, native execution**: the agent generates the + composition, the motion declarations, and the asset bindings as data; the + host runs them on native rails. The result looks like a hand-tuned demo, + but it was generated. + +### Touch & pointer input — first-class, not an afterthought (kiosk · iPad) + +Kiosks and iPads are touch-first, so touch is in the Lumen event model from +day one — it is **not** mouse events with a shim. A Lumen's `events` declare +**pointer-semantic** inputs that resolve identically across mouse, trackpad +and touch (the same abstraction `CONCEPT.md` already uses for *context-invoke* += long-press): + +| Declared event | Touch | Mouse/trackpad | Use | +|---|---|---|---| +| `tap` | tap | click | activate a control, place, select | +| `longPress` (~400 ms) | press-and-hold | right-click / hold | **context-invoke** → action panel + Beam (per `CONCEPT.md`) | +| `drag` | one-finger drag | press-move | move a piece, pan a board, reorder | +| `pinch` | two-finger pinch | ctrl+wheel / trackpad pinch | zoom a map / scene | +| `swipe` | flick | wheel / two-finger | next/prev, dismiss, scroll | +| `pointerMove` *(opt-in)* | finger track | hover | drawing, aiming — `continuous-input` | + +Rules that make it kiosk-grade: + +- **Hit-targets, not pixels.** A Lumen declares interactive scene elements + with a **minimum 44×44 pt hit area** (Apple HIG), independent of the drawn + glyph size. The runtime enforces the minimum and does hit-testing against + stable element ids → the existing `TargetRef`/beam model works by touch. +- **Gesture arbitration is the host's job**, reusing `CONCEPT.md`'s long-press + arbitration (move > 6 px before 400 ms ⇒ drag, else context-invoke). A Lumen + never re-implements gesture disambiguation. +- **No hover dependency.** Hover is dropped as a *required* affordance (not + touch-capable); any hover effect is pure decoration with a tap/long-press + equivalent. This matches `CONCEPT.md`'s interaction model. +- **Touch-tuned density.** `style: "spacious"` and larger Lume hit-areas are + the kiosk default; the agent is told (UI Skill) to compose touch-first when + the canvas is flagged as a kiosk/tablet surface. +- **On-screen input.** Text entry uses the platform soft-keyboard via the + `input` primitive; a kiosk with no keyboard still works. No raw key events + are *required* — declared keys (`ArrowLeft`, `Space`) are an *enhancement* + for hardware-keyboard hosts, and every key-driven action has a touch + equivalent (on-screen control) when the host reports no keyboard at + handshake. + +The handshake already carries client capabilities; it is extended to report +**input modalities** (`touch` / `mouse` / `keyboard` / `pen`) so Tier 2 +composes the right affordances — a kiosk Lumen ships on-screen controls, a +desktop Lumen may add keyboard shortcuts on top. + +### Thin-client / kiosk capability ladder + +Because the client only ever does *bounded interpretation* + *raster of a +draw-list it already has* (everything heavy is brokered to Tier 2/3), the +thin client is the design centre, not the stress test: + +- 🟢 **Flüssig:** puzzle/board/card games (2048, chess, solitaire), interactive + dashboards & workflows, data-viz incl. defrag-style grids to ~5–10 k cells + (canvas2d; more on WebGL), maps (tiles are GPU-composited images fetched by + Tier 2/3 — the client doesn't compute the map). +- 🟡 **Mit Maßnahmen:** large cellular automata / sims over big grids (drop to + 30 Hz, smaller grid, or push the step to Tier 3), very large scenes (WebGL + rasteriser, same declarative draw-list). +- 🔴 **Nicht in reinem LX:** realtime 3D, thousand-body physics, 100 k-particle + systems, per-pixel image processing, heavy solvers/ML. → escape hatches: + **native local-ops catalog** (Class B, pixel work), **Tier 3** (heavy + compute returns a `DataRef` the Lumen merely visualises), **v2 WASM** (hard + gated, rare outliers). + +The decisive kiosk property no "arbitrary-code-in-sandbox" approach has: the +**gas budget guarantees a clean halt** — a badly generated Lumen never freezes +an always-on display, it is stopped with a `surface_error`. + +### Mapping to the four priorities + +| Priority | How the cadence/motion model delivers it | +|---|---| +| **Security** | host-owned capped clock; `static`/`reactive` branches execute *no logic* at rest; motion is declarative (no per-frame code); capability default-deny unchanged; gas guarantees a clean halt | +| **Performance** | ~0 % CPU at rest; only ticking/animating sub-trees cost frames; GPU does the pretty part — built for fanless always-on kiosks | +| **High-quality visuals** | native Lume effects + GPU compositing + real assets via `DataRef`; consistent, on-theme, never a foreign iframe | +| **Wow-effect** | declarative transitions, parallax, Ken-Burns, particles, glow — generated by the agent, run natively at 60 fps even on thin clients | +| **Generative** | the agent authors structure, motion and asset bindings as data; existing material is referenced, not regenerated | + +--- + +## 6. Capabilities — the mediated doors + +Capabilities are the heart of "better than Live Artifacts". Each is declared +in the Lumen, effect-classified, and brokered by Tier 2. Default deny. + +| Capability | Effect class | Broker path | Example | +|---|---|---|---| +| `persist(key, value)` | `internal` | Tier 2 → `memoryStore@1` under a Lumen-scoped namespace | a game's high score; map's last viewport | +| `loadData(dataRef)` | `internal` | Tier 2 hands the Lumen a **read-only, size-capped projection** of an existing `DataRef` | drive a defrag-style viz from a real dataset; bind a map to a places table | +| `writeData(target, value)` | `internal` / `external-effect` | reuses the **Class-D mutation contract** + `writeCapabilities` manifest | an interactive triage workflow that commits status changes back to Jira | +| `tiles(provider, z/x/y)` | `internal` | Tier 2/3 fetches from a **provider-allowlisted** endpoint, returns sprite `DataRef`s | OpenStreetMap / Mapbox map tiles | +| `fetch(declaredEndpoint)` | `internal` / `external-effect` | Tier 3 tool call against an **allowlisted, agent-approved** endpoint only | a live feed into a visualisation | +| `clipboard(text)` | `external-effect` | confirmation-modal gate | "copy result" | +| `generateAsset(spec)` | `internal` / `external-effect` | Tier 2 → **omadia-core LLM connectors** (Tier 3); returns a `DataRef` | generate an image / sound / TTS voice line for the scene | +| `share(lumen)` / `savePreset(lumen)` | `external-effect` | §7 | share a game with a colleague; save as a gallery preset | + +Mechanics: a capability call from the running Lumen is *not* a direct call. It +emits a capability-request action (effect-classified) up through the channel; +Tier 2 validates it against the Lumen's granted manifest, brokers it +(memoryStore / Tier-3 tool / allowlisted endpoint), and patches the result +back — or, for `external-effect`, raises the standard confirmation modal +first. **The Lumen's deterministic local loop keeps running** while a +capability call is in flight (async-by-default, `CONCEPT.md` +§"Async Architecture") — a map pans smoothly while new tiles stream in; a +workflow stays interactive while a write-back resolves. + +This is the precise inversion of Live Artifacts: the network is *not* banned, +it is **named, allowlisted, agent-brokered, user-confirmed, and audited in +Trace**. + +### 6.1 Assets — transport, content-addressed caching, never-stale + +Images, sounds, video, voice lines and any other binary an asset-bearing +Lumen references all travel as **`DataRef`s** — the canonical mechanism from +`CONCEPT.md` §"DataRef lifecycle". The key property the user asked for — +**no stale-cache problem** — falls out of `DataRef` being **content-addressed**: + +> `id = "-"` — `kind ∈ {pixel, audio, video, …}`. +> **Same bytes → same id. Different bytes → different id. Always.** + +This is **cache-busting by construction**, the structural fix for the +"annoying browser still shows the old image because it didn't notice it +changed" behaviour: the id *is* the content hash, so a changed asset is a +**different id** and there is no way to address new content with an old +reference. The client cache is keyed by that hash, so a cache hit is a +*provable* byte-identity, never a heuristic on a URL + `Cache-Control` guess. + +**Transport path (host → UI client):** + +1. **Origin.** A Tier-3 tool or an omadia-core LLM connector (see §6.2) + produces the binary and returns a `DataRef {id, signedToken, expiresAt}` — + the binary itself stays server-side until fetched. +2. **Announce.** Tier 2 emits `surface_data_ref_created {DataRef, schema, + sizeHint}` on the surface stream; the Lumen's `view` references the asset + by `dataRef` on a `sprite`/`image`/`media` node. +3. **Fetch.** The client fetches the bytes **once** from the channel endpoint + using the **HMAC-signed token** (scope = tenant ‖ user ‖ canvasSession ‖ + body ‖ expiry; re-validated server-side — `CONCEPT.md` §"Security Surface"). +4. **Cache.** The client stores the bytes in a local content-addressed store + keyed by `id`. Every later reference to the same `id` is an instant local + hit — across turns, across Lumens, across canvases. Dedup is automatic + (identical assets share one entry). +5. **Invalidate.** Two triggers, both explicit: `expiresAt` reached, or a + `surface_data_ref_invalidated {id, reason}` from Tier 2 when a durable op + replaces the buffer. There is no time-based "maybe it's stale" guesswork. +6. **GC.** The client drops a local buffer once no live primitive references + its `id` **and** its expiry has passed. + +Large client-authored buffers (a `canvas-region` the user painted) are +content-hashed **locally** and only uploaded if a Tier-3 op needs them — the +same content-addressing in the other direction. + +### 6.2 Generated material comes from omadia-core, not from the Lumen + +Generative assets — images, sounds, music, **synthesised voice** — are +produced by the **LLM connectors wired into the omadia host (omadia-core)**, +**never** by the Lumen or the Tier-1 client. The UI side only *supports* them: +it **requests** via the `generateAsset` capability and **renders** the +returned `DataRef`. The division of labour: + +| Layer | Role in asset generation | +|---|---| +| **Lumen / Tier 1** | declares the need (`generateAsset(spec)`), renders the resulting `sprite`/`media`, runs declarative motion (Ken-Burns, etc.) on it. Generates **nothing**. | +| **Tier 2** | validates the capability grant, shapes the request, brokers it to the right connector, caches the returned `DataRef`, patches it into the scene. | +| **Tier 3 / omadia-core connectors** | the actual image/audio/voice model. Owns the generation, the model choice, the cost, the rate limits. Returns a content-addressed `DataRef`. | + +This keeps the security model intact (no model keys or generation logic on the +client), keeps the protocol model-agnostic (swap connectors without touching +the Lumen), and means generated material flows through the **exact same +content-addressed, never-stale cache** as any other asset (§6.1). A +regenerated image is simply a **new `id`** — the scene updates, the old bytes +GC out, nothing stale lingers. + +--- + +## 7. Sharing & presets — safe because it's data + +Two user-facing features the user explicitly asked for, both essentially free +once a Lumen is declarative+deterministic+capability-manifested: + +**Share a canvas / Lumen with selected users.** A Lumen serialises cleanly +(it is validated data + a capability manifest). Sharing rides the +forward-compat hooks already in `CONCEPT.md` §"Forward Compatibility": extend +`canvasOwnership` from `{kind:"single-user"}` to `{kind:"group", members}`, +and the channel plugin (already the designated **fan-out point**) multicasts +surface events to connected members. The recipient's Tier 1 **re-validates** +the Lumen and shows its **capability manifest for consent before first run**. +Because behaviour is deterministic, every member sees the identical Lumen; +because capabilities are per-user-granted, a shared game can save *your* +high score without touching *mine*. Real-time multiplayer (two people in one +game) is a v2 topic but is *unblocked* by determinism — deterministic ops +are exactly what lockstep/CRDT need. + +Asset references travel as content-addressed `DataRef` **ids** (or an asset +manifest), **never** the author's HMAC-scoped tokens (§6.1, scoped to the +author's `tenant ‖ user ‖ canvasSession`); on import Tier 2 **re-mints** each +token scoped to the recipient, and an asset the recipient may not access renders +**inert** rather than reusing a borrowed token. + +**Save as a preset.** A Lumen can be named, optionally **parameterised** +(declare which parts of `state` are preset inputs), and stored in a preset +store (`memoryStore@1` namespace, e.g. `lumen-presets//`). A +**Lumen gallery** lets the user re-instantiate "my defrag-style project +viewer" or "the team standup board" on any canvas, with fresh data bound via +`loadData`. Presets are the natural unit for an eventual community/library of +Omadia interactive templates — and they are safe to distribute precisely +because they are validated, deterministic, capability-declared data, not code. + +--- + +## 8. Lumen lifecycle & reuse — author once, then instantiate & patch (never rebuild) + +Regenerating a kiosk surface or a game from scratch on every turn would be +expensive (tokens), slow (a full generation), and inconsistent (two "builds" +drift). It must not happen, and the architecture already has the pieces to +stop it. This is the Omadia counterpart to Claude's artifact logic +(create-once, then `update`) — extended from "one artifact per conversation" +to a **persistent, versioned, shareable component library**. + +### 8.1 Author once, edit by patch — never regenerate + +A Lumen is generated **once** (`surface_snapshot`) and then **lives in +canvas-state** (`CONCEPT.md` §"State Model"), persisting across turns and Host +App restarts. The agent does **not** re-emit it to keep it alive. Subsequent +changes are **targeted `surface_patch`es** addressed by stable id/path, +bumping `treeRevision`: + +| User says | What the agent emits | Cost | +|---|---|---| +| "Build me a game" *(nothing reusable exists)* | one `surface_snapshot` with the full Lumen | one expensive turn (once) | +| "Make it faster as the score climbs" | `surface_patch` touching only the `tick` transition | a few tokens | +| "Bigger board" | `surface_patch` on one `state` field + the `view` grid bounds | a few tokens | +| "Change the accent to Atelier" | accent re-tint patch (existing palette mechanic) | trivial | + +This is exactly Claude's create-then-update model — but a patch targets a +**validated tree by stable id**, not a string-replace on source code, so there +is no resync/drift surface and no risk of an edit corrupting the program. The +*structure is preserved by construction* across edits → consistency for free. + +### 8.2 The reuse library — presets & templates (the real "not from scratch") + +The durable answer is the **preset library** (§7). A vetted Lumen is authored +**once** and saved as a preset: **named, versioned, content-addressed, +parameterised**. Thereafter the agent does not build it — it **instantiates** +it: + +> "Instantiate the game" → Tier 2 resolves the intent to the `arcade@2` preset, +> binds its parameters (board size, palette, data), and renders it. +> **Near-zero LLM, sub-second, byte-identical every time.** + +Library scopes, outermost to innermost (first match wins, like CSS cascade): + +| Scope | Namespace | Holds | +|---|---|---| +| **First-party** | shipped with the orchestrator | curated, audited Lumens (an arcade game, map, defrag-viz, standup board) | +| **Tenant / org** | `lumen-presets//shared/**` | the organisation's vetted library | +| **User** | `lumen-presets///**` | "my defrag-style project viewer" | +| **Canvas-local** | canvas-state | the instance currently on screen | + +Because a preset is **content-addressed** (`preset-`) and +**deterministic**, every instantiation is identical — there is no +"two-divergent-instances" problem, and a shared preset replays the same on every +machine (§2 determinism, §6.1 content-addressing). + +### 8.3 Two-speed generation — the cost & consistency model + +| Mode | When | Model | Latency | Frequency | +|---|---|---|---|---| +| **Cold authoring** | nothing in the library fits | strong model (Sonnet/Opus) | seconds | **rare** — once per genuinely new idea | +| **Warm reuse** | a preset matches (exactly or after a small edit) | none, or a fast model for a parametric patch | sub-second | **the common case** | + +After a cold authoring, Tier 2 **offers to save the result as a preset** +("Save this game to your gallery?"). The expensive build is paid **once**; +every later use is warm. This is the same economics that makes Claude +artifacts feel cheap after creation — generalised into a persistent library +instead of a per-conversation artifact. + +### 8.4 Resolve-then-generate — the lookup before the build + +Before generating anything, Tier 2 runs a **library lookup** — the *same +shape* as the existing "check the per-canvas data cache before calling Tier 3" +rule in `CONCEPT.md` §"Per-canvas data cache": + +``` +intent ──▶ library lookup (semantic + capability match across scopes) + ├─ exact hit ──▶ instantiate preset (no LLM) + ├─ near hit ──▶ fork preset + targeted patch (fast model) + └─ miss ──▶ cold-author + offer to save (strong model) +``` + +So "build me X" is, by default, a **retrieve-and-bind** operation, not a +generation. Generation is the fallback, not the first move. + +### 8.5 Fork & vary — copy-on-write, with lineage + +"the game, but on a hexagonal board" does **not** rebuild it. Tier 2 **forks** +the `arcade` preset (copy-on-write → new content-addressed id, parent id recorded +for provenance) and applies a **targeted patch** to the affected +transitions/view. Cheap, consistent with the original, and the lineage is +auditable — important for the trust model when forks are shared (§2 point 5). + +### 8.6 Compose from idioms, not from zero + +Even cold authoring is not truly from scratch. The UI Skill carries a +**behaviour-idiom library** — the behaviour-layer extension of the +composition-idiom library already in `CONCEPT.md` §"The UI Skill" (Norton +Commander → two panes, etc.). Vetted building blocks the agent assembles: + +- a **scene-grid** skeleton (board/cell-matrix render), +- a **tick-loop** skeleton (gravity / animation step), +- **input-binding sets** (WASD/arrows/touch-swipe → moves), +- **Lume effect bundles** (a glow-pulse, a Ken-Burns hero, a condensation-in). + +Cold builds therefore assemble audited LX fragments rather than inventing +control flow each time — cheaper to generate **and** more consistent across +builds. This is the single biggest lever on both cost and reliability of cold +authoring. + +### 8.7 vs. Claude artifact logic + +| | Claude artifacts | Omadia Lumens | +|---|---|---| +| Persistence | per-conversation | cross-session canvas-state + **cross-user preset library** | +| Edit mechanism | string `update` / `rewrite` on source | **structural patch on a validated tree by stable id** (no drift) | +| Reuse | copy text into a new chat | **named, versioned, parameterised preset** — instantiate without an LLM | +| Consistency | re-generation can drift | content-addressed + deterministic ⇒ byte-identical | +| Sharing | share the code | share a **vetted, capability-manifested** preset (§7) | +| Variation | re-prompt | **fork + patch** with tracked lineage | + +The thesis: **the agent's job is to author rarely and reuse constantly.** A +Lumen is a durable, versioned component — built once, instantiated and patched +forever after. + +--- + +## 9. The agent relationship — generation *and* live introspection + +Lumens are generated agentically (Tier 2 composes the Lumen the way it +composes primitive trees today; heavy generation or data binding can recruit +Tier 3). But the deeper win over Live Artifacts is **bidirectionality**: + +- **Agentic generation.** "Build me a game" → Tier 2 emits a Lumen + (state/transitions/view/events) via `surface_snapshot`. "Make it faster as + the score climbs" → `surface_patch` adjusting the `tick` transition. The + agent refines the *running* behaviour conversationally. +- **Live introspection.** Because `state` is structured and readable, the + agent can answer "what's my high score?" or "summarise the current board" + by reading Lumen state from canvas-state — a Live Artifact is an opaque box + the model cannot see into. +- **Beam into a Lumen.** A user can beam a scene element ("why is this block + red?") or a region; the existing `TargetRef`/beam machinery resolves inside + the Lumen. +- **Composability on one canvas.** A Lumen is ordinary tree content (see §9.1), + so a map Lumen sits next to a Jira `table` and they interact through normal + canvas mechanics. Live Artifacts are isolated islands; Lumens are first-class + canvas citizens. + +### 9.1 Composition & cross-element interaction (it is just one tree) + +A Lumen is a node in the same primitive tree as every other primitive. There +is **no separate surface, no iframe, no island**. Two consequences: + +**(a) It composes like any primitive.** A Lumen lives inside `container` / +`pane` / `grid` / `tabs`, obeys the same layout, the same surface-nesting +ladder, the same Lume material, the same validator and the same render pass. +The agent places a map Lumen in one pane, a Jira `table` in another, a `chart` +below — one canvas, one tree. The boundary is porous *both ways*: a Lumen's +`view` itself emits ordinary primitives (a `button`, a `table`, a `status`), +so a Lumen can contain regular UI and regular UI can contain a Lumen. + +**(b) Elements interact — bidirectionally, deterministically, on Tier 1.** +The wiring reuses machinery that already exists, so it costs **no turn** and +runs at 60 fps: + +| Mechanism | How it wires | Direction | Tier | +|---|---|---|---| +| **Shared selection / view-state** | an element **publishes** a lightweight read-only interface (`expose`, e.g. `selection`); a neighbour referencing the same `DataRef` + **stable IDs** reads the *published* field by name — no explicit wire, and un-exposed state stays private (`CONCEPT.md` Authority Model) | UI ⇄ Lumen | **1 (Class A)** — no server | +| **Declarative ports & wires** | a node declares typed **inputs** and **outputs**; the agent declares **wires** (`from` → `to`) by stable id. The host routes values at Tier 1 | UI ⇄ Lumen, Lumen ⇄ Lumen | **1 (Class A)** | +| **Lumen capability → patch** | a Lumen output bound to `writeData`/`generateAsset`/etc. patches another container via Tier 2 | Lumen → UI (semantic) | 2 / 3 | +| **Beam / agent** | user beams across both ("compare these rows with the map"); agent reasons over the wired set | either, semantic | 2 | + +Concrete: + +- **Map Lumen ⇄ Jira `table`.** Select rows in the table → the bound markers + on the map glow (shared selection by `rowKey`). Tap a marker → the table + filters/scrolls to that `rowKey` (Lumen output wired to the table's + view-state). Both directions are pure Tier 1. +- **`form`/slider → simulation Lumen.** A `choice`/slider primitive is wired to + a Lumen `state` input (gravity, speed, grid size). Dragging the slider + retunes the running sim live — declarative, no turn. +- **Lumen output → primitive.** A game's `game-over` (a Lumen output) wired to a + `status` primitive ("New high score!") and to a `writeData` capability that + persists the score back through Tier 3. +- **List ⇄ defrag-viz Lumen.** Hover/select a file in a `list` → its blocks + light up in the scene; click a block cluster → the list scrolls to that file. + +**The boundary that keeps it safe.** Cross-element interaction is **declared +data** (validated by the whitelist parser), and a node can only read what is +**wired or published to it** — an explicit `wire`, or a neighbour's declared +**`expose`** interface (a lightweight, read-only set of view-state the neighbour +*chose* to offer, bound by shared id). It cannot reach arbitrary other elements' +internals, and **un-exposed state stays private** — so an imported or untrusted +Lumen observes no ambient neighbour state and itself leaks nothing it did not +publish. This is least-privilege by construction, *ambient-by-declaration* not +ambient-by-default: the same boundary that makes a Lumen safe to share makes +cross-element interaction auditable and shared-canvas-safe (wires and `expose` +resolve by stable id, deterministically, so they replay and multicast +unchanged). The **authority split is unchanged**: the agent owns *which +elements, wires and published interfaces exist* (structure); the client owns +*the values currently flowing through them* (view-state). + +> **SDK delta:** declarative **`ports`** (typed inputs/outputs) and **`expose`** +> (a published read-only view-state interface, bindable by shared id without a +> wire) on primitives and Lumens, and **`wires`** (a `[{from: TargetRef+port, +> to: TargetRef+port}]` list) at the container/canvas level — additive tree +> content, Tier-1-resolved, whitelist-validated. No new transport. + +--- + +## 10. The user's four use cases, mapped + +| Use case | state | view | events | capabilities | Tier split | +|---|---|---|---|---|---| +| **An arcade game** (build · play · share) | board, moving token(s), score, level | `scene` cell grid | declared keys + host `tick` (≤60 fps) | `persist` (high score), `share` | loop is pure **Class A** on Tier 1; generation/share on Tier 2 | +| **Interactive data workflow** | working set, step, selection, edits | primitives (table/form) or `scene` | pointer/submit | `loadData` (real data in), `writeData` (commit back via Class-D + `writeCapabilities`) | Tier 1 interaction; Tier 2 brokers writes; Tier 3 owns the system of record | +| **Defrag-style / unusual viz** | dataset projection, animation cursor | `scene` coloured-cell grid | host `tick` for animation | `loadData` | data fetch via Tier 2/3; animation pure Tier 1 | +| **Interactive map** | viewport, markers, selection | `scene` (sprites = tiles, markers) + pan/zoom | pointer (pan/zoom/click) | `tiles` (provider-allowlisted), `loadData` (places), `persist` (last viewport) | pan/zoom/render pure Tier 1; tiles brokered Tier 2/3 | + +Every one of these is impossible-or-crippled in both Omadia-today (no +behaviour model) and Live-Artifacts (no real capabilities). Each is natural +in the Lumen model. + +--- + +## 11. How it fits the existing architecture (deltas, not rewrites) + +Everything below is **additive** and stays inside the `CONCEPT.md` tier model, +authority split, and security surface. No wire-grammar rewrite. + +| Area | Delta | +|---|---| +| **Primitives** | add `scene` (editor-class) — protocol minor bump (`1.x`); draw-list is whitelisted shape data | +| **Tree content** | add the `behavior`/`lumen` section (state/transitions/view/events/capabilities) — validated by an **extended whitelist parser** (schema + LX-AST validator) | +| **Composition / interaction** (§9.1) | declarative **`ports`** (typed inputs/outputs) + **`expose`** (published read-only interface) on primitives & Lumens + **`wires`** (`from`→`to` by `TargetRef`) at container/canvas level; Tier-1-resolved by stable id, whitelist-validated; shared `viewState.selection` read only via a published `expose` field (ambient-by-declaration). No new transport | +| **Tier-1 client** | new **Lumen runtime**: deterministic LX evaluator, gas + frame ceiling, scene rasteriser, event dispatch, seeded `random`/clock. All Class-A; the frame loop never touches the server | +| **Tier-2 orchestrator** | composes/patches Lumens; **brokers capability calls**; grants/scopes capability manifests; persists Lumen state & presets; manages share/ownership | +| **Lifecycle / reuse** (§8) | **resolve-then-generate**: library lookup before any build (same shape as the existing pre-Tier-3 data-cache check); preset **instantiate** (no LLM) / **fork+patch** (fast model) / **cold-author** (strong model); content-addressed, versioned, parameterised presets; behaviour-idiom library in the UI Skill | +| **Tier-3** | reached only via brokered capabilities (`tiles`, `fetch`, `writeData`, AI ops) — unchanged interface | +| **Transport** | Lumen rides existing `surface_snapshot`/`surface_patch`; capability calls reuse the effect-classified action path + `surface_action_result`/patch; **one** new optional event family `surface_capability_*` if streaming results need it | +| **Security** | extend whitelist parser to LX-AST + scene + capability manifest; reuse HMAC scoping for capability tokens; reuse `local`/`internal`/`external-effect` + confirmation modal | +| **Handshake** | client declares supported **LX version**, **gas limits**, **scene support**, and **granted capability classes** alongside `localOperations` | +| **Identity / sharing** | reuse `canvasOwnership` group extension + channel fan-out; new `lumen-presets/**` and `lumen-state/**` memory namespaces | +| **Versioning** | LX, scene vocabulary, and capability catalog versioned with `omadia-canvas-protocol`; capability catalog negotiated like the ops catalog | + +What classic channels see: **nothing** — same as `CONCEPT.md`. All additive, +engaged only behind the `canvas` capability. + +--- + +## 12. The sweet-spot dial (answering the user's central ask) + +The user's framing: Live Artifacts errs too far toward restriction; find the +better-tuned point between *possibility* and *safety*. The Lumen model tunes +**two independent dials** instead of the one coarse dial a sandbox gives you: + +1. **Compute dial — set permanently to "constrained".** Declarative, + bounded, deterministic, interpreted. This is not a per-Lumen choice; it is + the architecture. It removes the entire arbitrary-code threat class. (v2+ + may add an opt-in WASM dial for genuine outliers, gated hard.) +2. **Capability dial — set per-Lumen, per-user, per-call, openable far.** + This is where expressiveness lives. Because compute is safe, we can afford + to open capabilities *generously but explicitly*: real data, write-back, + persistence, even allowlisted network — each declared, brokered, confirmed, + audited. + +Live Artifacts collapses both dials into one ("how much of the sandbox do we +trust?") and is forced to keep it low, which is why "even the simplest use +cases are blocked". Splitting the dials is the smarter approach: **lock the +dangerous one, open the useful one.** + +--- + +## 13. Open questions for the spike (flagged, not answered) + +1. **Gas & frame-budget numbers.** Initial caps for LX gas/frame, state size, + scene draw-list length, tick rate, the `timer` minimum period and the + **combined `tick`+`timer` wakeup budget** per Lumen — measured against the + four reference Lumens (an arcade game, workflow, defrag-viz, map). + Spike-tunable, like the `viewState` budget in `CONCEPT.md`. +2. **LX surface area.** Exactly which standard-library functions ship in + `LX/1.1` (LX is versioned with the protocol — the boot handshake declares + `lxVersion: "1.1"`, `lumens-spec.md` §13). Bias small; grow by minor bump. + Risk: too small blocks real Lumens; too large grows the audit surface. +3. **Scene performance ceiling.** Draw-list size at which Tier-1 raster drops + below 60 fps; whether WebGL is required for v1 or canvas-2d suffices for + the reference set. +4. **Capability granularity & consent fatigue.** How fine-grained the consent + prompt should be without nagging; defaults for trusted first-party + capabilities (`persist`) vs. always-confirm ones (`external-effect`). +5. **Determinism vs. real time.** Maps and live feeds are inherently + non-deterministic at the *data* edge; confirm that seeding the *compute* + while treating capability results as external inputs keeps replay coherent + (replay re-feeds recorded capability results, like a recorded test). +6. **LLM reliability emitting LX.** Can a fast (Haiku-class) model emit valid + LX ASTs reliably, or is Lumen generation a Sonnet/Opus job? Mirrors + Riskiest-Assumption #1 in `CONCEPT.md`. Likely: generation on a stronger + model, in-session tweaks cheaper. +7. **Preset trust & distribution.** Signing, capability-manifest review UX, + and whether a shared community gallery needs a moderation/attestation + layer (likely v2+). +8. **Capability-broker egress bounds.** The exact anti-DoS / anti-cost contract + for the Tier-2 broker — per-capability rate/quota, max-in-flight, idempotency + keys and backpressure — so a `tick`/`timer`-driven `generateAsset`/`fetch`/ + `writeData` cannot push load or spend onto Tier 2/3, plus the rule that + state/`DataRef`-derived outbound requests are `external-effect` unless + pre-approved (`lumens-spec.md` §6, §11). Spike-tunable, measured against the + reference Lumens. +9. **Native-kernel cut & budget.** Which bounded algorithms ship in the `LX/1.1` + kernel whitelist (`lumens-spec.md` §2.6) — sort / group / aggregate / scale / + timeBucket / layout for business work vs. the game-ward `pathfind`/`floodFill` + outliers — plus per-kernel signatures and the **kernel-gas** schedule. Bias + small; grow by minor bump. Risk mirrors §2 (LX surface): too small blocks real + Lumens, too large grows the audit surface. +10. **Generation-reliability net.** Whether **declared invariants** (§2.7) + the + **golden-trace** author-time gate (§14) + preset/idiom assembly as the + primary path reduce silent-wrong cold-authoring to an acceptable rate — + measured on the five hand-written reference Lumens. The validator catches + syntax, never correctness; these are the correctness net. + +--- + +## 14. What this is not (scope discipline) + +- **Not arbitrary code execution.** No `eval`, no iframe-with-script, no WASM + in v1. If a use case truly needs Turing-complete compute, it is a flagged + v2+ escape hatch, gated hard — not the default. +- **Not a new design language.** Lumens render in Lume, in the active palette, + in the type registers. A game looks like Omadia. +- **Not a bypass of the authority/security model.** Lumens obey the same + authority split (agent owns structure, client owns view-state), the same + stable-ID discipline, the same effect classification and confirmation + contract, the same DataRef/HMAC scoping. +- **Not an implementation plan.** This document is concept only. Protocol + schema, the LX grammar spec, the scene vocabulary, the capability catalog, + reference-Lumen walkthroughs, and the PR sequence are spike deliverables, + authored after this concept is accepted. diff --git a/docs/lumens-spec.md b/docs/lumens-spec.md new file mode 100644 index 0000000..de25186 --- /dev/null +++ b/docs/lumens-spec.md @@ -0,0 +1,846 @@ +# omadia-canvas-protocol — Lumens (Live Interactivity) · draft 1.1 + +> **The normative definition** of the Live-Interactivity extension. Where +> [`interactivity-concept.md`](interactivity-concept.md) holds the *rationale* +> (the *why*, narrative), this document holds the *definition* (the *what*, +> normative) — the types, the grammar, the contracts a renderer and an +> orchestrator must implement. It is the Lumen counterpart to +> [`protocol/1.0.md`](protocol/1.0.md) and a companion to +> [`visual-spec.md`](visual-spec.md) (Lume) and [`../CONCEPT.md`](../CONCEPT.md) +> (the canvas architecture). + +**Status:** `draft 1.1`. Additive, **minor** bump over +`omadia-canvas-protocol/1.0`, and **negotiation-gated**: all 1.1 content (the +`scene` primitive, the `behavior`/`lumen` tree section, ports/wires, the +`surface_capability_*` events) is emitted **only to clients that negotiated +support at the boot handshake** (§13). A 1.0-only client never negotiates it, so +Tier 2 never sends it; and if such content ever reached a 1.0 renderer it would +be **hard-rejected** by the whitelist parser (`protocol/1.0.md` §2), never +silently mis-rendered — additivity is enforced **fail-closed by negotiation, not +by clients silently ignoring unknown primitive types**. (Only additive *fields +on already-known types* ride 1.0's "ignore unknown fields" forward-compat, +`protocol/1.0.md` §0; unknown *types / tree sections* are gated, not ignored.) +Nothing here breaks the 1.0 wire grammar. The machine-validatable truth will +live in `schema/` (Lumen, LX-AST, scene, ports/wires, capability manifest) and +is a spike deliverable; where prose and schema disagree, the **schema wins**. + +> **Rev 2 (Codex review).** Clarifications within `draft 1.1` (no protocol-minor +> change): additivity is negotiation-gated / fail-closed (§0, §12); the agent +> owns capability *requests*, never *grants* (§0.5); `timer` is bounded like +> `tick` via a combined wakeup budget (§0.2, §4); capability-broker egress +> bounds + state/`DataRef`-derived `fetch`/`writeData` classification (§6, §11); +> shared/preset assets travel by content `id` with recipient-scoped token +> re-mint (§9); ambient cross-element reads replaced by a declared `expose` +> interface (§7, §11). + +> **Rev 3 (expressiveness & practice-fit).** Closes three normative gaps the +> prose hid when LX is actually hand-written for a board-game-class Lumen: the +> `map`/`filter`/`fold` **binder node forms** and computed-index `at`/`setAt` +> (§2.2) — without them no iteration or board mutation is expressible. Adds +> **native kernels** (§2.6) — bounded, host-owned algorithms (sort, group, +> aggregate, scale, layout, pathfind, …) that pure first-order LX cannot +> express, exposed as pure calls (the *capability pattern applied to compute*). +> Adds **declared invariants** (§2.7, silent-wrong → loud-error) and a +> **golden-trace authoring gate** (§14). Adds **transactional / high-frequency +> patterns** (§6.3) so kiosk/ordering flows fit without a confirmation modal per +> tap, plus a fifth (transactional) reference Lumen. + +> **Rev 3.1 (hand-author test).** Hand-writing the arcade tick + an ordering flow +> in real LX-AST validated rev 3 and surfaced follow-ups, now closed: the `{var}` +> read node (§2.2 — binders were unreadable without it, so iteration was not +> actually expressible); an immutable **`const`** section + `{const}` node (§1.2) +> as the structural fix for static-table explosion (and the unit the idiom +> library ships); 2-D `at`/`setAt` over `list`, not only `grid`, with a +> spatial `[x,y]` convention (§2.2); and ergonomics for the hot render path — the +> optional `idx` binder and `flatten` (§2.2/§2.3). + +> **Rev 3.2 (colour authority).** A Lumen's **own content** is **not** +> palette-locked: the agent picks `colorMode: 'theme'|'brand'|'free'` (+ a +> declared `palette`) from the **request + embedding context** (§3.1). `theme` is +> the *no-direction default* — justified only by the assumption that the Lumen +> embeds in an existing Lume UI, an assumption that is **not universal**; a +> kiosk / branded / product surface gets `brand`/`free` **directly**, no opt-out +> to fight. Scoped to the Lumen's subtree: **Omadia chrome always stays Lume** (v1 +> identity boundary, no host white-label). In `brand`/`free` the normaliser no +> longer clips colour and enforces **no** contrast floor — accessibility of +> free-colour content is the author's responsibility (44 pt hit-targets and +> reduced-motion still apply; those are interaction-safety, not colour). + +A Lumen is the Omadia answer to "an interactive artifact": **declarative data, +not code**, run by a small deterministic interpreter on Tier 1, generated and +brokered agentically on Tiers 2/3, safe to share and to save as a preset +*because* it is data. See `interactivity-concept.md` §0 for the thesis (most of +what sandbox-artifacts block is blocked by missing *capabilities*, not missing +*compute* — so we constrain compute and open capabilities, mediated). + +--- + +## 0. Conventions & non-negotiable constraints + +Inherited from `protocol/1.0.md` §0 (two-axis versioning, opaque `RevisionId`, +stable IDs as the lingua franca, one JSON value per frame) plus: + +1. **No arbitrary code, ever.** A Lumen carries *data* — a typed state schema, + validated expression ASTs, a view template, event bindings, a capability + manifest. The Tier-1 runtime is an **interpreter shipped in the Host App**, + never `eval`/`Function()`. CSP stays `default-src 'self'`, no `unsafe-eval`. + The whitelist-parser discipline of 1.0 **extends** to LX-AST, scene shapes, + ports and capabilities: any unknown node/type → hard reject (`surface_error`). +2. **Bounded & total.** Every transition / view evaluation runs under a gas + budget and a wall-clock ceiling; iteration is bounded (no open `while`, no + general recursion); state is size-capped; the **wakeup rate** (`tick` + + `timer` combined) is itself capped and count-limited per Lumen (§4). A Lumen + can never hang the host — exceeding any budget halts it with `surface_error`. +3. **Deterministic.** All non-determinism is host-seeded (`random`, `now`, + tick). `(state, event) → state` is identical on every machine — the basis + for replay, undo, safe sharing and v2 multi-user. +4. **Default-deny capabilities.** A Lumen reaches nothing outside its own state + except through declared, granted, effect-classified capabilities brokered by + Tier 2 (`CONCEPT.md` §"Security Surface" effect classes). +5. **Authority split unchanged.** The agent owns *structure* (which Lumens, + elements, wires, published `expose` interfaces, and capability **requests** + exist). The client owns *view-state* (the values flowing through them, + current selection, scroll). + Capability **grants** are *not* agent-owned: a request is granted only by + Tier-2 policy plus user consent where the effect class requires it (§6) — an + agent-authored patch can *ask* for a capability, never *self-grant* one + (no `fetch` / `writeData` / `share` escalation via a patch). Stable IDs bind + the two (`CONCEPT.md` §"Authority Model"). +6. **Lume is the material.** Lumens render in Lume — light-as-material, **not** + glassmorphism (`visual-spec.md` §1.3). §10. + +--- + +## 1. Definition — the Lumen + +A **Lumen** is a self-contained interactive unit on the canvas. It is delivered +as **tree content** (a node, or a `behavior` section attached to a `container`) +inside an ordinary `surface_snapshot` / `surface_patch` — **no new transport**. + +```ts +type Lumen = { + id: string; // stable; patches/wires/beam target it + state: StateSchema; // §1.1 — typed, bounded, serialisable (mutable memory) + const?: ConstSchema; // §1.2 — typed, bounded, immutable author-time tables (not serialised) + transitions: Record; // §2 — pure (state,event)->state + view: LXNode; // §2,§3 — pure state -> primitive/scene tree + events: EventBinding[]; // §4 — declared inputs -> transitions + cadence?: CadenceSpec; // §5 — default "reactive" + colorMode?: 'theme'|'brand'|'free';// §3.1 — default 'theme'; opens colour for THIS Lumen's content only + palette?: PaletteSpec; // §3.1 — declared brand colours (used with colorMode 'brand') + capabilities?: CapabilityRequest[]; // §6 — default-deny doors out + ports?: PortSpec[]; // §7 — typed inputs/outputs for explicit wiring + expose?: ExposeSpec[]; // §7 — published read-only view-state (the ambient-readable interface) + invariants?: LXNode[]; // §2.7 — boolean assertions checked after every transition + preset?: PresetRef; // §8 — provenance if instantiated/forked +}; +``` + +A Lumen is valid iff: its `state`/`const` conform to §1.1/§1.2, every `LXNode` in +`transitions`/`view`/`invariants` passes the §2 AST whitelist + static bounds +check (every `{call}` target in §2.3, every `{kernel}` target in §2.6, every +`{const}`/`{state}` path resolving against the declared schema), every +`EventBinding` names a declared transition and a §4 event, every +`CapabilityRequest` names a §6 catalog capability, every `invariants` entry is a +boolean `LXNode`, and every `PortSpec` / `ExposeSpec` is §7-typed. Any failure → +the Lumen is rejected wholesale with `surface_error` (scope = the Lumen `id`); it +never partially renders. + +### 1.1 State schema + +`state` is a typed, **closed** record. Every leaf declares a type from the LX +value set (§2.1) and bounds: + +```ts +type StateSchema = { + [key: string]: + | { type: 'int' | 'number', min?: number, max?: number, init: number } + | { type: 'bool', init: boolean } + | { type: 'string', maxLength: number, init: string } + | { type: 'enum', values: string[], init: string } + | { type: 'list', of: StateLeaf, maxLen: number, init: unknown[] } + | { type: 'record', fields: StateSchema, init: object } + | { type: 'grid', w: number, h: number, of: StateLeaf, init?: unknown } // bounded 2D — boards, defrag cells + | { type: 'dataRef', init?: DataRef }; // §6.1 read-only projection handle +}; +``` + +Total serialised `state` size is capped (initial default **256 KB**, +spike-tunable). `state` persists in canvas-state (`CONCEPT.md` §"State Model"); +it is the *only* mutable memory a Lumen has. + +### 1.2 Constants — immutable author-time data + +`const` is a typed, bounded, **immutable** record of author-time data — lookup +tables, shape sets, maze layouts, colour maps, any constant a transition reads +every frame. It uses the same typed-leaf grammar as `state` (§1.1, minus the +mandatory `init`/with a required `value`) but differs in three ways that make it +the structural fix for **table explosion** (the friction of inlining a tetromino +shape set as a giant `match`/`lit` tree in every transition): + +1. **Agent-owned structure, not view-state.** It travels with the Lumen spec + (itself content-addressed), is whitelist-validated like `state`, and is the + unit the preset/idiom library (§8) ships — an idiom fragment carries its table + **once**, declared, not re-inlined. +2. **Read-only.** No transition writes it; there is **no `setAt` into `const`**. + Read it via `{const: path}` (§2.2), computed-indexable with `at`, exactly like + `{state: path}`. +3. **Not serialised.** It does **not** count against the 256 KB `state` cap, does + not persist per-turn, and does not appear in undo/replay — it is part of the + program, not the memory. A patch to a transition no longer drags a re-inlined + table. + +`const` is bounded like `state` (size-capped, spike-tunable), so the static gas +bound covers `map`/`fold` over `const` collections too. For genuinely **large** +blobs (a tile atlas, a big level) prefer a **`DataRef`** (§6.1 — content- +addressed, fetched once, async) over an inline `const`; `const` is for the small, +hot, synchronously-read tables a transition needs inside the frame loop. + +--- + +## 2. Lume Expressions (LX) + +LX is the **pure, total** expression language of `transitions` and `view`. It is +delivered as a **JSON AST**, never as source text (no parser-injection surface; +the validator is a tree-walk, exactly like the primitive whitelist). + +### 2.1 Values + +`int`, `number`, `bool`, `string`, `list`, `record{…}`, plus the read-only +`state` and `event` bindings in scope. No closures over host objects, no `this`, +no prototypes, no functions-as-values beyond the named std-lib. + +### 2.2 AST node catalog (whitelist) + +| Node | Form | Meaning | +|---|---|---| +| `lit` | `{lit: value}` | literal | +| `state` | `{state: path}` | read a `state` slice (dotted path; `grid` via `{state, at:[x,y]}`) | +| `const` | `{const: path}` | read an immutable `const` slice (§1.2); computed-indexable via `at`, exactly like `state` | +| `event` | `{event: field}` | read a field of the triggering event | +| `let` | `{let:{name:expr}, in:expr}` | bind a local (readable via `{var}`); lexically scoped, immutable; nest for multiple bindings | +| `var` | `{var:name}` · `{var:name, path:"f.g"}` | **read** a bound local — a `let` name or a `map`/`filter`/`fold` binder (`as`/`acc`); optional dotted sub-path into a record/list. The *only* way to read a binder; without it no `let`/iteration body can reference what it binds | +| arithmetic | `{"+":[a,b]}` `-` `*` `/` `mod` | numeric | +| comparison | `{">":[a,b]}` `>=` `<` `<=` `==` `!=` | boolean | +| logic | `{and:[…]}` `or` `not` | boolean | +| `if` | `{if:c, then:a, else:b}` | total conditional (both branches required) | +| `match` | `{match:expr, cases:[{when,then}], else}` | total switch | +| record/list ctor | `{record:{…}}` `{list:[…]}` | construction | +| `set` | `{set:{path: expr}}` | **functional** update at a *static* path → returns a new state (no mutation) | +| `setAt` | `{setAt: coll, index:[xExpr] \| [xExpr,yExpr], to: expr}` | **functional** write at a computed index → new collection; 1-D indexes a `list`, 2-D `[x,y]` a `grid` **or** a `list` (as `coll[y][x]`); out-of-bounds is a **no-op** (total) | +| `at` | `{at: coll, index:[xExpr] \| [xExpr,yExpr], default: expr}` | random-access **read** at a computed index → element or, on out-of-bounds, `default` (total); 1-D for `list`, 2-D `[x,y]` for `grid` **or** `list` (`coll[y][x]`); also the form for `{state, at:[…]}` with **expression** indices | +| `map` | `{map: listExpr, as:"x", idx?:"i", body: expr}` | element-wise; binds item `x` (and optional index `i`) per item → new list | +| `filter` | `{filter: listExpr, as:"x", idx?:"i", body: predExpr}` | keep items where `body` is true → new list | +| `fold` | `{fold: listExpr, as:"x", idx?:"i", acc:"a", init: expr, body: expr}` | left fold; binds accumulator `a`, item `x` (and optional index `i`) → final `a` | +| std-lib call | `{call:name, args:[…]}` | first-order helper from the §2.3 whitelist | +| native kernel | `{kernel:name, args:[…]}` | bounded, host-implemented algorithm from the §2.6 whitelist | + +`map`/`filter`/`fold` are the **only** iteration; their binders (`as`/`acc`/the +optional `idx`) are syntactic lexical scopes read via `{var}`, **not** first-class +function values (no closures, §2.1). They iterate only over `state`- or +`const`-bounded collections, so the gas bound stays static (§2.4). The optional +`idx` binder gives position-dependent iteration (rendering a board, indexed maps) +without the `map(range,…) + at` detour. + +`at`/`setAt` make random-access **total** by requiring an out-of-bounds answer — +the load-bearing forms for any board/cell mutation. **2-D `[x,y]` is spatial — `x` +horizontal, `y` vertical — for both `grid` and `list` (the latter resolving +to `coll[y][x]`), so an author thinks in coordinates regardless of backing.** Pick +**`grid`** for fixed dimensions (Pacman maze, defrag cells: clean random R/W); +pick **`list`** when rows are added/removed (Tetris line-clear: clean +`filter`/`concat`) — both now index identically by `[x,y]`. + +### 2.3 Standard library (whitelist, bounded, first-order) + +Scalar/collection helpers callable as `{call:name,…}`: `range` `len` `min` +`max` `clamp` `abs` `floor` `round` `mod` `concat` `flatten` `slice` `contains` +`indexOf` `keys` `values`, string ops (`upper` `lower` `pad` `fmt` `split` +`join`) and a small math set. `flatten` (one level) turns nested iteration into a +flat `list` — e.g. a board's per-row node lists into one `scene` draw-list — +without a `fold`/`concat` accumulator. Iteration is **not** here — it is the +dedicated `map`/`filter`/`fold` binder nodes (§2.2), bounded by `state`/`const` +size, so the gas bound stays a *static* property. **No `while`, no general recursion, no +first-class functions.** `random()` and `now()` read host-seeded context values +(§0.3). Genuinely iterative algorithms (sort, group/aggregate, pathfind, +layout, …) are **not** open-coded in LX — they are the bounded **native +kernels** of §2.6. + +### 2.4 Gas & determinism contract + +- Each `transition`/`view` evaluation is metered (instruction count). Initial + default **50 000 gas / evaluation**, spike-tunable. Over budget → + `surface_error`, the Lumen is halted (not the canvas). +- A wall-clock ceiling per frame is a secondary guard. +- Given identical `(state, event, seed)` the result is byte-identical + everywhere. Renderers MUST NOT introduce ambient non-determinism. + +### 2.5 Validation + +A Lumen's LX is accepted iff every node is in §2.2, every `call` target is in +§2.3, every `kernel` target is in §2.6, every `state`/`event` path resolves +against the declared schema, and a static pass proves iteration bounds and a gas +ceiling. `view` MUST return a valid primitive/scene tree (§3); `transitions` MUST +return a value conforming to `state`. Anything else → reject. + +### 2.6 Native kernels — bounded algorithms the host owns + +Pure LX is first-order and non-recursive: it expresses **state machines and +local/greedy logic** but **not** genuinely iterative algorithms (pathfinding, +connected-components, graph layout, sort, grouped aggregation). Rather than +re-open the Turing-complete hole this whole model exists to avoid, those +algorithms are **native kernels** — fixed, audited, host-implemented functions +exposed to LX as pure calls `{kernel:name, args:[…]}`. This is the **capability +pattern applied to compute**: the agent never *writes* a kernel, only *calls* one +from the whitelist, exactly as it draws a primitive from the render whitelist. +The compute dial stays "constrained" (§12); only the *vocabulary of bounded +primitives* widens — additively, by minor bump, like a new primitive. + +Every kernel is **(1) deterministic** (seeded, no IO, no capability — a kernel is +*not* a door out; that is what §6 capabilities are); **(2) internally bounded** — +it runs its loop in native code under a per-call **kernel-gas** ceiling +proportional to its (state-capped) input; and **(3) total** — a degenerate input +or an exceeded ceiling returns a declared empty/identity result, or halts the +Lumen with `surface_error` on a hard breach, but never hangs. Kernels are +versioned and negotiated with LX (§13); a client implementing a subset advertises +it, and a Lumen needing an unsupported kernel degrades or is rejected — same +discipline as `localOperations`. + +Initial blessed set (bias small — grow by **minor** bump, §13), ordered by +business value: + +| Kernel | Signature (sketch) | For | +|---|---|---| +| `sortBy` | `(list, keyExpr, dir) → list` (stable) | tables, leaderboards, any ordering | +| `groupBy` | `(list, keyExpr) → record` | pivots, segmentation | +| `aggregate` | `(list, {op, field}) → number` — `sum∣avg∣count∣min∣max∣median∣pNN` | KPIs, rollups | +| `scaleValue` / `ticks` | `(domain, range, kind, v) → number` / `(domain, n) → list` (`linear∣log∣ordinal∣time`) | every chart axis | +| `timeBucket` | `(timestamps, unit) → record` (`day∣week∣month∣…`) | time series, calendars, gantt | +| `layoutGraph` | `(nodes, edges, kind) → positions` (`dag∣hierarchical∣force`) | org / dependency / flow charts, mind maps | +| `treemap` / `packRects` | `(weights, w, h) → rects` | dashboards, treemaps, defrag layout | +| `geo` | `pointInPolygon · bbox · segIntersect` | maps, scene hit-geometry | +| `floodFill` | `(grid, seed) → labelled grid / region` | selection regions, clustering, defrag | +| `pathfind` | `(grid∣graph, start, goal, opts) → path` (`bfs∣dijkstra∣astar`) | routing, wayfinding, maze games | + +`keyExpr`/predicates handed to a kernel are **LX expression ASTs** evaluated per +element under the same gas discipline (a kernel taking an expression budgets +`elements × eval` against kernel-gas). Per-kernel signatures, the kernel-gas +schedule, and the exact v1.1 cut line are a **schema/spike deliverable** (§14); +the list above is the *intent*, biased to the cases real business artifacts hit +(the last two are the game-ward outliers — lower priority, since we are not +building a game engine). The boundary stays clean: a kernel may iterate because +it is **audited native code with a hard internal ceiling**, not agent-authored +control flow — so "cannot hang / cannot DoS" (§0.2) and determinism (§0.3) hold +for kernels exactly as for the interpreter. + +### 2.7 Declared invariants (silent-wrong → loud-error) + +A Lumen MAY declare `invariants` — boolean LX expressions over `state` that MUST +hold **after every transition** (e.g. `score >= 0`, the active piece in bounds, +`len(cart) <= max`). The runtime evaluates them post-transition (cheap, bounded, +same gas pool); a violation **rolls back** the offending transition and raises +`surface_error` (scope = Lumen `id`) rather than letting corrupt state render. +Invariants do not *prove* correctness, but they convert a meaningful class of +generation bugs — the off-by-one the validator **cannot** catch because the +Lumen is syntactically valid — from **silent-wrong** into a caught, loud failure +the agent repairs by patch. They pair with the golden-trace authoring gate (§14). + +--- + +## 3. The `scene` primitive (editor-class, 1.1) + +`scene` is a declarative immediate-mode draw surface — the 25th primitive, an +editor-class addition (`protocol/1.0.md` §2). The Lumen `view` emits, per +render, a **draw-list** from a closed shape vocabulary. There is **no** canvas +`2d`/`webgl` script exposed to the agent. + +```ts +type Scene = { + type: 'scene', + id: string, + width: int, height: int, // buffer-native coordinate space + camera?: { x:number, y:number, zoom:number }, // pan/zoom; buffer-native + draw: SceneNode[], +}; + +type SceneNode = + | { kind:'rect', x,y,w,h, r?, fill?, stroke?, strokeW?, id? } + | { kind:'circle', cx,cy,r, fill?, stroke?, strokeW?, id? } + | { kind:'line', x1,y1,x2,y2, stroke, strokeW?, id? } + | { kind:'path', points:[number,number][], closed?, fill?, stroke?, id? } // reuses vector-path geometry + | { kind:'sprite', x,y,w,h, dataRef: DataRef, id? } // §6.1 — images, tiles, glyphs + | { kind:'text', x,y, text, size?, weight?, register?, fill?, id? } // Lume type registers + | { kind:'group', transform?, children: SceneNode[], id? }; +``` + +- **Colour is theme-bound by default, author-openable for the Lumen's own + content (§3.1).** By default a `scene` draws from Lume tokens (`accent`, + `accent.glow*`, surface/text/semantic tokens), is re-tintable, and is always + on-theme — a game still looks like Omadia. A Lumen MAY declare `colorMode: + 'brand'|'free'` + a `palette` (§3.1) for kiosk / branded / product surfaces; + this scopes to the **Lumen's own subtree only** (Omadia chrome always stays + Lume), and in `brand`/`free` the normaliser does **not** clip colours. +- **Coordinates are buffer-native**, independent of zoom/pan (`CONCEPT.md` + `bufferRegion`). Hit-testing maps pointer → buffer coords → the `id` of the + topmost hit node. +- A `SceneNode.id` is a **stable element id** → it is a `TargetRef` + (`{kind:'element', elementId}`) for beams, events (§4) and wires (§7). +- Tier 1 rasterises the draw-list natively (canvas2d or WebGL) at up to 60 fps + from local state — pure **Class A**, zero server contact per frame. + +`scene` coexists with `canvas-region` (the *pixel-editor buffer* for +brush/blur Class-B ops); `scene` is a *state-driven render target*, not an +editable pixel buffer. + +### 3.1 Colour authority (theme · brand · free) + +Colour inside a Lumen is a **declared, scoped** property the agent chooses from +the **request and the embedding context** — not a fixed preference for Lume. Two +fields on the Lumen: + +```ts +colorMode?: 'theme' | 'brand' | 'free'; // default 'theme' +palette?: { [name: string]: ColorToken | sRGBHex }; // bounded, declared brand colours +``` + +| `colorMode` | Colours | Re-tint | For | +|---|---|---|---| +| `theme` *(no-direction default)* | Lume tokens only | yes (palette switch re-tints) | a Lumen meant to sit **inside an existing Lume UI** — the default *only* when no colour direction is given | +| `brand` | a **declared** bounded `palette`, referenced by name | as a unit | kiosk / branded ordering / product surfaces | +| `free` | arbitrary sRGB/hex per node | no | photographic gradients, many-colour games, generative art | + +- **`theme` is the no-direction default, not a value judgement.** Absent any + colour direction, the agent assumes the Lumen will be **embedded in an existing + Lume-designed UI** and uses `theme` so it sits in seamlessly. **That assumption + is not universal.** A standalone kiosk, a branded ordering surface, a product + presentation or a game with its own art are **first-class** cases where the + agent reads the intent and chooses `brand`/`free` **directly** — the user must + never have to *fight* an opt-out. `colorMode` is **derived from the request + + embedding context** (the UI Skill carries the heuristic: explicit brand/colour + ask, or a standalone/full-bleed surface → `brand`/`free`; "add this to my + dashboard", no colour ask → `theme`). `theme` wins only when nothing points + elsewhere; it is not "safer" or "more correct" than `brand`/`free`. +- **Scope = the Lumen's own subtree.** `brand`/`free` colour governs the Lumen's + `scene` draw-list **and** the themeable surfaces of the primitives its `view` + emits. It does **not** touch anything outside the Lumen: **Omadia chrome + (header, action panel, Beam, canvas frame) and sibling canvas elements always + render in the active Lume theme.** The host stays recognisably Omadia; the + *content* is the author's brand. (No white-label of the host chrome in v1 — a + deliberate identity boundary.) +- **Brand colour can still ride the Lume material.** A declared brand colour may + render as an *illuminating* accent (glow / surface-luminosity, §5/§10) for a + premium look, **or** as a flat fill to match a brand exactly — author's choice. + The Lume **material technique** (no glassmorphism, no blur-as-chrome, + `visual-spec.md` §1.3) governs the host chrome regardless; a `flat` brand fill + is a colour choice, not a return to glass. +- **No clipping, no contrast enforcement.** In `brand`/`free` the Tier-1 + normaliser does **not** clip colours and does **not** enforce a contrast floor + — **accessibility of free-colour content (contrast, colour-blind safety) is the + author's responsibility.** Interaction-safety guarantees are *not* colour and + still hold: 44 pt hit-targets (§4) and reduced-motion (§5) apply regardless. +- **Still data, still safe.** A `palette` is declared, bounded, whitelist- + validated data (no code); `free` node colours are plain sRGB values in the + draw-list. Colour freedom touches the *look* only — determinism, gas and + default-deny capabilities are unchanged. + +--- + +## 4. Events & input (touch-first) + +`events` bind **pointer-semantic** inputs to transitions. They resolve +identically across mouse / trackpad / touch / pen — the same abstraction +`CONCEPT.md` uses for context-invoke. + +```ts +type EventBinding = { + on: 'tap'|'longPress'|'drag'|'pinch'|'swipe'|'pointerMove'|'key'|'tick'|'timer'|'wire', + target?: TargetRef, // a scene-node id, a primitive, or the Lumen (default) + key?: string, // for 'key' — declared keys only (e.g. 'ArrowLeft','Space') + rate?: number, // for 'tick' — declared, capped (≤60) — see §5 + everyMs?: number, // for 'timer' + run: TransitionName, // the transition to evaluate +}; +``` + +Rules (normative): + +- **44×44 pt minimum hit-target.** Interactive scene nodes declare/inherit a + ≥44 pt hit area regardless of drawn glyph size (Apple HIG); the runtime + enforces it. +- **Host owns gesture arbitration**, reusing `CONCEPT.md` long-press arbitration + (move >6 px before 400 ms ⇒ drag, else context-invoke). A Lumen never + re-implements disambiguation. +- **No hover dependency.** Hover is decoration only; every hover affordance has a + tap/long-press equivalent. +- **Declared keys are an enhancement.** Every key-bound action has a touch + equivalent (on-screen control) when the host reports no keyboard at handshake. +- **`longPress` is reserved for context-invoke** (action panel + Beam) per + `CONCEPT.md`; a Lumen may *also* bind it but the host's context-invoke wins + unless the Lumen declares `captureLongPress: true` on the target. +- **Bounded wakeups (timers are capped like ticks).** `tick` is rate-capped + (`rate` ≤ 60 Hz, §5). `timer` is bounded the *same* way: `everyMs` has an + enforced **minimum period**, the number of `tick` + `timer` bindings per + Lumen is **count-capped**, and their **combined wakeup rate shares one + per-Lumen budget**. A schedule that exceeds the budget is **rejected at + validation** (`surface_error`), never accepted-then-throttled. Caps are + spike-tunable initial defaults, like gas (§2.4); they keep the §0.2 + "cannot hang / cannot DoS the host" guarantee true for `timer`, not only for + `tick` (a swarm of 1 ms timers each individually under gas is still rejected + in aggregate). + +The handshake (§13) reports **input modalities** (`touch`/`mouse`/`keyboard`/ +`pen`) so Tier 2 composes the right affordances. + +--- + +## 5. Cadence & motion + +Cadence is declared **per node/region, not globally**. 60 Hz never applies to a +whole tree. + +```ts +type CadenceSpec = 'static' | 'reactive' | { tick: number /* Hz, ≤60 */ }; +``` + +| Cadence | Runs | Cost at rest | For | +|---|---|---|---| +| `static` | once; redraw only on patch | zero | layout, copy, imagery, KPIs | +| `reactive` *(default)* | the `view` sub-tree whose `state` slice changed | zero until change | controls, tables, selection | +| `{tick}` | host clock drives a transition at the capped rate, **scoped to that sub-tree** | one bounded transition/frame for that region | game loop, live chart, animation | + +The runtime **dirty-tracks** changed `state` slices and re-evaluates only +dependent `view` branches (retained-mode + memoisation); `requestAnimationFrame` +is scheduled only while a ticking/animating region is live. **At rest a Lumen +costs ~0 % CPU** (kiosk-critical). + +**Declarative animation ≠ LX tick.** *Presentation motion* (fade, glow-pulse, +count-up, camera ease, Ken-Burns, parallax) is a **declarative animation** the +host runs on the GPU — **zero LX per frame**: + +```ts +type Animate = { property, from, to, durationMs, easing, repeat?, delayMs? }; +``` + +Easing/durations come from the Lume motion tokens (`visual-spec.md` §2.11). +Only *simulation* (state evolving by rules) is an LX `tick`. Reduced-motion +(`prefers-reduced-motion`) collapses animations per `visual-spec.md` §2.11. + +--- + +## 6. Capabilities — the mediated doors + +Default-deny. Each capability is declared, effect-classified (`local` / +`internal` / `external-effect`, `CONCEPT.md` §"Security Surface"), granted by +Tier 2, and **brokered** — a Lumen never performs the effect directly. + +```ts +type CapabilityRequest = { cap: CapabilityName, scope?: object }; +``` + +| Capability | Effect | Broker | Notes | +|---|---|---|---| +| `persist` | internal | `memoryStore@1`, Lumen-scoped namespace | high scores, last viewport | +| `loadData` | internal | read-only, size-capped **projection** of a `DataRef` | data-driven viz/maps/workflows | +| `writeData` | internal / external-effect | **Class-D mutation contract** + `writeCapabilities` manifest | commit back (e.g. Jira) | +| `tiles` | internal | **provider-allowlisted** fetch → sprite `DataRef`s | map tiles (OSM/Mapbox) | +| `fetch` | internal / external-effect | allowlisted, agent-approved endpoint | live feed | +| `generateAsset` | internal / external-effect | **omadia-core LLM connectors** (Tier 3) → `DataRef` | image/sound/voice — §6.2 | +| `clipboard` | external-effect | confirmation-modal gate | copy | +| `share` / `savePreset` | external-effect | §8/§9 | publish a Lumen | + +**Mechanic.** A capability call emits an effect-classified action up the +channel; Tier 2 validates it against the granted manifest, brokers it, patches +the result back — or, for `external-effect`, raises the standard confirmation +modal first (`CONCEPT.md`). The deterministic local loop keeps running while a +call is in flight (async-by-default). Imported/shared Lumens surface their +**capability manifest for consent before first run**; capability tokens are +HMAC-scoped like `dataRef`. + +**Broker bounds (anti-DoS / anti-cost).** Because a capability call can be +emitted from a `tick`/`timer`, Tier 2 bounds **egress** the way Tier 1 bounds +compute (§0.2): per-capability **rate + quota**, a **max-in-flight** ceiling, +**idempotent de-duplication** of identical in-flight calls, and **backpressure** +when a broker saturates — so a ticking Lumen cannot move the DoS or cost problem +onto Tier 2/3. Caps are spike-tunable initial defaults (§14). Egress that +carries data **derived from Lumen state or a `DataRef`** (an outbound `fetch`, a +`writeData`) is treated as `external-effect` — per-call confirmation — *unless* +the endpoint **and** request shape were pre-approved at grant time; a bare +`internal` `fetch` may not smuggle state-derived data past the confirmation gate. +The exact quota/idempotency/backpressure contract is a spike deliverable (§14). + +### 6.1 Asset transport & content-addressed caching (never-stale) + +All binaries (images, audio, video, tiles, voice) travel as **`DataRef`s** +(`CONCEPT.md` §"DataRef lifecycle"), which are **content-addressed**: + +> `id = "-"` — same bytes → same id; different +> bytes → different id. **Always.** + +This is **cache-busting by construction** — the structural fix for stale-cache +behaviour: the id *is* the content hash, so changed content is a *different id* +and old content can never be addressed by a new reference. Path: origin +(Tier-3 / connector) → `DataRef{id, signedToken, expiresAt}` → +`surface_data_ref_created` → client fetches **once** via HMAC token → local +content-addressed store keyed by `id` (instant hits across turns/Lumens/canvases, +automatic dedup) → invalidation **only** explicit (`expiresAt` or +`surface_data_ref_invalidated`) → GC when unreferenced and expired. No +time-based "maybe stale" guesswork. + +### 6.2 Generated material comes from omadia-core, not the Lumen + +Generative assets (images, sounds, music, **synthesised voice**) are produced by +the **LLM connectors wired into omadia-core**, never by the Lumen or the Tier-1 +client. The UI only *requests* (`generateAsset`) and *renders* the returned +`DataRef`. Division of labour: **Lumen/Tier 1** declares + renders + animates +(Ken-Burns etc.), generates nothing; **Tier 2** validates/brokers/caches/patches; +**Tier 3 / core connectors** own the model, choice, cost, rate limits, and +return a content-addressed `DataRef`. A regenerated asset is simply a new `id` — +the old bytes GC out, nothing stale lingers. + +### 6.3 High-frequency & transactional interaction (kiosk / ordering) + +The per-call confirmation gate (§6) is sized for *occasional, agent-driven* +external effects, **not** *user-driven high-frequency* ones — a kiosk ordering +flow must not raise a modal on every "add to cart". Three normative patterns keep +it fluid without weakening the gate: + +1. **Local-first, commit-once.** Cart edits, quantities, navigation and form + state are **pure `state`** — `reactive`, no capability, **zero modals**; they + never touch a door out. Only the *terminal* commit (place order) crosses a + single `external-effect` gate. The "20 taps" are local; one tap is brokered. +2. **Session-scoped consent (batched grant).** For genuine multi-write flows a + capability grant MAY be **scoped to a bounded session** — N calls, a time + window, or "until a terminal event" — so the user consents **once** to a + declared, visible, revocable budget instead of per call. Authority is + unchanged (still Tier-2 policy + user consent; the agent still only + *requests*, §0.5); consent is merely amortised over the session. +3. **Optimistic + reconcile (server stays authoritative).** A Lumen is the + *interaction surface*, never the system of record. Inventory, pricing, + payment and order state are **server-authoritative**: the Lumen shows an + **optimistic** local state, Tier 2 brokers the authoritative `writeData` + (reusing the Class-D optimistic-mutation contract), and patches back a + **confirm or rollback**; determinism makes the rollback exact. Payment and + stock consistency live in Tier 3 / the system of record — out of Lumen scope + by design. + +A **transactional ordering Lumen** is therefore the right fifth conformance +artifact (§14): it exercises the capability axis a game never does. + +--- + +## 7. Ports & wires — cross-element interaction + +A Lumen is a node in the same primitive tree; elements interact bidirectionally, +deterministically, on Tier 1 (`interactivity-concept.md` §9.1). + +```ts +type PortSpec = { name: string, dir: 'in'|'out', type: PortType }; // on primitives & Lumens — explicit, wired +type ExposeSpec = { name: string, type: PortType }; // published read-only view-state, bindable by shared id WITHOUT a wire +type Wire = { from: { ref: TargetRef, port: string }, to: { ref: TargetRef, port: string } }; // at container/canvas level +``` + +- **Shared selection / view-state — via a published interface, not ambient + reach.** An element declares a **lightweight published interface**: a small, + named, read-only set of view-state it *offers* to neighbours (`expose`, e.g. + `selection`, `viewport`). A neighbour referencing the same `DataRef` + stable + IDs may then read a **published** field by name — selecting in a `table` that + exposes `selection` highlights the bound markers in a Lumen, with **no explicit + wire and no turn** (Tier-1). The producer decides what is readable; + **un-exposed state stays private**, so an imported or untrusted element can + neither observe a neighbour's internals nor expose more than it declared. This + is *ambient-by-declaration*, not ambient-by-default. +- **Wires** route a node's typed `out` port to another's `in` port by stable id; + the host resolves and propagates at Tier 1 (Class A). Examples: table + selection → map highlight; slider → sim `state` input; a game's `game-over` → + `status` text + `writeData`. +- **Least-privilege.** A node reads **only** what is **wired or published** to it + (an explicit `wire`, or a neighbour's declared `expose` interface bound by + shared id) — it cannot reach arbitrary other elements' internals. Wires and + `expose` declarations are declared data, whitelist-validated, resolve by stable + id ⇒ deterministic, replayable, shared-canvas-safe. Authority split unchanged + (the agent owns which wires and published interfaces exist; the client owns the + values flowing). + +--- + +## 8. Lifecycle, presets & reuse + +The agent **authors rarely and reuses constantly** (`interactivity-concept.md` +§8). A Lumen is a durable, versioned component. + +- **Author once, patch after.** Created once via `surface_snapshot`; lives in + canvas-state; edited by **targeted `surface_patch`** addressed by stable + id/path (not regeneration, not string-replace). "Faster" = a one-line patch to + `tick`. +- **Presets.** A vetted Lumen is saved once: **named, versioned, + content-addressed (`preset-`), parameterised**. Scopes + (first match wins): first-party → tenant (`lumen-presets//shared/**`) + → user (`lumen-presets///**`) → canvas-local. Instantiation is + **deterministic, near-zero-LLM**. +- **Resolve-then-generate.** Before any build, Tier 2 runs a library lookup (same + shape as the pre-Tier-3 data-cache check): exact hit → instantiate; near hit → + fork + patch; miss → cold-author (strong model) + offer to save. +- **Fork & vary.** Copy-on-write → new content-addressed id, parent id recorded + for provenance; targeted patch for the variation. +- **Behaviour-idiom library.** The UI Skill carries vetted LX fragments + (scene-grid, tick-loop, input-binding sets, Lume effect bundles) so even cold + builds assemble audited pieces — cheaper and more consistent. + +--- + +## 9. Sharing + +A Lumen serialises cleanly (validated data + capability manifest). Sharing rides +the `CONCEPT.md` forward-compat hooks: `canvasOwnership` extends to +`{kind:"group", members}`; the channel plugin (the fan-out point) multicasts +surface events. The recipient's Tier 1 **re-validates** and shows the capability +manifest for **consent before first run**. Determinism ⇒ every member runs the +identical Lumen; per-user capability grants ⇒ a shared game saves *your* score, +not *mine*. Real-time multiplayer is v2 but *unblocked* by determinism. + +**Assets travel by content `id`, not by token.** A shared or preset Lumen +carries its asset references as content-addressed `DataRef` **ids** (or an asset +manifest) only — *never* the author's `signedToken`s, which are HMAC-scoped to +the author's `tenant ‖ user ‖ canvasSession` (§6.1) and would either fail for +the recipient or, if made reusable, break the isolation model. On +import/instantiate the recipient's Tier 2 **re-authorises and re-mints** each +`DataRef` token scoped to the *recipient*; an asset the recipient may not access +renders **inert**, never via a borrowed token. + +--- + +## 10. Visual treatment (Lume) + +Lumens render in **Lume — light-as-material**, never glassmorphism +(`visual-spec.md` §1.3: "solid light, not see-through plastic"; no refraction, +no blur-as-chrome). The declarative animation layer (§5) *is* the Lume effect +vocabulary (`visual-spec.md` §3): surface-luminosity gradients, two-stop and +donut **glow**, `glow-core` inner light, directional borders, soft corners, +patch-condensation; Ken-Burns/parallax on assets; light-mote particles. `scene` +text uses the three Lume type registers (`visual-spec.md` §2.7). The only blur +is the transient 800 ms condensation (`visual-spec.md` §3.5). See +`visual-spec.md` §"Lumens & scene in Lume". + +Lume is the **host's own material** and the **no-direction default for a Lumen's +content**: Omadia chrome — header, action panel, Beam, canvas frame — and every +element outside a Lumen always render in Lume, never white-labelled (v1 identity +boundary). A Lumen's content uses `theme` *only* because the agent's default +assumption is that it embeds in an existing Lume UI; where the use case differs — +a kiosk, branded-ordering or product surface that needs the *customer's* colours, +not the accent — the agent chooses `colorMode: 'brand'|'free'` + a declared +`palette` directly (§3.1), no opt-out to fight. Palette and material are +independent: brand colour may ride the Lume *material* (glow, surface-luminosity) +for a premium look, or render flat to match a brand exactly. The no-glassmorphism +rule is about the *material technique* (no refraction, no blur-as-chrome) and +governs the host regardless of a Lumen's palette choice. + +--- + +## 11. Security model (summary) + +| Risk | Mitigation | +|---|---| +| Arbitrary code in the renderer | None runs — LX is a validated AST walked by a shipped interpreter; CSP `default-src 'self'`, no `unsafe-eval` | +| Runaway / DoS | Gas + frame ceiling + bounded iteration + state cap + **capped wakeup budget** (`tick` + `timer`, §4) → halt/reject with `surface_error`, never the canvas; capability **egress** is broker-bounded (rate/quota/max-in-flight/idempotent/backpressure, §6) so a tick-driven call cannot DoS Tier 2/3 | +| Iterative compute (sort/pathfind/layout) | **Native kernels** (§2.6), not agent-authored loops: audited host code under a per-call **kernel-gas** ceiling, total on degenerate input, deterministic, no IO — the agent calls but never writes one, so "no arbitrary code" and "cannot hang" both hold | +| Corrupt generated state (silent-wrong) | **Declared invariants** (§2.7) checked post-transition → rollback + `surface_error`; **golden-trace** author-time gate (§14) runs example traces before first render | +| Data exfiltration | Default-deny capabilities; Lumen reads only own state + **wired/`expose`-published** ports; all egress brokered, allowlisted, confirmed, Trace-audited; **state/`DataRef`-derived** `fetch`/`writeData` classified `external-effect` (confirmed) unless pre-approved at grant (§6) | +| Stale / poisoned assets | Content-addressed `DataRef` (id = content hash); HMAC-scoped fetch; explicit invalidation | +| Untrusted shared/imported Lumen | Re-validated on import; capability manifest consent before first run; HMAC scoping; fork lineage | +| Cross-element overreach | Port/wire **+ published-interface** least-privilege — a node reads only what is **wired or `expose`-published** to it; un-exposed state is unreadable (an imported Lumen sees no ambient neighbour state); declared, whitelist-validated, stable-id-resolved | +| Non-reversible effects | `external-effect` class → confirmation-modal gate before the real call | + +--- + +## 12. Wire & SDK deltas (additive over 1.0) + +- **Tree content:** `behavior`/`lumen` section (§1) and the `scene` primitive + (§3) — validated by the extended whitelist parser (schema + LX-AST). Carried + in existing `surface_snapshot` / `surface_patch`. +- **LX-AST:** `map`/`filter`/`fold` binder nodes (with optional `idx`) + the + `{var}` read node + `at`/`setAt` computed indexing (1-D and 2-D over `grid` + **and** `list`) + `flatten` (§2.2/§2.3), the **native-kernel** whitelist + (§2.6), and optional `invariants` (§2.7) — additive AST content, statically + validated. +- **Constants:** the immutable `const` section + `{const}` read node (§1.2) — + agent-owned, bounded, not serialised; the unit the idiom/preset library ships. +- **Ports & wires:** `ports` and `expose` (published read-only interface) on + primitives/Lumens, `wires` at container/canvas level (§7) — additive tree + content, Tier-1-resolved. +- **Cadence & animation:** `cadence` (§5) and `animate` descriptors — additive + trait content. +- **Colour authority:** `colorMode` + `palette` on the Lumen (§3.1) — opens + brand/free colour for the Lumen's own content; chrome stays Lume. Additive, + declared data; the normaliser stops clipping in `brand`/`free`. +- **Events:** `surface_capability_request` (client→Tier 2) and + `surface_capability_result` (Tier 2→client) for §6 brokering; results may also + arrive as ordinary `surface_patch`. Reuses the effect-classified action path + and `surface_action_result`. One optional new event family. +- **Presets:** `lumen-presets/**` + `lumen-state/**` memoryStore namespaces (§8). +- **Handshake (§13):** client declares LX version, gas limits, scene support, + granted capability classes, and **input modalities** alongside + `localOperations`. + +Classic channels and 1.0-only clients see none of this — all additive, behind +the `canvas` capability and **gated by boot negotiation** (§13): a client that +does not negotiate `scene` / LX / capability support is never sent 1.1 content, +and unknown *types / tree sections* are hard-rejected (not silently ignored) by +the 1.0 whitelist (`protocol/1.0.md` §2). + +--- + +## 13. Versioning & negotiation + +LX, the scene vocabulary and the capability catalog are versioned with +`omadia-canvas-protocol` and negotiated at boot. The 1.0 `handshake_select` +(`protocol/1.0.md` §1) gains additive fields: + +```ts +handshake_select += { + lxVersion?: string, // e.g. "1.1" + lxGasLimit?: number, // client's per-eval gas ceiling + sceneSupport?: 'none'|'canvas2d'|'webgl', + kernels?: KernelName[], // §2.6 native kernels this client implements + kernelGasLimit?: number, // client's per-kernel-call gas ceiling + capabilityClasses?: CapabilityName[], // what this client can broker/render + inputModalities?: ('touch'|'mouse'|'keyboard'|'pen')[], +} +``` + +A client may implement a subset (e.g. `scene` but not `tiles`); Tier 2 routes +accordingly and idioms degrade gracefully — the same principle as +`localOperations`. + +--- + +## 14. Conformance & open questions + +Conformance is the schema set in `schema/` (Lumen, LX-AST **incl. the +`map`/`filter`/`fold` binder nodes, `at`/`setAt`, and the §2.6 kernel +signatures**, scene, ports/wires/`expose`, invariants, capability manifest) + +accept/reject fixtures, plus **five** reference Lumens — an arcade game, +interactive workflow, **a transactional ordering flow (§6.3)**, defrag-viz, map — +each authored **by hand in real LX-AST** and traced end-to-end like +`walkthroughs.md`. + +**Hand-authoring the reference set is the acceptance gate *before* implementation +budget is committed.** It is the cheapest test that the binder forms, the kernel +cut, the invariant/golden-trace loop and the transaction patterns actually hold +against a non-trivial artifact — not only in prose. (Tracing a board-game-class +Lumen on paper is exactly what surfaced the Rev-3 gaps; doing the full set +converts "argued watertight" into "tested watertight".) + +**Golden-trace authoring gate.** Because behaviour is deterministic, cold +authoring SHOULD emit, alongside the Lumen, a few example `(input events → +expected state)` traces; Tier 2 runs them in the interpreter and ships the Lumen +only if they pass — converting a class of silent-wrong generation into a +caught-before-render failure. Preset/idiom **assembly** (§8) stays the primary +path; novel cold authoring is a strong-model job with a real failure rate, and +this gate plus declared invariants (§2.7) are its safety net. + +Open tuning items (gas/frame/state caps, the **kernel-gas schedule and v1.1 +kernel cut**, wakeup-budget caps for `tick`+`timer`, the capability-broker egress +contract — rate/quota/max-in-flight/idempotency/backpressure, **session-scoped +consent budgets**, LX std-lib surface, scene perf ceiling, capability-consent +granularity, determinism-vs-real-time, LLM reliability emitting LX, preset +trust/distribution) are enumerated in `interactivity-concept.md` §13 — research +items, not unspecified holes. diff --git a/docs/mockups/kiosk-lumen-aura.html b/docs/mockups/kiosk-lumen-aura.html new file mode 100644 index 0000000..a1b0291 --- /dev/null +++ b/docs/mockups/kiosk-lumen-aura.html @@ -0,0 +1,393 @@ + + + + + +Omadia UI — Kiosk Lumen mockup (Lume · Lagoon) + + + + + + + +
+
+ + +
+
+
+
+
+
+ +
+
+
+
New · 2026
+

AURA Halo

+

A speaker that lights the room it fills. Spatial sound, condensed out of light.

+
+
+ + +
+
+
AURA Studio
+
--:--:--
+
+ +

Tap a feature to explore. Everything you see was composed by the agent — layout, motion and copy — and rendered natively in Lume.

+ +
+ + + + +
+ +
+ + +
+
+
+ +
+ +
+ + + + diff --git a/docs/visual-spec.md b/docs/visual-spec.md index 00d0920..037ea35 100644 --- a/docs/visual-spec.md +++ b/docs/visual-spec.md @@ -5,6 +5,14 @@ > Geist (structural) · Source Serif 4 (prose) · Geist Mono (data/code). > Codex-review-ready in the CONCEPT.md cadence. +Version 0.5 — **Lumens & `scene` visual treatment (§4.13).** Pins how the +Live-Interactivity extension renders in Lume: `scene` is editor-class +(`radius.0`); draw-list colours are tokens only (always on-theme); presentation +motion is the declarative Lume effect vocabulary (§3) on the GPU, not pixel +math and **not glass** (the §1.3 NOT-list holds for Lumens); cadence is +per-region. Companion to `../docs/interactivity-concept.md` (concept) and +`../docs/lumens-spec.md` (definition). + Version 0.4 — **Surface-nesting ladder & chrome budget.** Closes the two spec gaps that produced doubled chrome in the first shipped canvases: the spec never said which surface a nested container gets (§2.13 — the ladder: @@ -1265,6 +1273,54 @@ Tier-1 boundary where the agent's chrome ends and the user's raw work begins. --- +## 4.13 Lumens & `scene` — visual treatment + +The Live-Interactivity extension (`../docs/interactivity-concept.md` rationale, +`../docs/lumens-spec.md` definition) adds a 25th primitive, **`scene`** — a +declarative immediate-mode draw surface for games, custom visualisations and +maps — and **Lumens** (self-contained interactive units). Both render in Lume; +this section pins how. + +**`scene` is editor-class — `radius.0`, sharp edges.** Like `canvas-region` +and `timeline` (§2.9), a `scene` is where Lume material stops; the hard edge is +the Tier-1 boundary marker. Its `camera` pan/zoom reuses the canvas-region +affordances. + +**Draw-list colours are tokens by default, chosen from intent for the Lumen's own +content.** The *no-direction default* is Lume tokens (`accent`, `accent.glow*`, +surface/text/semantic), justified by the assumption that the Lumen embeds in an +existing Lume UI — so a game board, defrag grid or map-marker layer sits in +on-theme. That assumption is **not universal**: where a user's kiosk, +branded-ordering or product surface needs *their* brand, the agent picks +`colorMode: 'brand'|'free'` + a `palette` (`../docs/lumens-spec.md` §3.1) +**directly** to use **any** colour — `theme` is the fallback, not a preference. This scopes to the **Lumen's own subtree only**: **Omadia chrome (header, +action panel, Beam, canvas frame) always stays Lume** — no host white-label in +v1. Brand colour may still ride the Lume material (glow/luminosity) or render +flat; in `brand`/`free` the normaliser does not clip and enforces no contrast +floor (author owns accessibility). `scene` `text` nodes use the three type +registers (§2.7). + +**Motion = the Lume effect vocabulary, declarative.** Presentation motion on a +Lumen (fade, glow-pulse, count-up, camera ease, Ken-Burns on a `sprite`, +parallax, light-mote particles) is a **declarative animation the host runs on +the GPU** — it composes the §3 primitives (two-stop glow §3.2, donut §3.3, +surface gradient §3.1, condensation §3.5) and the §2.11 motion tokens. It is +**not** per-frame pixel math, and it is emphatically **not glass** — the §1.3 +"NOT" list holds for Lumens too (no refraction, no blur-as-chrome, +glassmorphism stays out). The only blur is the transient condensation (§3.5). + +**Cadence is per-region (§ render-cadence in the concept).** `static` and +`reactive` regions cost ~0 % CPU at rest; only a `{tick}` region animates — a +kiosk Lumen is mostly static beautiful surface with a few lit, moving accents. +A reference mockup lives at +[`./mockups/kiosk-lumen-aura.html`](./mockups/kiosk-lumen-aura.html). + +**Touch-first.** Lumen hit-targets honour the 44 pt minimum; the material's +soft glow/halo affordances replace hover (which is dropped as a requirement) — +see `../docs/lumens-spec.md` §4. + +--- + ## 5. Composition idioms The five idioms from CONCEPT.md's Composition-Idiom Library, rendered in