Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CONCEPT.md
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,7 @@ The system uses a small, disciplined set of light-domain names because Lume is t
| **Trace** | the command history / audit log of a canvas — a "trace of light" through what the user asked and the agent answered | introduced v0.12 |
| **Flare** | reserved — for agent-initiated attention signals (e.g. notifications, "needs your input") | reserved, may activate if and when we introduce notification concepts |
| **Spark** | reserved — for a discrete generative-initiation event distinguishable from a beam | reserved, may activate if a use case clearly differs from Beam |
| **Lumen** | a self-contained, declarative, deterministic interactive unit — UI condensed into a portable, shareable quantum of light (the Live-Interactivity extension) | **proposed**, pending vocabulary sign-off — see [`docs/interactivity-concept.md`](docs/interactivity-concept.md) + [`docs/lumens-spec.md`](docs/lumens-spec.md) |

Anything else from the light domain (Photon, Shadow, Beacon, Cast, Lens, Prism, Reflection, Ray, Shine, Glimmer, …) does **not** enter the vocabulary unless it earns its slot by labelling something that has no good name yet. The default answer to "should we call this X?" is no.

Expand Down Expand Up @@ -1200,6 +1201,39 @@ This section becomes the input for the v2 design phase. It is not a v1 deliverab

---

## Extension — Live Interactivity (Lumens)

A planned **additive** extension brings rich, agent-generated, **Tier-1-fast**
interactivity to the canvas — small games, interactive data workflows,
unusual visualisations (defrag-style), live maps — the Omadia answer to
sandbox-style "live artifacts", deliberately re-aimed. Its thesis: most of what
arbitrary-code sandboxes block is blocked by missing **capabilities**, not
missing **compute** — so it **constrains computation** (a declarative, bounded,
deterministic, interpreted behaviour model — no arbitrary code, the whitelist
parser extends to it) and **opens capabilities** (real data, write-back,
allowlisted network, generated assets) **mediated** through the existing Tier-2/3
orchestration and effect classification.

The unit is a **Lumen**: `state + transitions + view + events + capabilities`,
plus a new `scene` primitive (a declarative draw surface), declarative
ports/wires for cross-element interaction, per-region render cadence, and a
preset library so the agent **authors once and reuses constantly** (not rebuilt
per turn). It is **forward-compatible** — Lumens ride the same `surface_*`
grammar, the same DataRef/HMAC scoping, the same authority split, the same
shared-canvas hooks; the deltas are additive (a `behavior` tree section, the
`scene` primitive → `omadia-canvas-protocol/1.1`, one optional
`surface_capability_*` event family).

- **Rationale / concept:** [`docs/interactivity-concept.md`](docs/interactivity-concept.md)
- **Normative definition / spec:** [`docs/lumens-spec.md`](docs/lumens-spec.md)
- **Lume visual treatment:** [`docs/visual-spec.md`](docs/visual-spec.md) §4.13
- **Reference mockup:** [`docs/mockups/kiosk-lumen-aura.html`](docs/mockups/kiosk-lumen-aura.html)

Not a v1 deliverable of this base concept; tracked as its own implementation
workstream.

---

## Riskiest Assumptions

1. **Top-tier and fast LLMs can reliably emit valid primitive trees as tool-use JSON.** Likely yes for Sonnet/Opus; unproven for Haiku-class on UI-tree synthesis (omadia uses Haiku only as classifier today). Mitigation: `ui_orchestrator_model` configurable; spike with Haiku first, fall back to Sonnet if reliability below threshold.
Expand Down
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ Omadia UI is the next layer — a desktop application where the agent
- 🗂️ **A real workspace.** Multiple live canvases with their own server
sessions, tiling split panes, named desktops, a canvas library — all synced
through the omadia registry and restored on restart.
- 🎮 **Live, safe interactivity (Lumens).** A planned additive extension lets
the agent generate self-contained interactive units — a game, an interactive
workflow, an unusual visualisation, a live map — as **declarative,
deterministic data** run by a bounded Tier-1 interpreter (no arbitrary code),
with capabilities mediated through Tiers 2/3. Shareable, presettable. See
[`docs/interactivity-concept.md`](docs/interactivity-concept.md) +
[`docs/lumens-spec.md`](docs/lumens-spec.md).

## ⚡ Quickstart

Expand Down Expand Up @@ -82,7 +89,9 @@ refactor).
| File | Purpose |
|---|---|
| [`CONCEPT.md`](CONCEPT.md) | Architecture, primitives, protocol, security, identity, SDK extension plan |
| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.4) |
| [`docs/interactivity-concept.md`](docs/interactivity-concept.md) | **Live Interactivity (Lumens)** — concept/rationale: Tier-1-fast, agent-generated, safe interactivity (games, workflows, maps) |
| [`docs/lumens-spec.md`](docs/lumens-spec.md) | **Lumens** — normative definition: LX, `scene`, events/touch, capabilities, ports/wires, presets (`omadia-canvas-protocol/1.1` draft) |
| [`docs/visual-spec.md`](docs/visual-spec.md) | Lume material system — tokens, rules, composition idioms (v0.5) |
| [`docs/protocol/1.0.md`](docs/protocol/1.0.md) | Protocol specification + machine-validatable JSON Schemas |
| [`docs/walkthroughs.md`](docs/walkthroughs.md) | Use-case walkthroughs — multi-source comparison + editor micro-task |
| [`docs/tech-stack.md`](docs/tech-stack.md) | Tech-stack decision for the host app (Electron), with reasoning |
Expand Down
27 changes: 27 additions & 0 deletions docs/implementation-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -699,6 +699,33 @@ tied to the phase that exposes each one.

---

## 10. Lumens (Live Interactivity) extension — implementation outline

A separable, **additive** workstream on top of the v1 baseline above (it
depends on the surface event family, the validator, and the orchestrator
already landing). Concept: [`interactivity-concept.md`](interactivity-concept.md);
normative definition: [`lumens-spec.md`](lumens-spec.md). Tracked for the
maintainer as a GitHub issue. Suggested phasing, smallest mergeable units first:

| Phase | Deliverable | Where |
|---|---|---|
| **L0 — schemas** | JSON Schemas for the Lumen `behavior` section, the LX-AST node set, `scene`, `ports`/`wires`, the capability manifest; accept/reject fixtures | `docs/protocol/schema/` (this repo) → `omadia-canvas-protocol/1.1` |
| **L1 — LX interpreter (Tier 1)** | deterministic AST evaluator with gas + frame ceiling, bounded iteration, seeded `random`/`now`; the extended whitelist validator | host app `app/src/renderer/` |
| **L2 — `scene` primitive** | draw-list rasteriser (canvas2d first, WebGL behind it), token-only styling, buffer-native hit-testing → `TargetRef` | host app renderer |
| **L3 — cadence & animation** | per-region `static`/`reactive`/`{tick}` dirty-tracking + rAF scheduling; declarative `animate` layer on the Lume effect vocabulary; reduced-motion | host app renderer |
| **L4 — events & touch** | `tap`/`longPress`/`drag`/`pinch`/`swipe`/`key`/`tick`; 44 pt hit-targets; host gesture arbitration; input-modality handshake fields | host app + channel |
| **L5 — capabilities broker (Tier 2)** | `persist`/`loadData`/`writeData`/`tiles`/`fetch`/`generateAsset`/`clipboard`; effect-classified brokering + confirmation gate; `surface_capability_*` events; asset transport + content-addressed cache | `omadia-ui-orchestrator` + `omadia-ui-channel` + `byte5ai/omadia` core connectors for `generateAsset` |
| **L6 — ports & wires** | typed ports on primitives/Lumens, Tier-1 wire resolution, shared `viewState.selection` cross-element | host app + orchestrator |
| **L7 — lifecycle & presets** | author-once/patch; `lumen-presets/**` + `lumen-state/**` stores; resolve-then-generate lookup; fork+patch; behaviour-idiom library in the UI Skill | `omadia-ui-orchestrator` |
| **L8 — sharing** | `canvasOwnership` group extension + channel fan-out + import consent (rides the v2 shared-canvas hooks) | channel + orchestrator |
| **L9 — reference Lumens** | an arcade game · interactive workflow · defrag-viz · map, traced end-to-end like `walkthroughs.md`; conformance fixtures | this repo |

**Riskiest items** (mirror `interactivity-concept.md` §13): LLM reliability
emitting valid LX (likely a strong-model authoring job, fast-model patching);
gas/scene-perf calibration on the four reference Lumens; capability-consent UX.

---

## Appendix — corrected critical-file map (omadia core @ `83ef79b`)

The concept's "Critical files" section with line numbers refreshed against live
Expand Down
Loading
Loading