You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During integration testing of the realtime examples (livekit-chat, livekit-translation, livekit-voice), we discovered multiple issues caused by incompatible I/O between engrams:
Output schema mismatches that kill hub streams (bobravoz-grpc#63, livekit-bridge-engram#18)
Silent template resolution failures when packet.* fields do not exist (bobravoz-grpc#64)
The shouldEncodeStructuredStreamPayload MimeType trap (bubu-sdk-go#74 comment)
All of these boil down to: engram I/O contracts are implicit and untested. You only discover incompatibilities at runtime in a full Kubernetes deployment.
Proposal
Add a registry validation layer that tests engram I/O compatibility outside Kubernetes in a lightweight sandbox:
What it validates
Output schema conformance — feed sample inputs to an engram, verify outputs match the EngramTemplate outputSchema
Pairwise compatibility — given engram A output schema and engram B input schema, verify they are compatible without deploying
Runtime template resolution — verify that packet.* and steps.* references in story templates can actually resolve given the upstream engram output structure
Payload isolation — verify that a step output contains ONLY the declared fields, not leaked upstream config
How it works
Runs in a sandbox (Docker container or local process), not in Kubernetes
Uses the engram container image directly with mock transport
bobravoz-grpc#67 — per-engram image overrides enable testing candidate versions in the sandbox before promoting
Why this matters
Right now, the only way to test engram compatibility is to deploy a full story on a Kubernetes cluster and read pod logs. This does not scale. A sandbox validator would catch I/O mismatches before deployment, enable CI/CD pipelines for engram development, and reduce the debugging cycle from reading 8 pod logs to running one test command.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Context
During integration testing of the realtime examples (livekit-chat, livekit-translation, livekit-voice), we discovered multiple issues caused by incompatible I/O between engrams:
packet.*fields do not exist (bobravoz-grpc#64)shouldEncodeStructuredStreamPayloadMimeType trap (bubu-sdk-go#74 comment)All of these boil down to: engram I/O contracts are implicit and untested. You only discover incompatibilities at runtime in a full Kubernetes deployment.
Proposal
Add a registry validation layer that tests engram I/O compatibility outside Kubernetes in a lightweight sandbox:
What it validates
packet.*andsteps.*references in story templates can actually resolve given the upstream engram output structureHow it works
Registry integration
Relationship to existing RFCs
Why this matters
Right now, the only way to test engram compatibility is to deploy a full story on a Kubernetes cluster and read pod logs. This does not scale. A sandbox validator would catch I/O mismatches before deployment, enable CI/CD pipelines for engram development, and reduce the debugging cycle from reading 8 pod logs to running one test command.
Beta Was this translation helpful? Give feedback.
All reactions