Proposal: Incremental Streaming Support for Large LLM Outputs using updateDataModel

[Emiyaaaaa]

Background

A2UI provides a clean separation between UI structure and application state through the updateDataModel message.

This model works well for typical UI updates. However, when building AI-native applications (chat interfaces, report generation, document synthesis), developers frequently stream large LLM outputs into a single data model field.

Modern LLM responses may reach:

• tens of thousands of tokens
• hundreds of KB of text
• continuously streamed outputs

---

Problem

According to the current v0.9 specification, updateDataModel replaces the value at the specified path:

{
  "version": "v0.9",
  "updateDataModel": {
    "surfaceId": "chat",
    "path": "/messages/0/content",
    "value": "Hello world"
  }
}

During streaming generation, servers typically accumulate tokens:

"H"
"He"
"Hel"
"Hell"
"Hello"

and repeatedly send updates:

{
  "version": "v0.9",
  "updateDataModel": {
    "surfaceId": "chat",
    "path": "/messages/0/content",
    "value": "Hello"
  }
}

This causes the transmitted payload size to grow continuously as content becomes larger.

For long responses, total network usage grows approximately O(n²).

---

Observation

The current protocol semantics are replace-based, not incremental.

While functionally correct, this creates inefficiencies for one of A2UI’s primary emerging workloads: LLM streaming.

---

Proposed Improvement

Introduce an incremental update pattern compatible with the existing updateDataModel structure.

Instead of redefining message types, this proposal suggests extending semantics via an optional operation mode.

Example:

{
  "version": "v0.9",
  "updateDataModel": {
    "surfaceId": "chat",
    "path": "/messages/0/content",
    "op": "append",
    "value": " world"
  }
}

Client behavior:

existing_value += value

rather than replacing the value.

---

Why This Matters

Efficient streaming enables:

✅ scalable chat interfaces
✅ document-scale generation
✅ reduced bandwidth consumption
✅ smoother progressive rendering
✅ improved mobile performance

Without incremental updates, developers must repeatedly resend the full accumulated state.

---

Alternative (Non-Spec) Workarounds Today

Current implementations typically resort to:

1. Server-side batching (time-based buffering)
2. Splitting messages into multiple chunks
3. Creating new components per segment

While workable, these approaches introduce additional complexity that could be standardized.

---

Compatibility

This proposal is fully backward compatible:

• Existing clients ignore unknown properties
• Default behavior remains replace semantics
• No breaking schema changes required

If op is omitted:

updateDataModel → replace (current behavior)

If:

op = "append"

client performs incremental mutation.

---

Motivation

A2UI is increasingly positioned as a protocol for AI-driven, streaming-first interfaces.

Standardizing an incremental data update mechanism would:

• reduce duplicated adapter implementations
• align naturally with token streaming LLMs
• improve performance for large-scale outputs

[shan-s]

Hi, this can be achieved in the catalog/component level by buffering tokens without modifying the protocol. Please have a look at the implementation in Freesail Chat Catalog

[Emiyaaaaa]

Hey, this token diff plan can't handle it well when the LLM out is ["this", "is", "a", "very", "very", "important", "thing"], multiple "very" will be combined into one.

[shan-s]

Yes, indeed this is a bug. Thanks for spotting it. Right now it does not impact us because we are doing a full updateDataModel at the end of the stream. But we are looking to introduce timestamps in the front-end so other use cases can be accommodated.

[shan-s]

We have handled this with timestamps now.