fix(examples-chat): handle Responses-API function_call content blocks#266
Merged
Conversation
gpt-5 streams tool-call deltas via OpenAI's Responses API, which the
langchain-openai integration surfaces as content blocks on AIMessageChunk
rather than as classic tool_call_chunks:
message.content = [{type: 'function_call', name?, call_id?, arguments,
index}, ...]
The first block for each tool call carries name + call_id; subsequent
blocks for the same index carry only the args delta. The previous
handler only read message.tool_call_chunks (empty under Responses API),
so the bridge stayed dormant despite on_llm_new_token firing.
This patch:
- Reads BOTH message.tool_call_chunks (Chat Completions) AND
message.content blocks of type 'function_call' (Responses API).
- Maintains an index → tool_call_id mapping so subsequent Responses-API
blocks (which omit call_id) are attributed to the right call.
- Filters non-target tools at both paths.
- Adds 2 new tests covering the Responses-API path: incremental block
attribution by index, and ignored non-target tools.
Live verification (langgraph log): handler now fires per token and
adispatch_custom_event reaches the wire. Frontend partial-args bridge
will populate liveSurfaceStore mid-stream.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
blove
added a commit
that referenced
this pull request
May 12, 2026
PR #266 fixed the on_llm_new_token signature, after which live diagnostics confirmed the handler ran per token and adispatch_custom_event completed successfully — yet the SSE stream still carried zero a2ui-partial events. Root cause: adispatch_custom_event (langchain_core) and stream_mode= 'custom' (LangGraph) are different layers. langchain_core dispatches visible via stream_mode='events'; LangGraph's 'custom' channel is fed by get_stream_writer() returned from langgraph.config. The transport already requests stream_mode='custom', but the handler was writing to the wrong sink. Fix: replace adispatch_custom_event with get_stream_writer(). The writer is contextvar-scoped to the currently-executing LangGraph node and is inherited by nested callbacks, so the handler can call it from inside the LLM's callback chain. The payload shape is {name, data} so the existing transport-side parser (stream-manager.bridge.ts:509) extracts name + data correctly. Tests updated to mock get_stream_writer (instead of adispatch_custom_event) and assert on writer.call_args. Adds a new test asserting graceful behavior when invoked outside a stream context (writer raises RuntimeError; handler swallows). Live smoke now confirms 758 a2ui-partial events on the wire across a 2.5MB SSE stream for a dashboard prompt. The bridge consumes all events into agent.customEvents() and forwards them to the partial-args bridge. Note: a follow-up frontend issue remains in the partial-args bridge's incremental dispatch logic — early surfaceUpdate envelopes dispatch with incomplete components arrays (no ids yet), preventing beginRendering synthesis, and the dispatchedCount counter then skips re-attempt. Tracking separately.
3 tasks
blove
added a commit
that referenced
this pull request
May 12, 2026
#268) PR #266 fixed the on_llm_new_token signature, after which live diagnostics confirmed the handler ran per token and adispatch_custom_event completed successfully — yet the SSE stream still carried zero a2ui-partial events. Root cause: adispatch_custom_event (langchain_core) and stream_mode= 'custom' (LangGraph) are different layers. langchain_core dispatches visible via stream_mode='events'; LangGraph's 'custom' channel is fed by get_stream_writer() returned from langgraph.config. The transport already requests stream_mode='custom', but the handler was writing to the wrong sink. Fix: replace adispatch_custom_event with get_stream_writer(). The writer is contextvar-scoped to the currently-executing LangGraph node and is inherited by nested callbacks, so the handler can call it from inside the LLM's callback chain. The payload shape is {name, data} so the existing transport-side parser (stream-manager.bridge.ts:509) extracts name + data correctly. Tests updated to mock get_stream_writer (instead of adispatch_custom_event) and assert on writer.call_args. Adds a new test asserting graceful behavior when invoked outside a stream context (writer raises RuntimeError; handler swallows). Live smoke now confirms 758 a2ui-partial events on the wire across a 2.5MB SSE stream for a dashboard prompt. The bridge consumes all events into agent.customEvents() and forwards them to the partial-args bridge. Note: a follow-up frontend issue remains in the partial-args bridge's incremental dispatch logic — early surfaceUpdate envelopes dispatch with incomplete components arrays (no ids yet), preventing beginRendering synthesis, and the dispatchedCount counter then skips re-attempt. Tracking separately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Third hotfix in the progressive A2UI streaming chain. PR #264 fixed the callback method name (`on_llm_new_token` instead of the non-existent `on_chat_model_stream`); live diagnostics confirmed the handler now fires per token — but `adispatch_custom_event` still never ran because the handler was reading the wrong field on AIMessageChunk.
Root cause
gpt-5 uses OpenAI's Responses API, not Chat Completions. The langchain-openai integration surfaces tool-call deltas under that API as content blocks on AIMessageChunk:
```python
chunk.message.content = [
{"type": "function_call", "name": "render_a2ui_surface",
"call_id": "call_ABC", "arguments": "", "index": 1}, # 1st block
{"type": "function_call", "arguments": "{\"en", "index": 1}, # subsequent
{"type": "function_call", "arguments": "velopes", "index": 1},
...
]
```
`message.tool_call_chunks` is empty under this API. The previous handler only read tool_call_chunks → dormant.
Fix
Read BOTH delta shapes:
For the Responses-API path, track an `index → tool_call_id` mapping (the first block per call carries call_id; subsequent blocks for that index carry only the args delta) so we attribute later deltas to the right call.
Tests
Live diagnostic confirms handler now reaches `adispatch_custom_event` for every `function_call` delta.
Test plan