Add live web-search status streaming for chat completions#84
Open
adambalogh wants to merge 2 commits into
Open
Add live web-search status streaming for chat completions#84adambalogh wants to merge 2 commits into
adambalogh wants to merge 2 commits into
Conversation
When a provider runs a native web search mid-stream, surface it to the
client so the UI can show a "searching the web" indicator instead of a
silent pause.
- llm_backend: add extract_web_search_events(), a per-chunk best-effort
detector for web_search_call (OpenAI) / server_tool_use (Anthropic)
blocks, deduped by block id/index across chunks, with best-effort query
extraction
- chat_controller: in the streaming loop, emit a lightweight SSE frame
({"web_search": {"status": "searching", "query"?}}) on each newly-seen
search. The frame carries no content delta, so it is excluded from the
signed output hash and does not affect per-search billing
- tests: per-chunk detection/dedup/query extraction and streaming-path
emission (one frame per deduped search; none when web_search is off)
https://claude.ai/code/session_01N5NjTA5rQhVGuLzNDmAs86
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds real-time "searching the web" status indicators to streaming chat completions when providers (Anthropic, OpenAI) perform native web searches mid-stream. Clients can now display live UI feedback while the model browses the web, improving perceived responsiveness.
Changes
New function
extract_web_search_events()inllm_backend.py:server_tool_useand OpenAIweb_search_callformats)inputoractionfields)Helper function
_web_search_query_from_block()inllm_backend.py:Noneduring streaming when query is not yet fully accumulatedWeb-search status frame emission in
chat_controller.py:"web_search": {"status": "searching", "query": <str|null>}when new searches are detectedComprehensive test coverage in
test_web_search.py:TestExtractWebSearchEvents: 10 unit tests covering None/plain-text inputs, both provider formats, query extraction, deduplication, and edge casesTestChatControllerWebSearchStreaming: 2 integration tests verifying status frames are emitted exactly once per search and not emitted when web search is disabledImplementation Details
extract_web_search_count()(which counts completed searches for billing)(block_type, id_or_index)tuples to handle Anthropic's incremental JSON delta streaming where the same block reappears across chunksNonewhen query is not yet availablehttps://claude.ai/code/session_01N5NjTA5rQhVGuLzNDmAs86