You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Master issue: [Streaming Tools] support streaming intermediate results for tools for non-streaming case #2014
** Please make sure you read the contribution guide and file the issues in the right place. ** Contribution guide.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Also very interested in this functionality. Have a requirement to support streaming intermediate tool results for non-Live models in my current application. Any word on whether the proposed PRs are viable approaches to enable this functionality?
I'm using streaming tools to make an Agent Tool asynchronous, since otherwise the voice agent will freeze while the agent tool is doing its thing.
This is how I implemented it:
defmake_agent_tool_async(agent_tool: AgentTool):
"""Create a streaming tool that wraps an LlmAgent via AgentTool. The returned function: - yields exactly once with {"status": "started"} - then waits for the agent to finish - then yields {"status": "done", "reply": <agent reply>} """asyncdefexecute_async_agent_tool(
request: str,
tool_context: ToolContext,
) ->AsyncGenerator[dict[str, Any], None]:
pending=tool_context.state.get("pending_jobs", {})
ifagent_tool.agent.nameinpending:
yield {
"status": "already running - wait for completion - do not call again until done",
}
log.debug(
f"Agent tool {agent_tool.agent.name} already running"
)
returnpending[agent_tool.agent.name] ="started"tool_context.state["pending_jobs"] =pendingyield {
"status": "started",
}
log.debug(
f"Started long-running agent tool for agent_name={agent_tool.agent.name}"
)
reply=awaitagent_tool.run_async(
args={"request": request},
tool_context=tool_context,
)
yield {
"status": "done",
"reply": reply,
}
tool_context.state["pending_jobs"].pop(agent_tool.agent.name, None)
log.debug(f"Completed long-running agent tool for agent_name={agent_tool.agent.name}")
execute_async_agent_tool.__name__=agent_tool.name# TODO maybe use _get_declaration to copy over function signature?base_desc=agent_tool.agent.descriptionor""execute_async_agent_tool.__doc__=base_desc+ (
"\n\nNOTE: This is a long-running streaming version of this tool. ""It first returns status='started', then later returns status='done' with the reply. ""If called while already running, returns status='already running'. ""Wait for status='done' before calling this tool again."
)
returnexecute_async_agent_toolvoice_agent=LlmAgent(
name="voice_agent",
model=voice_model,
instruction=....,
tools=[
make_agent_tool_async(
AgentTool(
text_agent,
skip_summarization=True,
)
),
],
)
Is there a better way to do this?
With some prompting on the voice agent side this sort of works.
I also have a question: I do not see the reply in the events, but the model sees it.
How are those streaming results treated? Why don’t they generate a function response event?
Yes, ADK has a built-in Streaming Tools pattern for Live API. Instead of wrapping with AsyncGenerator, you can define a tool that accepts an input_stream: LiveRequestQueue parameter:
fromgoogle.adk.agents.live_request_queueimportLiveRequestQueuefromgoogle.genaiimporttypesasyncdefexecute_agent_tool(
request: str,
tool_context: ToolContext,
input_stream: LiveRequestQueue, # ADK auto-injects this
) ->dict:
"""Long-running agent tool that streams progress to the model."""# Send status update to modelinput_stream.send_content(
types.Content(parts=[types.Part(text="Starting agent execution...")])
)
# Do the workreply=awaitsome_long_running_work(request, tool_context)
return {"status": "done", "reply": reply}
How it works:
ADK inspects tools at run_live() initialization and detects the LiveRequestQueue type annotation (runners.py:828-865)
ADK creates a dedicated LiveRequestQueue for each streaming tool, stored in invocation_context.active_streaming_tools[tool_name]
When the model calls the tool, ADK automatically injects the queue as input_stream (function_tool.py:238-253)
Updates sent via input_stream.send_content() go directly to the Live API
2. Why don't streaming results generate function response events?
This is by design. Streaming tool updates sent via LiveRequestQueue go directly to the Live API connection, bypassing the event pipeline your application receives via run_live():
Streaming Tool → input_stream.send_content() → Live API (model sees it)
↓
(NOT through your event stream)
The model receives and can respond to these intermediate updates, but they won't appear as events in your async for event in runner.run_live() loop. The final tool return value should still generate a function response event.
If you need updates visible to the UI (not just the model), you'll need to handle that separately—e.g., via a side-channel WebSocket message or by having the model echo the status in its response.
Hi @seanzhou1023, here's a summary of the outstanding work on this issue:
Core Feature Request
The main ask is to support streaming intermediate tool results for non-Live/SSE mode (run_async() with models like gemini-1.5-pro, gemini-1.5-flash).
Current State
Live API (BIDI mode): Already supported via Streaming Tools pattern - tools can accept input_stream: LiveRequestQueue parameter to send intermediate updates (see Part 3: Long-Running and Streaming Tools)
SSE mode: No built-in mechanism for streaming intermediate tool results yet
Long-running tools (2+ minutes) that perform multiple steps need to provide progress updates to the UI (e.g., "Creating image...", "Scanning for policy violations...", "Adding watermark...") rather than leaving users waiting without feedback.
Have a requirement to support streaming intermediate tool results for non-Live models in my current application. Any word on whether the proposed PRs are viable approaches to enable this functionality?
Let us know if there are any updates on SSE mode support or if we can help with testing!
Why don't streaming results generate function response events?
This is by design. Streaming tool updates sent via LiveRequestQueue go directly to the Live API connection, bypassing the event pipeline your application receives via run_live()
what is the reasoning behind this design?
Same as the use case to add streaming tools for non/live run_async:
Long-running tools (2+ minutes) that perform multiple steps need to provide progress updates to the UI (e.g., "Creating image...", "Scanning for policy violations...", "Adding watermark...") rather than leaving users waiting without feedback.
Also in a Live Context, I could want to update the UI when intermediate results are returned.
And having events in the event stream also helps with debug, now it's difficult to see when a streaming tool returned something.
Suppose you have 3 streaming tools and the model calls all 3, then you do not have any feedback on who returned a result when.
You need to add a parallel event stream only for this, and manage it yourself.
Since everything is in the event stream, I think this also should be in the event stream.
Unless there is a strong reason not to.
Looking at the source code (functions.py:670-679), streaming tool results are sent directly to the model via live_request_queue.send_content(), which goes straight to llm_connection.send_content() in base_llm_flow.py:280-281. Events are only generated from _receive_from_model() - not from content sent TO the model.
The design prioritizes low latency for real-time audio/video scenarios.
Your points are valid
You're right that this creates gaps for observability and debugging. However, in practice, developers often need a separate UI update channel anyway - for example, when displaying hundreds of database search results, large images, or real-time progress that shouldn't go through the LLM. Sending all of this through the event stream would be inefficient and noisy.
A common pattern is to maintain a side-channel (e.g., separate WebSocket messages or callbacks) for direct UI updates, keeping the ADK event stream focused on agent-model communication.
Activity
AdamPaternostro commentedon Jul 17, 2025
I have the following use case:
Example output from tool:
The user would see the outputted status and the final LLM output.
Pseudo code for tool. "raise_status_message" would send back statuses, to the UI via the REST SSE, during the tool execution.
usamaasfar commentedon Jul 30, 2025
Any updates on the issue?
ChristopherEeles commentedon Oct 1, 2025
Also very interested in this functionality. Have a requirement to support streaming intermediate tool results for non-Live models in my current application. Any word on whether the proposed PRs are viable approaches to enable this functionality?
Lin-Nikaido commentedon Oct 1, 2025
@ChristopherEeles
In our application my PR function is running well. Is there any things I can help you?
AdamPaternostro commentedon Oct 2, 2025
I ended up writing this where a tool can yield events to the UI: https://github.com/GoogleCloudPlatform/data-analytics-golden-demo/blob/main/data-analytics-demos/data-analytics-agent/agent-code/data_analytics_agent/dataplex/data_profile_workflow/data_profile_tool_workflow.py
simone-viozzi commentedon Dec 2, 2025
Hi, I hope this is the right place.
I'm using streaming tools to make an Agent Tool asynchronous, since otherwise the voice agent will freeze while the agent tool is doing its thing.
This is how I implemented it:
Is there a better way to do this?
With some prompting on the voice agent side this sort of works.
I also have a question: I do not see the reply in the events, but the model sees it.
How are those streaming results treated? Why don’t they generate a function response event?
kazunori279 commentedon Dec 5, 2025
Hi @simone-viozzi, great questions!
1. Is there a better way to do this?
Yes, ADK has a built-in Streaming Tools pattern for Live API. Instead of wrapping with
AsyncGenerator, you can define a tool that accepts aninput_stream: LiveRequestQueueparameter:How it works:
run_live()initialization and detects theLiveRequestQueuetype annotation (runners.py:828-865)LiveRequestQueuefor each streaming tool, stored ininvocation_context.active_streaming_tools[tool_name]input_stream(function_tool.py:238-253)input_stream.send_content()go directly to the Live APISee Part 3: Long-Running and Streaming Tools in the ADK Bidi-streaming Developer Guide for the complete queue lifecycle and implementation details.
2. Why don't streaming results generate function response events?
This is by design. Streaming tool updates sent via
LiveRequestQueuego directly to the Live API connection, bypassing the event pipeline your application receives viarun_live():The model receives and can respond to these intermediate updates, but they won't appear as events in your
async for event in runner.run_live()loop. The final tool return value should still generate a function response event.If you need updates visible to the UI (not just the model), you'll need to handle that separately—e.g., via a side-channel WebSocket message or by having the model echo the status in its response.
kazunori279 commentedon Dec 5, 2025
Hi @seanzhou1023, here's a summary of the outstanding work on this issue:
Core Feature Request
The main ask is to support streaming intermediate tool results for non-Live/SSE mode (
run_async()with models likegemini-1.5-pro,gemini-1.5-flash).Current State
input_stream: LiveRequestQueueparameter to send intermediate updates (see Part 3: Long-Running and Streaming Tools)Use Case (from @AdamPaternostro)
Long-running tools (2+ minutes) that perform multiple steps need to provide progress updates to the UI (e.g., "Creating image...", "Scanning for policy violations...", "Adding watermark...") rather than leaving users waiting without feedback.
Open Question (from @ChristopherEeles)
Let us know if there are any updates on SSE mode support or if we can help with testing!
simone-viozzi commentedon Dec 5, 2025
Hi @kazunori279 thank you,
what is the reasoning behind this design?
Same as the use case to add streaming tools for non/live
run_async:Also in a Live Context, I could want to update the UI when intermediate results are returned.
And having events in the event stream also helps with debug, now it's difficult to see when a streaming tool returned something.
Suppose you have 3 streaming tools and the model calls all 3, then you do not have any feedback on who returned a result when.
You need to add a parallel event stream only for this, and manage it yourself.
Since everything is in the event stream, I think this also should be in the event stream.
Unless there is a strong reason not to.
kazunori279 commentedon Dec 5, 2025
Hi @simone-viozzi, great question!
Why the current design?
Looking at the source code (
functions.py:670-679), streaming tool results are sent directly to the model vialive_request_queue.send_content(), which goes straight tollm_connection.send_content()inbase_llm_flow.py:280-281. Events are only generated from_receive_from_model()- not from content sent TO the model.The design prioritizes low latency for real-time audio/video scenarios.
Your points are valid
You're right that this creates gaps for observability and debugging. However, in practice, developers often need a separate UI update channel anyway - for example, when displaying hundreds of database search results, large images, or real-time progress that shouldn't go through the LLM. Sending all of this through the event stream would be inefficient and noisy.
A common pattern is to maintain a side-channel (e.g., separate WebSocket messages or callbacks) for direct UI updates, keeping the ADK event stream focused on agent-model communication.
feat(tools): run non-live generator FunctionTools via FunctionNode