Skip to content

Master issue: [Streaming Tools] support streaming intermediate results for tools for non-streaming case #2014

Closed
1 of 1 issue completed

Description

@hangfei

** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Activity

  1. added
    live[Component] This issue is related to live, voice and video chat
    on Jul 17, 2025
  2. self-assigned this
    on Jul 17, 2025
  3. AdamPaternostro commented on Jul 17, 2025

    @AdamPaternostro

    I have the following use case:

    1. I have a single tool that runs for 2 minutes which does a series of steps (each step can take 2 to 30 seconds)
    2. The user interface waits on the tool for 2 minutes and the user does not know if the tool is frozen
    3. I would like to provide progress updates during the long running execution

    Example output from tool:

    • Creating image prompt
    • Creating the image
    • Scanning the image for policy violations... (maybe this takes 30 seconds)
    • Done scanning image: No policy violations found
    • Adding watermark to image
    • Checking image into source control
    • Updating readme
    • { returns }

    The user would see the outputted status and the final LLM output.

    Pseudo code for tool. "raise_status_message" would send back statuses, to the UI via the REST SSE, during the tool execution.

    def image_curation_process(prompt: str)-> dict:
        """
        Creates an image, scans it and checks it into source control.
    
        Returns:
            dict: A dictionary containing the status and a boolean result.
            {
                "status": "success" or "failed",
                "tool_name": "image_curation_process",
                "generated_image_url" : "generated_image_url"
                }
            }
        """ 
        generate_enhanced_prompt_result = generate_enhanced_prompt(prompt)
        raise_status_message("Enhanced prompt generated")
        
        generate_image_result = generate_image(generate_enhanced_prompt_result["enhanced_prompt"]"])
        raise_status_message("Image generated")
        
        raise_status_message("Starting image scan...")
        scan_image_result = scan_image(generate_image_result["generated_image_url"])
        raise_status_message("Image scanned")
        
        if scan_image_result == False:
            return { "status": "failed", "tool_name" : "image_curation_process", "messages": ["Image voliated policy 123."] }
    
        add_watermark_result = add_watermark(generate_image_result["generated_image_url"])
        raise_status_message("Watermark added")
        
        check_into_source_control_result = check_into_source_control(add_watermark_result["generated_image_url"])
        raise_status_message("Image checked into source control")
        
        update_readme_result = update_readme(check_into_source_control_result["generated_image_url"])
        raise_status_message("Readme updated")  
    
        return { "status": "success", "tool_name" : "create_image_tool", "generated_image_url" : generate_image_result["generated_image_url"] }
    
    
    image_agent = LlmAgent(name="ImageAgent", 
                    description="Provide the abilty to generate image, verify corporate polcies on the image and check into source control.",
                    tools=[ image_curation_process],
                    model="gemini-2.5-flash")
    
  4. usamaasfar commented on Jul 30, 2025

    @usamaasfar

    Any updates on the issue?

  5. added this to the 08/11 FixIt Week milestone on Aug 8, 2025
  6. ChristopherEeles commented on Oct 1, 2025

    @ChristopherEeles

    Also very interested in this functionality. Have a requirement to support streaming intermediate tool results for non-Live models in my current application. Any word on whether the proposed PRs are viable approaches to enable this functionality?

  7. Lin-Nikaido commented on Oct 1, 2025

    @Lin-Nikaido
    Contributor

    @ChristopherEeles
    In our application my PR function is running well. Is there any things I can help you?

  8. removed their assignment
    on Nov 29, 2025
  9. simone-viozzi commented on Dec 2, 2025

    @simone-viozzi

    Hi, I hope this is the right place.

    I'm using streaming tools to make an Agent Tool asynchronous, since otherwise the voice agent will freeze while the agent tool is doing its thing.

    This is how I implemented it:

    def make_agent_tool_async(agent_tool: AgentTool):
        """Create a streaming tool that wraps an LlmAgent via AgentTool.
    
        The returned function:
        - yields exactly once with {"status": "started"}
        - then waits for the agent to finish
        - then yields {"status": "done", "reply": <agent reply>}
        """
    
        async def execute_async_agent_tool(
            request: str,
            tool_context: ToolContext,
        ) -> AsyncGenerator[dict[str, Any], None]:
            pending = tool_context.state.get("pending_jobs", {})
            if agent_tool.agent.name in pending:
                yield {
                    "status": "already running - wait for completion - do not call again until done",
                }
                log.debug(
                    f"Agent tool {agent_tool.agent.name} already running"
                )
                return
    
            pending[agent_tool.agent.name] = "started"
            tool_context.state["pending_jobs"] = pending
    
            yield {
                "status": "started",
            }
    
            log.debug(
                f"Started long-running agent tool for agent_name={agent_tool.agent.name}"
            )
    
            reply = await agent_tool.run_async(
                args={"request": request},
                tool_context=tool_context,
            )
    
            yield {
                "status": "done",
                "reply": reply,
            }
    
            tool_context.state["pending_jobs"].pop(agent_tool.agent.name, None)
            log.debug(f"Completed long-running agent tool for agent_name={agent_tool.agent.name}")
    
        execute_async_agent_tool.__name__ = agent_tool.name
    
        # TODO maybe use _get_declaration to copy over function signature?
        base_desc = agent_tool.agent.description or ""
        execute_async_agent_tool.__doc__ = base_desc + (
            "\n\nNOTE: This is a long-running streaming version of this tool. "
            "It first returns status='started', then later returns status='done' with the reply. "
            "If called while already running, returns status='already running'. "
            "Wait for status='done' before calling this tool again."
        )
    
        return execute_async_agent_tool
    
    voice_agent= LlmAgent(
        name="voice_agent",
        model=voice_model,
        instruction=....,
        tools=[
            make_agent_tool_async(
                AgentTool(
                    text_agent,
                    skip_summarization=True,
                )
            ),
        ],
    )

    Is there a better way to do this?

    With some prompting on the voice agent side this sort of works.

    I also have a question: I do not see the reply in the events, but the model sees it.
    How are those streaming results treated? Why don’t they generate a function response event?

  10. kazunori279 commented on Dec 5, 2025

    @kazunori279
    Collaborator

    Hi @simone-viozzi, great questions!

    1. Is there a better way to do this?

    Yes, ADK has a built-in Streaming Tools pattern for Live API. Instead of wrapping with AsyncGenerator, you can define a tool that accepts an input_stream: LiveRequestQueue parameter:

    from google.adk.agents.live_request_queue import LiveRequestQueue
    from google.genai import types
    
    async def execute_agent_tool(
        request: str,
        tool_context: ToolContext,
        input_stream: LiveRequestQueue,  # ADK auto-injects this
    ) -> dict:
        """Long-running agent tool that streams progress to the model."""
        
        # Send status update to model
        input_stream.send_content(
            types.Content(parts=[types.Part(text="Starting agent execution...")])
        )
        
        # Do the work
        reply = await some_long_running_work(request, tool_context)
        
        return {"status": "done", "reply": reply}

    How it works:

    • ADK inspects tools at run_live() initialization and detects the LiveRequestQueue type annotation (runners.py:828-865)
    • ADK creates a dedicated LiveRequestQueue for each streaming tool, stored in invocation_context.active_streaming_tools[tool_name]
    • When the model calls the tool, ADK automatically injects the queue as input_stream (function_tool.py:238-253)
    • Updates sent via input_stream.send_content() go directly to the Live API

    See Part 3: Long-Running and Streaming Tools in the ADK Bidi-streaming Developer Guide for the complete queue lifecycle and implementation details.

    2. Why don't streaming results generate function response events?

    This is by design. Streaming tool updates sent via LiveRequestQueue go directly to the Live API connection, bypassing the event pipeline your application receives via run_live():

    Streaming Tool → input_stream.send_content() → Live API (model sees it)
                                                  ↓
                                        (NOT through your event stream)
    

    The model receives and can respond to these intermediate updates, but they won't appear as events in your async for event in runner.run_live() loop. The final tool return value should still generate a function response event.

    If you need updates visible to the UI (not just the model), you'll need to handle that separately—e.g., via a side-channel WebSocket message or by having the model echo the status in its response.

  11. kazunori279 commented on Dec 5, 2025

    @kazunori279
    Collaborator

    Hi @seanzhou1023, here's a summary of the outstanding work on this issue:

    Core Feature Request

    The main ask is to support streaming intermediate tool results for non-Live/SSE mode (run_async() with models like gemini-1.5-pro, gemini-1.5-flash).

    Current State

    • Live API (BIDI mode): Already supported via Streaming Tools pattern - tools can accept input_stream: LiveRequestQueue parameter to send intermediate updates (see Part 3: Long-Running and Streaming Tools)
    • SSE mode: No built-in mechanism for streaming intermediate tool results yet

    Use Case (from @AdamPaternostro)

    Long-running tools (2+ minutes) that perform multiple steps need to provide progress updates to the UI (e.g., "Creating image...", "Scanning for policy violations...", "Adding watermark...") rather than leaving users waiting without feedback.

    Open Question (from @ChristopherEeles)

    Have a requirement to support streaming intermediate tool results for non-Live models in my current application. Any word on whether the proposed PRs are viable approaches to enable this functionality?

    Let us know if there are any updates on SSE mode support or if we can help with testing!

  12. simone-viozzi commented on Dec 5, 2025

    @simone-viozzi

    Hi @kazunori279 thank you,

    1. Why don't streaming results generate function response events?

    This is by design. Streaming tool updates sent via LiveRequestQueue go directly to the Live API connection, bypassing the event pipeline your application receives via run_live()

    what is the reasoning behind this design?

    Same as the use case to add streaming tools for non/live run_async:

    Long-running tools (2+ minutes) that perform multiple steps need to provide progress updates to the UI (e.g., "Creating image...", "Scanning for policy violations...", "Adding watermark...") rather than leaving users waiting without feedback.

    Also in a Live Context, I could want to update the UI when intermediate results are returned.

    And having events in the event stream also helps with debug, now it's difficult to see when a streaming tool returned something.
    Suppose you have 3 streaming tools and the model calls all 3, then you do not have any feedback on who returned a result when.
    You need to add a parallel event stream only for this, and manage it yourself.

    Since everything is in the event stream, I think this also should be in the event stream.
    Unless there is a strong reason not to.

  13. kazunori279 commented on Dec 5, 2025

    @kazunori279
    Collaborator

    Hi @simone-viozzi, great question!

    Why the current design?

    Looking at the source code (functions.py:670-679), streaming tool results are sent directly to the model via live_request_queue.send_content(), which goes straight to llm_connection.send_content() in base_llm_flow.py:280-281. Events are only generated from _receive_from_model() - not from content sent TO the model.

    The design prioritizes low latency for real-time audio/video scenarios.

    Your points are valid

    You're right that this creates gaps for observability and debugging. However, in practice, developers often need a separate UI update channel anyway - for example, when displaying hundreds of database search results, large images, or real-time progress that shouldn't go through the LLM. Sending all of this through the event stream would be inefficient and noisy.

    A common pattern is to maintain a side-channel (e.g., separate WebSocket messages or callbacks) for direct UI updates, keeping the ADK event stream focused on agent-model communication.

  14. added
    tools[Component] This issue is related to tools
    needs review[Status] The PR/issue is awaiting review from the maintainer
    and removed
    live[Component] This issue is related to live, voice and video chat
    on Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

needs review[Status] The PR/issue is awaiting review from the maintainertools[Component] This issue is related to tools

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions