Skip to content

fix(runtime): keep text and reasoning streamed in the terminal chunk - #4765

Draft
nicho810 wants to merge 1 commit into
docker:mainfrom
nicho810:fix/stream-text-in-terminal-chunk
Draft

nicho810 wants to merge 1 commit into
docker:mainfrom
nicho810:fix/stream-text-in-terminal-chunk

Conversation

@nicho810

Copy link
Copy Markdown

Summary

handleStream returns as soon as a chunk carries a terminal finish reason (stop, length, refusal). Tool call deltas and media in that chunk are already collected first (see TestHandleStream_ToolCallAndStopInSameChunk and TestHandleStream_MediaInTerminalChunkIsAccumulated), but delta.content and delta.reasoning_content are handled after the early return, so text in the terminal chunk is dropped.

Some OpenAI-compatible endpoints do send the last content delta together with finish_reason and usage. DeepSeek V4.1 Flash does it on part of its replies: a stream ends with "mar", then {"content":"igold","finish_reason":"stop","usage":{...}}, and docker-agent prints mar. (The OpenAI adapter only holds back stop / length when the chunk has no usage, so a chunk that carries usage reaches this path.)

This moves the reasoning, thinking-signature and content handling above the terminal finish-reason check, so they are collected before the early return. Behaviour for chunks without a finish reason is unchanged, and the tool-call short-circuit above is untouched.

I work on cheapaimodel, an OpenAI-compatible gateway, where this showed up while testing the provider alias in #4764.

Testing

  • New TestHandleStream_ContentInTerminalChunkIsAccumulated (stop and length): fails on main ("mar" instead of "marigold", empty reasoning) and passes with this change.
  • go vet ./pkg/runtime/ clean; go test ./pkg/runtime/... passes except TestDoCompactAfterHookFires, which fails the same way on main in my environment (golang:1.27 container).
  • Live: a filesystem-toolset agent on deepseek-v4-1-flash answered mar / marig on main (2 of 2 runs) and the full word in 6 of 6 runs with this change.

Signed-off-by: Nicho Deng <nicho810d@gmail.com>
@nicho810
nicho810 force-pushed the fix/stream-text-in-terminal-chunk branch from 8db4418 to df70107 Compare October 29, 2026 13:49
@aheritier aheritier added area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/fix PR fixes a bug (maps to fix:). Use on PRs only. status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key labels Oct 29, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push.

Commits that are not verified: df70107

See GitHub's guide on signing commits for setup instructions. I've added status/needs-signed-commits; it will be removed automatically once every commit in this PR carries a valid GitHub-verified signature.

@aheritier

Copy link
Copy Markdown
Collaborator

👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push.\n\nCommits that are not verified: df70107\n\nSee GitHub’s guide on signing commits for setup instructions. I’ve added status/needs-signed-commits; it will be removed automatically once every commit in this PR carries a valid GitHub-verified signature.

@aheritier
aheritier requested a balanced review from Copilot October 29, 2026 22:31
@aheritier
aheritier marked this pull request as draft October 29, 2026 22:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the change to preserve text and reasoning in terminal stream chunks.

No open issues 🎉

What changed in this PR

Fixes runtime stream handling so text and reasoning sent alongside a terminal finish reason are preserved rather than dropped, addressing truncated replies from some OpenAI-compatible providers.

Changes:

  • Collects reasoning, thinking signatures, and text before the terminal-chunk early return.
  • Adds regression tests for content and reasoning arriving with stop or length and usage data.
File Description
pkg/​runtime/​streaming.go Moves text and reasoning handling before terminal finish-reason processing.
pkg/​runtime/​streaming_test.go Tests that terminal chunks preserve complete content and reasoning.

Review effort: Balanced

@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Oct 29, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added ; it'll be picked back up automatically once the conflicts are cleared.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/fix PR fixes a bug (maps to fix:). Use on PRs only. status/needs-rebase PR has merge conflicts or is out of date with main status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants