Repository navigation
Conversation
Signed-off-by: Nicho Deng <nicho810d@gmail.com>
8db4418 to
df70107
Compare
|
👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push. Commits that are not verified: See GitHub's guide on signing commits for setup instructions. I've added |
|
👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push.\n\nCommits that are not verified: |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the change to preserve text and reasoning in terminal stream chunks.
No open issues 🎉
What changed in this PR
Fixes runtime stream handling so text and reasoning sent alongside a terminal finish reason are preserved rather than dropped, addressing truncated replies from some OpenAI-compatible providers.
Changes:
- Collects reasoning, thinking signatures, and text before the terminal-chunk early return.
- Adds regression tests for content and reasoning arriving with
stoporlengthand usage data.
| File | Description |
|---|---|
pkg/runtime/streaming.go |
Moves text and reasoning handling before terminal finish-reason processing. |
pkg/runtime/streaming_test.go |
Tests that terminal chunks preserve complete content and reasoning. |
Review effort: Balanced
|
👋 This PR has merge conflicts with the base branch. Please rebase or merge the latest base branch and resolve them. I've moved it to draft and added ; it'll be picked back up automatically once the conflicts are cleared. |
Summary
handleStreamreturns as soon as a chunk carries a terminal finish reason (stop,length,refusal). Tool call deltas and media in that chunk are already collected first (seeTestHandleStream_ToolCallAndStopInSameChunkandTestHandleStream_MediaInTerminalChunkIsAccumulated), butdelta.contentanddelta.reasoning_contentare handled after the early return, so text in the terminal chunk is dropped.Some OpenAI-compatible endpoints do send the last content delta together with
finish_reasonandusage. DeepSeek V4.1 Flash does it on part of its replies: a stream ends with"mar", then{"content":"igold","finish_reason":"stop","usage":{...}}, and docker-agent printsmar. (The OpenAI adapter only holds backstop/lengthwhen the chunk has no usage, so a chunk that carries usage reaches this path.)This moves the reasoning, thinking-signature and content handling above the terminal finish-reason check, so they are collected before the early return. Behaviour for chunks without a finish reason is unchanged, and the tool-call short-circuit above is untouched.
I work on cheapaimodel, an OpenAI-compatible gateway, where this showed up while testing the provider alias in #4764.
Testing
TestHandleStream_ContentInTerminalChunkIsAccumulated(stopandlength): fails onmain("mar"instead of"marigold", empty reasoning) and passes with this change.go vet ./pkg/runtime/clean;go test ./pkg/runtime/...passes exceptTestDoCompactAfterHookFires, which fails the same way onmainin my environment (golang:1.27 container).filesystem-toolset agent ondeepseek-v4-1-flashansweredmar/marigonmain(2 of 2 runs) and the full word in 6 of 6 runs with this change.