[deadstrobe5]

Description

When a request has tools and the model answers in prose, the streaming endpoints (OpenAI chat, Anthropic messages, Open Responses) re-parse the whole accumulated reply on every token. ParseJSONIterative and ParseJSON find no JSON in prose and fall back to parseJSONLegacy, which creates a json.Decoder at every byte offset and lets it fail. That makes the parse cost of a reply grow with the square of its length: an 8 KB prose reply streamed in 4-byte tokens spends about 5.8 s of CPU in these parsers.

This change lets parseJSONLegacy skip, without a decoder, the offsets where the decoder fails before reading any value: the first non-space byte cannot start a JSON value, or a true/false/null is broken off by a wrong byte. It moves to the offset the decoder's SyntaxError would report, so the objects found and the errors returned stay the same. Objects, arrays, strings, numbers, complete literals and input cut off at the end still go through the decoder.

To check equivalence I compared the old and new parseJSONLegacy on every string up to 5 bytes over a 16-symbol alphabet (literal letters, braces, quotes, digits, spaces, a stray UTF-8 byte), 6,000 random mixes of prose, tool-call JSON, broken literals, tags and raw bytes plus every prefix of a fifth of them, and a short go test -fuzz run: about 1.28 million inputs, identical results and errors under Go 1.26.0 and Go 1.27 (both encoding/json implementations). The pkg/functions and core/http/endpoints/openai tests pass, and I added a case for prose with broken literals between objects.

case (Go 1.26.0, process CPU time)	before	after
one parse, prose 200 B	109 us	1.1 us
one parse, prose 8 KB	3.0 ms	39 us
one parse, Go code answer 8 KB	4.9 ms	0.27 ms
whole stream, prose 2 KB in 4-byte tokens (with CleanupLLMResult and ParseXMLIterative)	464 ms	55 ms
whole stream, prose 8 KB in 4-byte tokens (same)	5.8 s	0.25 s

Medians of alternating old/new rounds in one process (old code copied in), Ryzen 5 5600X. Short inputs do not regress: an input that is only a tool call takes the same time, and Hi goes from 2.2 us to 0.27 us.

Notes for Reviewers

The skip only applies where the decoder would return a SyntaxError before reading a value, and it returns the offset the decoder reports there, so later objects are found from the same positions as before. Quirks of the legacy loop stay as they are (for example nul{...} still swallows that object).

Signed commits

• Yes, I signed my commits.
• Documentation updated (docs/content/) for user-facing changes, or not applicable