Skip to content

fix(cloud-proxy): accept array message content in OpenAI translate mode - #13152

Open
walcz-de wants to merge 1 commit into
mudler:masterfrom
walcz-de:fix/cloud-proxy-mistral-content
Open

walcz-de wants to merge 1 commit into
mudler:masterfrom
walcz-de:fix/cloud-proxy-mistral-content

Conversation

@walcz-de

@walcz-de walcz-de commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Mistral's chat completions API types message.content and delta.content as
string | null | ContentChunk[] (AssistantMessage / DeltaMessage in
https://docs.mistral.ai/openapi.yaml). Reasoning models answer with a
ThinkChunk plus a TextChunk:

"content": [
  {"type": "thinking", "thinking": [{"type": "text", "text": "..."}]},
  {"type": "text", "text": "Hallo!"}
]

and, per https://docs.mistral.ai/capabilities/reasoning/, the stream sends
chunk arrays during thinking and plain strings once the answer starts.

Translate mode decoded content into a string, so every such response failed:

cloud-proxy: decode response: json: cannot unmarshal array into Go struct
field openAIMessage.choices.message.content of type string

Reproduction: a model with backend: cloud-proxy, proxy.mode: translate,
provider: openai, upstream_url: https://api.mistral.ai/v1/chat/completions
and a Mistral reasoning model as upstream_model; any chat request fails
with the error above.

Change:

  • Decode content as a string or an array of parts. text parts become the
    answer, thinking parts go to ChatDelta.ReasoningContent, other part
    types are ignored. Requests still send content as a string.
  • Streaming deltas get the same handling, so reasoning streams as
    reasoning_content.
  • When a non-streaming reply carries a ChatDelta (tool calls or reasoning),
    the delta now also holds the answer text, because consumers take the answer
    from the delta when one is present. Plain string content is unchanged.

Tests: new Ginkgo specs in provider_openai_content_test.go with a fake
upstream returning Mistral-shaped bodies (non-streaming, tool calls,
streaming across both shapes) plus counter-checks for plain string content
and deltas. go test ./... in backend/go/cloud-proxy passes.

Docs: docs/content/operations/cloud-proxy.md (translate mode section).

Breaking changes: none for plain string content. Behaviour change: a non-streaming reply that carries tool calls or reasoning now keeps its answer text in the ChatDelta (previously dropped).

AI assistance: prepared with Claude Code (see the Assisted-by trailer); reviewed and signed off by me.

Mistral's chat completions API types message.content and delta.content
as string | null | ContentChunk[]. Its reasoning models answer with a
ThinkChunk ({"type":"thinking","thinking":[TextChunk...]}) and a
TextChunk, and the stream switches from chunk arrays to plain strings
once the answer starts. Translate mode decoded content into a string,
so every such response failed with "cannot unmarshal array into Go
struct field openAIMessage.choices.message.content of type string".

Decode content as either shape: text parts become the answer, thinking
parts go to ChatDelta.ReasoningContent, other part types are ignored.
When a non-streaming reply carries a ChatDelta (tool calls or
reasoning), the delta now holds the text as well, because consumers
take the answer from the delta when one is present. Plain string
content is handled as before.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant