Skip to content

fix(openai): return unparseable raw responses untraced in the Responses wrapper - #4718

Open
kratos0718 wants to merge 1 commit into
traceloop:mainfrom
kratos0718:fix-responses-raw-unparseable
Open

kratos0718 wants to merge 1 commit into
traceloop:mainfrom
kratos0718:fix-responses-raw-unparseable

Conversation

@kratos0718

@kratos0718 kratos0718 commented Nov 19, 2026 •

Copy link
Copy Markdown

Fixes #4717

client.responses.with_raw_response.create() returns the raw response and leaves .parse() to the caller. The sync and async Responses wrappers call parse_response / async_parse_response themselves to read the ID, model and usage, but outside their error handling, so a body that can't be parsed raised JSONDecodeError from the instrumentation while the same call returned normally without it.

Both calls are now wrapped as the issue suggests: if parsing fails, the wrapper returns the raw response untraced, the same way it already does when building TracedData fails. No span is created and nothing is recorded as an error, since the API call itself succeeded.

Tests: test_responses_with_raw_response_unparseable_body_is_returned_untraced and its async counterpart use an httpx.MockTransport that returns {not json with status 200 (the reproduction from the issue). They check that the raw response comes back with its status and body and that no span was exported. Both fail without the change. The package's tests pass (uv run pytest tests: 301 passed, 1 skipped), and ruff check is clean.

  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change. (not applicable: the change only affects a failure path, which produces no span)
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly.

Summary by CodeRabbit

  • Bug Fixes
    • Synchronous and asynchronous raw Responses API calls now return the original response when its body contains invalid JSON, instead of raising an error during parsing.
    • In these cases, the response’s status and body text remain available, and no tracing spans are emitted.

…es wrapper

with_raw_response.create() leaves parsing to the caller, but the sync and
async Responses wrappers parsed the raw response outside their error
handling, so a body that can't be parsed raised from the instrumentation
while the uninstrumented call returned normally. Both wrappers now return
the raw response untraced when parsing fails.

Fixes traceloop#4717
@CLAassistant

CLAassistant commented Nov 19, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The synchronous and asynchronous OpenAI Responses API wrappers now return the raw response when parsing fails. Regression tests verify this behavior for invalid JSON and confirm that no spans are emitted.

Changes: Raw response handling

Layer / File(s) Summary
Parsing fallback and regression tests
packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/v1/responses_wrappers.py, packages/opentelemetry-instrumentation-openai/tests/traces/test_responses.py
The synchronous and asynchronous wrappers return the original response when parsing fails. Tests check that invalid JSON responses retain their status and body and emit no spans.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

  • traceloop/openllmetry#4713: Changes the sync and async Responses create/parse wrappers and records metrics for raised errors on the same code path.

Suggested reviewers: dvirski, dinmukhamedm

Merge Risk: Minimal · up to a8b99

Responses raw-response calls with malformed bodies now return normally instead of raising from instrumentation. No actionable merge risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title says the Responses wrapper now returns unparseable raw responses untraced, which matches the raw summary: the sync and async wrappers catch parsing exceptions and return the response without…
Linked Issues check Passed [#4717] requires that a failed parse of the raw response in the Responses wrapper return the response untraced, without raising from instrumentation. The diff at head wraps parse_response(response) …
Out of Scope Changes check Passed The whole-PR diff (merge base 864f859 to head a8b9906) changes only responses_wrappers.py (guarding the two parse calls, with comments) and tests/traces/test_responses.py (two regression tests).…
Full details: Title check

Explanation

The title says the Responses wrapper now returns unparseable raw responses untraced, which matches the raw summary: the sync and async wrappers catch parsing exceptions and return the response without a span. The title names the main behavior change, and the conventional-commit scope (openai) fits the package. The title is 79 characters, which is within a normal length for a commit subject.

Full details: Linked Issues check

Explanation

[#4717] requires that a failed parse of the raw response in the Responses wrapper return the response untraced, without raising from instrumentation. The diff at head wraps parse_response(response) in the sync wrapper and await async_parse_response(response) in the async wrapper in try/except Exception, and returns response on failure. This matches the issue's suggested fix and its expected behavior. The PR adds sync and async tests (test_responses_with_raw_response_unparseable_body_is_returned_untraced and its async counterpart) that use an httpx.MockTransport returning status 200 with body {not json. They check that the raw response keeps its status and body and that no span is exported. The issue requests no other coding changes. Residual uncertainty: the file read was truncated at 16000 bytes, so I could not confirm where the span is started relative to the parse call. The tests only check exported spans, so they would not detect an unended span. This does not change the verdict on the stated requirement.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff (merge base 864f859 to head a8b9906) changes only responses_wrappers.py (guarding the two parse calls, with comments) and tests/traces/test_responses.py (two regression tests). Every change maps directly to #4717. No unrelated refactoring or behavior change appears in the reviewed hunks. The summary reports no changes to exported declarations. The truncated file read leaves some unseen context, but the diff hunks shown are all in scope.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 38232ead-8829-4e02-8cba-f2dc74c25d0b
Commits

Reviewing files that changed from the base of the PR and between 864f859 and a8b9906.

Files selected for processing: 2

  • packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/v1/responses_wrappers.py
  • packages/opentelemetry-instrumentation-openai/tests/traces/test_responses.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: OpenAI Responses: with_raw_response.create() raises when the body can't be parsed, though the uninstrumented call returns

2 participants