Skip to content

fix(configcat)!: report evaluation errors instead of a silent default - #1964

Open
RaphaelFakhri wants to merge 2 commits into
open-feature:mainfrom
RaphaelFakhri:fix/configcat-report-evaluation-errors
Open

RaphaelFakhri wants to merge 2 commits into
open-feature:mainfrom
RaphaelFakhri:fix/configcat-report-evaluation-errors

Conversation

@RaphaelFakhri

@RaphaelFakhri RaphaelFakhri commented Nov 3, 2026 •

Copy link
Copy Markdown

This PR

  • Checks EvaluationDetails.getErrorCode() after each ConfigCat evaluation. When ConfigCat reports an error, the provider now returns the default value with reason=ERROR, the ConfigCat error message, and an error code.
  • Maps ConfigCat's EvaluationErrorCode to the OpenFeature error code (FLAG_NOT_FOUND, TYPE_MISMATCH, PARSE_ERROR, or GENERAL).
  • Adds tests for a missing flag (through the provider and the client) and for an existing flag.

Before this change, a missing or mistyped flag returned the default with no error code or reason, so error hooks did not run and the evaluation looked successful.

Related Issues

Fixes #1963

Notes

Bumps configcat-java-client from 9.4.4 to 10.1.0, which adds EvaluationDetails.getErrorCode(). The provider maps it as follows: SETTING_KEY_MISSING to FLAG_NOT_FOUND, SETTING_VALUE_TYPE_MISMATCH to TYPE_MISMATCH, INVALID_CONFIG_MODEL to PARSE_ERROR, and anything else to GENERAL.

Breaking change

configcat-java-client 10.0.0 removed the option to set a custom OkHttpClient (ConfigCatClient.Options.httpClient(...)). Code that sets one through ConfigCatProviderConfig.options(...) no longer compiles. HTTP timeouts and the HTTP proxy are now set through the new HttpOptions. The PR title carries ! so release-please treats this as a breaking change.

Follow-up Tasks

The client onError hook also fires for evaluation errors such as a missing flag. Each one emits PROVIDER_ERROR, so the provider state flips between READY and ERROR. This PR does not change that.

How to test

./mvnw -pl providers/configcat clean verify

missingFlagReportsFlagNotFound fails on main with expected: <FLAG_NOT_FOUND> but was: <null> and passes with this change.

BREAKING CHANGE: configcat-java-client 10.x removes the option to set a custom OkHttpClient in ConfigCatProviderConfig.options(...). Use HttpOptions for HTTP timeouts and the HTTP proxy.

Signed-off-by: Raphael Fakhri <153192858+RaphaelFakhri@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Walkthrough

The ConfigCat provider now reports OpenFeature error details when ConfigCat evaluation fails. Tests check missing-flag error metadata and verify that an existing flag has no error code.

Changes: ConfigCat Evaluation Error Reporting

Layer / File(s) Summary
Map evaluation errors and validate details
providers/configcat/src/main/java/dev/openfeature/contrib/providers/configcat/ConfigCatProvider.java, providers/configcat/src/test/java/dev/openfeature/contrib/providers/configcat/ConfigCatProviderTest.java
When ConfigCat reports an evaluation error, getEvaluation returns the evaluated value, Reason.ERROR, the error message, and an error code. It uses FLAG_NOT_FOUND when the key is absent from a nonempty key list; otherwise, it uses GENERAL. Tests check missing-flag details and successful evaluation details.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: toddbaert, aepfli

Merge Risk: Low · up to 754cb

This change makes ConfigCat evaluation errors visible as OpenFeature errors, and the common missing-flag case now reports FLAG_NOT_FOUND. Two follow-ups remain: with lazy-loading polling, a failed fetch can cause a second blocking request on each failed evaluation, and a missing key in an empty loaded configuration is reported as GENERAL. Direct provider callers can also receive a null value instead of a structure default. These are bounded and mostly confined to non-default or narrow setups.

Pre-merge checks: Passed 4

Check name Status Explanation
Linked Issues check Passed Issue #1963 requires ConfigCat evaluation failures to return the default value with reason=ERROR, an error code, and the ConfigCat error message. The provider now checks `EvaluationDetails.getError(…
Out of Scope Changes check Passed The changes are limited to ConfigCat evaluation error handling and focused tests in the ConfigCat provider module. The changes support issue #1963 and do not introduce unrelated behavior.
Title check Passed The title names the main change: ConfigCat evaluation errors are now reported with an error code instead of returning a silent default. The fix(configcat) scope and the ! breaking-change marker match …
Description check Passed The description is directly related to the changeset. It explains the EvaluationDetails.getErrorCode() check, the mapping from ConfigCat error codes to OpenFeature error codes, the added tests for mis…
Full details: Linked Issues check

Explanation

Issue #1963 requires ConfigCat evaluation failures to return the default value with reason=ERROR, an error code, and the ConfigCat error message. The provider now checks EvaluationDetails.getError(). It returns Reason.ERROR, the error message, FLAG_NOT_FOUND when the key is absent from a non-empty key list, and GENERAL otherwise. The tests cover a missing flag through the provider and client, plus a successful existing flag evaluation.

Full details: Title check

Explanation

The title names the main change: ConfigCat evaluation errors are now reported with an error code instead of returning a silent default. The fix(configcat) scope and the ! breaking-change marker match the changeset, which adds error mapping in ConfigCatProvider and the breaking HttpOptions/OkHttpClient change noted in the description. The title is 69 characters, slightly over a concise length, but it remains clear and specific.

Full details: Description check

Explanation

The description is directly related to the changeset. It explains the EvaluationDetails.getErrorCode() check, the mapping from ConfigCat error codes to OpenFeature error codes, the added tests for missing and existing flags, the configcat-java-client bump, and the breaking change. It also links the issue it fixes and gives a test command.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 82632b90-f8c5-41f0-8313-c32cb710d965
Commits

Reviewing files that changed from the base of the PR and between 4a3bbc9 and 754cbd2.

Files selected for processing: 2

  • providers/configcat/src/main/java/dev/openfeature/contrib/providers/configcat/ConfigCatProvider.java
  • providers/configcat/src/test/java/dev/openfeature/contrib/providers/configcat/ConfigCatProviderTest.java

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

@z4kn4fein

z4kn4fein commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Hi @RaphaelFakhri, thanks for this PR! In the v10.1.0 version of the ConfigCat Java SDK we extended the resulting EvaluationDetails object with an EvaluationErrorCode property. It can now be used to map to the error cases covered by this PR.

…r codes

Bump configcat-java-client to 10.1.0, which exposes the error code on
EvaluationDetails, and use it instead of probing getAllKeys().

Signed-off-by: RaphaelFakhri <153192858+RaphaelFakhri@users.noreply.github.com>
@RaphaelFakhri

Copy link
Copy Markdown
Author

@z4kn4fein thanks, switched to it in 8b24351. The provider now bumps configcat-java-client to 10.1.0 and maps getErrorCode() instead of calling getAllKeys():

SETTING_KEY_MISSING -> FLAG_NOT_FOUND, SETTING_VALUE_TYPE_MISMATCH -> TYPE_MISMATCH, INVALID_CONFIG_MODEL -> PARSE_ERROR, and CONFIG_JSON_NOT_AVAILABLE / UNEXPECTED_ERROR -> GENERAL.

That also takes care of the extra fetch on the error path and the empty override map case CodeRabbit flagged. The error result now returns the caller's default value too, so a direct getObjectEvaluation call no longer gets null back. Added a TYPE_MISMATCH test. ./mvnw -pl providers/configcat -am clean verify passes (16 tests).

One thing for the maintainers: going from 9.x to 10.x pulls in the 10.0.0 breaking change that removed ConfigCatOptions.httpClient(OkHttpClient) in favor of HttpOptions. Users who set a custom OkHttp client through ConfigCatProviderConfig.options would need to update. Let me know if you want that flagged as breaking in the commit, or the bump split into its own PR.

@z4kn4fein

Copy link
Copy Markdown
Contributor

@RaphaelFakhri great, thank you for the update! I think the options change should be flagged as breaking, and the bump can remain in this PR.

@RaphaelFakhri

Copy link
Copy Markdown
Author

@z4kn4fein Done. The PR title is now fix(configcat)!: ..., so release-please treats it as a breaking change. The description has a "Breaking change" section and a BREAKING CHANGE: footer: configcat-java-client 10.0.0 removed Options.httpClient(OkHttpClient), so code that sets a custom OkHttp client through ConfigCatProviderConfig.options(...) has to move to HttpOptions. The bump stays in this PR.

@RaphaelFakhri RaphaelFakhri changed the title fix(configcat): report evaluation errors instead of a silent default fix(configcat)!: report evaluation errors instead of a silent default Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ConfigCat] Evaluation errors are returned as a successful default with no error code

4 participants