Repository navigation
Resolve feature flag attributes once per evaluation and cache split feature metadata - #13489
Draft
leoromanovsky wants to merge 6 commits into
Draft
leoromanovsky wants to merge 6 commits into
leoromanovsky wants to merge 6 commits into
Conversation
… at load A flag with many allocations re-read the same context attributes in every allocation: a context lookup and a value conversion each time, and the conversion boxes numbers. Each evaluation now keeps the attributes it has resolved, so a flag with 1,000 allocations reads its attribute once. A matched split's features were turned into metadata entries on every evaluation, concatenating a prefixed key for each feature and destination. The split now resolves its entries once, when it is built, and an evaluation only copies them. Unknown destinations are now reported when the configuration loads, still once per name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The split caches what the evaluator derives from its features instead of building it in its constructor, so the configuration model does not depend on the evaluator and an unknown destination is still reported when a split is served, as before. The allocation and rule scans keep their for-each loops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most flags read one attribute, so the evaluation scope keeps it in fields and allocates its arrays only for a second attribute. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR is stacked on #13485, which adds the scale-suite benchmarks and the baseline from
master. Review only the commits after it. It is a sibling of #13484, #13486, and #13487.Evaluating a flag with many allocations costs less time and memory, with the same results:
__dd_feature.*,__dd_exposure_feature.*and__dd_evaluation_feature.*entries) is built once, the first time the split is served, and cached on the split. Before, each evaluation that served the split concatenated a new key for each feature and destination.The served allocation, variant, reason, error codes, and metadata do not change.
Motivation
We need to win a performance-sensitive customer, and the scale benchmarks of #13485 are the basis for this work. In those benchmarks, a flag with 1,000 allocations (
scale/variations) costs 53–61 µs and 28 KB per evaluation, and its worst case costs 113 µs and 83 KB. A profile of the worst cases shows two costs:scale/rules-sweep(up to 2,500 conditions over a few attributes), most of the time is inStructure.convertValue, because each condition converts the same numeric attribute again.scale/variations, the matched split carries 200 features for 3 destinations. Each evaluation builds 600 metadata keys by string concatenation, which is about 48 KB of the 83 KB.flowchart LR subgraph Before A1[allocation 1..N] --> C1[each condition: context.getValue + convertValue] S1[matched split] --> F1[each evaluation: prefix + key for every feature and destination] end subgraph After A2[allocation 1..N] --> E2[EvaluationScope: resolve once, then reuse] S2[matched split] --> F2[SplitFeatures, built on first serve and cached on the split] endAdditional Notes
Scale suite results
Same machine and settings as the #13485 baseline: AWS workspace, Intel Xeon Platinum 8175M, 16 vCPUs, JDK 21.0.10, JMH AverageTime, 1 fork, 3 warm-up and 5 measurement iterations of 2 s,
-prof gc. Themaster(#13485) and branch jars ran back to back. Other benchmarks and builds shared the machine, with a load average of 10–12, so differences of about 10% or less are within noise. Rows marked "all" are the mean over the cases of the file.master(#13485)scale/variations, allscale/variations, matches allocation 999 (200 features)scale/variations, matches the fallback after 1,000 allocationsscale/rules-sweep, allscale/rules-sweep,50x10x5match (2,500 conditions)scale/rulesscale/time-windowsscale/wide-contextscale/flagsAn earlier back-to-back pair on the same machine gave the same picture. For example, the
rules-sweep50x10x5match went from 102 µs to 78 µs there.The remaining 28 KB on the 200-feature case is the
HashMapbehindImmutableMetadata: about 600 entries and its resizes. The OpenFeature builder takes entries one at a time and cannot be presized. The remaining time of the 1,000-allocation scan is mostly theONE_OFlist scan, which #13486 replaces with an index.ScaleSuiteLoadBenchmarkis not affected: the configuration parse does no new work.Decisions
EvaluationScopeholds the context and the attributes resolved so far. It is created only when the first allocation with rules is reached, andevaluateRulesandevaluateConditiontake it in place of the context. The shard-hashing and SemVer sibling PRs also keep state for one evaluation, and they can add their state to this object.HashMaponly past 8 attributes, so lookups do not grow quadratically on very wide flags. A flag with one condition pays one 40-byte object, which accounts for the +30–40 B above. A flag without rules pays nothing.idattribute still falls back to the targeting key only when the context has noidkey. An explicitidthat maps to null stays null. An attribute that is read once and then changes in the context during the same evaluation is not read again.ufc) does not depend on the evaluator.Splitcaches what the evaluator derives from its features. The entries have the same keys, the same value types (numbers become doubles, as before), and the same rule that the first feature of each key wins. The warning for an unknown destination still fires when a split is served, once per destination name.ONE_OFlookups (Index ONE_OF and NOT_ONE_OF lists when the configuration loads #13486), regex (Evaluate feature flag regex conditions in linear time #13487), shard hashing, and SemVer parsing belong to sibling PRs, and this PR does not change those code paths.Testing
EvaluationScopeTest: each attribute is read from the context once, below and past 8 attributes, a missing attribute is cached as missing, and theidfallback is checked, including an explicit nullid.SplitFeaturesTest: the entries for each destination, first-key-wins deduplication, incomplete and unsupported features, numbers widened to doubles, the evaluation-features marker, one shared empty preparation, the warning for unknown destinations, and that the split computes its entries once.CanonicalFixtureTeststill checks the exact HOOK, EXPOSURE, and EVALUATION feature maps,serialId,doLog, and the allocation key for every fixture case. The relevant files includetest-case-split-features,test-case-serial-id-metadata,test-case-experimentation-opt-out-do-log,test-case-start-and-end-date-flag,test-case-microsecond-date-flag, andtest-case-time-bounded-holdout-flag. TheFlagEvaluatorTestfeature and warning tests pass unchanged.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issue🤖 Generated with Claude Code