You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ManifestFilterManager rebuilds metrics evaluators for every data file #1117
ManifestFilterManager::ShouldDelete builds a new StrictMetricsEvaluator and InclusiveMetricsEvaluator for every data file it checks manifest_filter_manager.cc#L366-L375). Make() rewrites and binds the expression each time, even though the residual only changes per partition. The existing TODO already notes this:
TODO(Guotao): Cache strict/inclusive metrics evaluators per partition residual.
Measured with the benchmarks from #1116, building and evaluating per file costs about 6x as much as evaluating with a prebuilt evaluator (16 predicates: ~9 µs vs ~1.5 µs per file).
Proposal: cache the evaluators per partition residual, and add a benchmark for the build-per-file case to measure the change.
Activity