Skip to content

Read array slots through generated accessors and drop duplicates - #11430

Open
robert3005 wants to merge 1 commit into
developfrom
claude/inspiring-ritchie-cwpkl4
Open

robert3005 wants to merge 1 commit into
developfrom
claude/inspiring-ritchie-cwpkl4

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #11418 and #11423. The #[array_slots] accessors now return references borrowed for the array, or for the ArrayView's lifetime 'a. So there's no longer a reason to index slots() by XSlots constants and unwrap by hand, or to keep hand-written accessors that duplicate the generated ones.

Changes

Direct slot reads replaced with generated accessors

  • Validity children read via slots()[XSlots::VALIDITY_CHILD].as_ref() now call validity_child(), across Bool, Primitive, Decimal, List, ListView, FixedSizeList, VarBin, VarBinView, Masked, ByteBool, OnPair, ParquetVariant, Pco and Zstd.
  • Required slots read via .clone().vortex_expect(..) in List, ListView, FixedSizeList and Sparse now use their accessors. So do BitPacked's block_offsets, Variant's shredded and Sparse's patch_chunk_offsets.
  • ChunkedArrayExt and InterleaveArrayExt read chunks and values through the generated SlotSlice accessors. The index arithmetic with CHUNKS_OFFSET and VALUES_OFFSET is gone.
  • Dict, DateTimeParts and Patched call slots_view() instead of building XSlotsView::from_slots(array.slots()).

Hand-written duplicates removed

  • PrimitiveArrayExt::validity_child and DecimalArrayExt::validity_child.
  • InterleaveArrayExt::array_indices and row_indices. InterleaveSlots is now pub, and arrays::interleave re-exports InterleaveArraySlotsExt, InterleaveSlots and InterleaveSlotsView, so the accessors stay public.
  • ChunkedArrayExt::chunk_offsets_array, replaced by the generated chunk_offsets(). arrays::chunked now re-exports ChunkedArraySlotsExt.

ChunkedArrayExt::chunks is kept. It returns an owned Vec<ArrayRef>, unlike the generated SlotSlice accessor of the same name, so ChunkedArrayExt calls the generated accessor through its trait path.

Left as they are: code that copies or rewrites every slot generically, and validation of raw slot slices before an array exists.

No behavior changes are intended. Existing tests cover these paths.

Checks run:

  • cargo clippy --workspace --all-targets -- -D warnings: clean. Excludes vortex-duckdb, vortex-sqllogictest, duckdb-bench and lance-bench, whose build scripts could not run in the sandbox. None of them call the removed methods.
  • cargo test -p vortex-array --lib: 5910 passed. Before the squash and rebase, cargo test also passed on the eight changed encoding crates.

API Changes

  • Removed PrimitiveArrayExt::validity_child, DecimalArrayExt::validity_child, InterleaveArrayExt::array_indices, InterleaveArrayExt::row_indices and ChunkedArrayExt::chunk_offsets_array. The generated accessors replace them, so callers need the encoding's …ArraySlotsExt trait in scope. Use chunk_offsets() in place of chunk_offsets_array().
  • Newly public: InterleaveSlots, InterleaveSlotsView and InterleaveArraySlotsExt from arrays::interleave, and ChunkedArraySlotsExt from arrays::chunked.

Generated by Claude Code

Now that the #[array_slots] accessors return references borrowed for the
array (or the ArrayView's lifetime 'a), there is no reason to index
slots() by XSlots constants and unwrap by hand, or to hand-write accessors
that duplicate the generated ones.

Replace direct slot indexing with the generated accessors:

- validity children read via slots()[XSlots::VALIDITY_CHILD].as_ref()
  across Bool, Primitive, Decimal, List, ListView, FixedSizeList, VarBin,
  VarBinView, Masked, ByteBool, OnPair, ParquetVariant, Pco and Zstd now
  call validity_child();
- required slots read via .clone().vortex_expect(..) (List, ListView,
  FixedSizeList, Sparse) and the BitPacked block offsets, Variant
  shredded child and Sparse patch chunk offsets now use their accessors;
- the Chunked and Interleave ext methods read chunks and values through
  the variadic SlotSlice accessors;
- Dict, DateTimeParts and Patched build their slot view with
  slots_view() instead of XSlotsView::from_slots(array.slots()).

Remove hand-written accessors identical to the generated ones:

- PrimitiveArrayExt::validity_child and DecimalArrayExt::validity_child;
- InterleaveArrayExt::array_indices and row_indices. InterleaveSlots is
  now pub, and InterleaveArraySlotsExt, InterleaveSlots and
  InterleaveSlotsView are re-exported from arrays::interleave, so the
  accessors stay public;
- ChunkedArrayExt::chunk_offsets_array, replaced by the generated
  chunk_offsets(). ChunkedArraySlotsExt is now re-exported from
  arrays::chunked.

ChunkedArrayExt::chunks is kept: it returns an owned Vec<ArrayRef>, unlike
the generated SlotSlice accessor of the same name, so ChunkedArrayExt calls
the generated accessor through its trait path. Callers of the removed
methods now need the encoding's ArraySlotsExt trait in scope.

Generic whole-slot rewrites that clone or map every slot, and validation
of raw slot slices before an array exists, are left as they are.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SWbErpV8gppbsHrfq7fJD
@robert3005 robert3005 added the changelog/break A breaking API change label Nov 12, 2026 — with Claude
@codspeed

codspeed Bot commented Nov 12, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 14.26%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 5 improved benchmarks
✅ 1916 untouched benchmarks
⏩ 50 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ CPU Simulation chunked_bool_canonical_into[(1000, 10)] 37.7 µs 29.7 µs +26.87%
⚡ Walltime decode_chunked_neon[i64, 524288] 15.4 µs 13.5 µs +13.84%
⚡ CPU Simulation encode_primitives[f32, (2000, 512)] 272.2 µs 245.1 µs +11.07%
⚡ Walltime filter_avx2[80] 1.6 µs 1.5 µs +10.21%
⚡ Walltime starts_with_avx512[64, VarBinView] 127.2 µs 115.4 µs +10.16%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/inspiring-ritchie-cwpkl4 (775dcea) with develop (d81bd4c)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant