Repository navigation
Read array slots through generated accessors and drop duplicates - #11430
robert3005 wants to merge 1 commit into
Conversation
Now that the #[array_slots] accessors return references borrowed for the array (or the ArrayView's lifetime 'a), there is no reason to index slots() by XSlots constants and unwrap by hand, or to hand-write accessors that duplicate the generated ones. Replace direct slot indexing with the generated accessors: - validity children read via slots()[XSlots::VALIDITY_CHILD].as_ref() across Bool, Primitive, Decimal, List, ListView, FixedSizeList, VarBin, VarBinView, Masked, ByteBool, OnPair, ParquetVariant, Pco and Zstd now call validity_child(); - required slots read via .clone().vortex_expect(..) (List, ListView, FixedSizeList, Sparse) and the BitPacked block offsets, Variant shredded child and Sparse patch chunk offsets now use their accessors; - the Chunked and Interleave ext methods read chunks and values through the variadic SlotSlice accessors; - Dict, DateTimeParts and Patched build their slot view with slots_view() instead of XSlotsView::from_slots(array.slots()). Remove hand-written accessors identical to the generated ones: - PrimitiveArrayExt::validity_child and DecimalArrayExt::validity_child; - InterleaveArrayExt::array_indices and row_indices. InterleaveSlots is now pub, and InterleaveArraySlotsExt, InterleaveSlots and InterleaveSlotsView are re-exported from arrays::interleave, so the accessors stay public; - ChunkedArrayExt::chunk_offsets_array, replaced by the generated chunk_offsets(). ChunkedArraySlotsExt is now re-exported from arrays::chunked. ChunkedArrayExt::chunks is kept: it returns an owned Vec<ArrayRef>, unlike the generated SlotSlice accessor of the same name, so ChunkedArrayExt calls the generated accessor through its trait path. Callers of the removed methods now need the encoding's ArraySlotsExt trait in scope. Generic whole-slot rewrites that clone or map every slot, and validation of raw slot slices before an array exists, are left as they are. Signed-off-by: Robert Kruszewski <robert@spiraldb.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SWbErpV8gppbsHrfq7fJD
Merging this PR will improve performance by 14.26%2 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Summary
Follow-up to #11418 and #11423. The
#[array_slots]accessors now return references borrowed for the array, or for theArrayView's lifetime'a. So there's no longer a reason to indexslots()byXSlotsconstants and unwrap by hand, or to keep hand-written accessors that duplicate the generated ones.Changes
Direct slot reads replaced with generated accessors
slots()[XSlots::VALIDITY_CHILD].as_ref()now callvalidity_child(), across Bool, Primitive, Decimal, List, ListView, FixedSizeList, VarBin, VarBinView, Masked, ByteBool, OnPair, ParquetVariant, Pco and Zstd..clone().vortex_expect(..)in List, ListView, FixedSizeList and Sparse now use their accessors. So do BitPacked'sblock_offsets, Variant'sshreddedand Sparse'spatch_chunk_offsets.ChunkedArrayExtandInterleaveArrayExtread chunks and values through the generatedSlotSliceaccessors. The index arithmetic withCHUNKS_OFFSETandVALUES_OFFSETis gone.slots_view()instead of buildingXSlotsView::from_slots(array.slots()).Hand-written duplicates removed
PrimitiveArrayExt::validity_childandDecimalArrayExt::validity_child.InterleaveArrayExt::array_indicesandrow_indices.InterleaveSlotsis nowpub, andarrays::interleavere-exportsInterleaveArraySlotsExt,InterleaveSlotsandInterleaveSlotsView, so the accessors stay public.ChunkedArrayExt::chunk_offsets_array, replaced by the generatedchunk_offsets().arrays::chunkednow re-exportsChunkedArraySlotsExt.ChunkedArrayExt::chunksis kept. It returns an ownedVec<ArrayRef>, unlike the generatedSlotSliceaccessor of the same name, soChunkedArrayExtcalls the generated accessor through its trait path.Left as they are: code that copies or rewrites every slot generically, and validation of raw slot slices before an array exists.
No behavior changes are intended. Existing tests cover these paths.
Checks run:
cargo clippy --workspace --all-targets -- -D warnings: clean. Excludesvortex-duckdb,vortex-sqllogictest,duckdb-benchandlance-bench, whose build scripts could not run in the sandbox. None of them call the removed methods.cargo test -p vortex-array --lib: 5910 passed. Before the squash and rebase,cargo testalso passed on the eight changed encoding crates.API Changes
PrimitiveArrayExt::validity_child,DecimalArrayExt::validity_child,InterleaveArrayExt::array_indices,InterleaveArrayExt::row_indicesandChunkedArrayExt::chunk_offsets_array. The generated accessors replace them, so callers need the encoding's…ArraySlotsExttrait in scope. Usechunk_offsets()in place ofchunk_offsets_array().InterleaveSlots,InterleaveSlotsViewandInterleaveArraySlotsExtfromarrays::interleave, andChunkedArraySlotsExtfromarrays::chunked.Generated by Claude Code