Latest Results
Store i128 assembly into a pre-sized buffer, and dedupe part-wise kernels
Re-running `benches/decimal_assemble.rs` after the review corrected a
claim the previous commit made. Specializing the part count is worth
1.85x on `i256`, as reported, but on `i128` it is worth only ~1.04x — the
1.25x figure did not reproduce. What actually costs on `i128` is the
write: pushing into a reserved buffer instead of storing into a pre-sized
one is the whole difference at 16 bytes per row.
A new `i128_row_write` variant isolates it, holding the loop shape fixed
and changing only the output buffer. Over 65,536 rows, `fastest` of three
runs each:
| shape | i128 | i256 |
| ---------------------------- | ------- | ------- |
| row, runtime part count | 143 µs | 351 µs |
| row, const part count, push | 138 µs | 190 µs |
| row, const part count, write | 83 µs | 196 µs |
| column, lane writes | 103 µs | 438 µs |
So the columnar shape was never the interesting axis: it beats the
*pushing* row loop on `i128` but still loses to the single-pass write,
and the second pass buys nothing once the push is gone. On `i256` the
write shape ties the push shape, because 32 bytes of stores per row
dominate either way, so only `assemble_i128` changes. Through the array
API, one lower part goes 138 µs -> 83 µs (1.6x); three parts is unchanged
at ~209 µs.
The rest is cleanup from the same review.
Seven kernels open-coded "map every part, rebuild the array", and two of
them had already been fixed in this branch for dropping the lower parts
on the floor. `map_parts`, `with_msp` and `decimal_dtype` replace all
seven, so a part-wise op cannot silently lose a part, and the argument
for why an MSP-only rebuild is sound lives in one doc comment instead of
being restated or omitted per site.
Dead code: `DecimalBytePartsDataParts` had exactly one reference in the
repository — its own definition — and this branch had been growing it a
field and doc comments. The `[first]` arm of the `i256` dispatch is
unreachable, since one lower part under a <=64-bit MSP always lands in an
`i128`; a comment now says so where the arm was.
Visibility: `assemble_decimal`, `assembled_values_type` and
`LOWER_PART_DTYPE` had no callers outside the crate and are now
crate-private. `assemble_decimal` was public only so the benchmark could
call it, but `canonicalize_byte_parts` already measures the same assembly
through the array API, so the two `*_assemble_shipped` benches go with
it. As public API it could also panic rather than error on an unsigned
MSP, since signedness is only checked on the zero-parts path.
The metadata accessor `lower_parts()` returned a count while the
generated slots accessor of the same name returns the arrays, both in
scope in the same module; it is now `lower_part_count()`. The btrblocks
scheme spelled the child layout as `1 + MAX_LOWER_PARTS` and `idx + 1`
where the encoding crate has named slot constants; it now uses them.
Three hand-rolled LCGs become `StdRng::seed_from_u64`, matching the rest
of the repo. Four one-line rejection tests became one `rstest` in the
previous commit; the two removed columnar bench variants are recorded in
the module doc with their numbers rather than kept as dead code.
Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>claude/decimal-byte-parts-pr-p0ugog perf(sparse): build the fixed-size-list fill tile once, not once per gap
`append_fill` handed the fill scalar to `Constant::append_to_builder` per gap,
which materialized the scalar's elements into an array every time. Hoist that
out of the loop the way the list path already does: the fill's elements become
an array once, up front, and every gap tiles that same array through
`FixedSizeListBuilder::append_array_as_repeated_list`.
Elements that are all the same scalar stay a constant array, so tiling them
over a gap costs nothing at all however many rows it covers. Anything else is
copied in per row, because a fixed-size list holds its elements back to back
and cannot point its rows at one shared copy.
`canonicalize_sparse_fixed_size_list` medians, against develop:
(512, 7, 4) 21.1 us -> 25.6 us (was 55.0 us)
(1024, 17, 8) 29.1 us -> 32.9 us (was 63.9 us)
(8192, 1024, 4) 150.1 us -> 140.1 us (was 142.3 us)
What is left of the gap against develop is the per-row copy: each row pays its
own append for `list_size` elements, where tiling a run could double the region
already written instead.
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>claude/constant-fast-paths-9ze0t6 perf(array): accumulate nested builder validity without a null buffer
A nested builder learns about validity from two sources: one row at a time
as scalars are appended, and a whole array's worth at a time as arrays are.
Only the first needs a null buffer, but `LazyBitBufferBuilder` treated both
the same, so every appended array had its validity executed into a `Mask`
and its bits copied.
`ValidityBuilder` keeps a whole array's validity as a run and concatenates
the runs at the end, the way `Validity::concat` already does for
`StructArray::try_concat`. `AllValid` and `AllInvalid` runs cost nothing,
array-backed runs are bool arrays that are already built, and a builder that
only ever saw uniform validity still answers from its nullability rather
than producing a bool array. However few values a run covers, it is kept as
it arrived, so a builder's validity is split on exactly the boundaries its
children are.
`StructBuilder`, `ListBuilder`, `ListViewBuilder` and `FixedSizeListBuilder`
use it; the leaf builders keep `LazyBitBufferBuilder`.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>claude/builders-lazy-validity-9ze0t6 feat(array): builders no longer canonicalize their children
Nested builders used to push every appended child array through
`append_to_builder`, which decoded it into the child's canonical builder.
That work is wasted: `Canonical` only promises a canonical *top level*, so
struct fields, list elements and extension storage are free to stay
compressed.
Introduce `ChildBuilder`, which accumulates a child as a `Vec<ArrayRef>` of
chunks plus a scalar builder for the values that cannot come from an array,
and stitches them into a `ChunkedArray` on `finish` when more than one chunk
accumulated. `StructBuilder`, `ListBuilder`, `ListViewBuilder`,
`FixedSizeListBuilder` and `ExtensionBuilder` now hold their children this
way.
However short the appended array, it becomes a chunk. Deciding on the caller's
behalf that its values are cheaper copied than referenced would be guessing at
a boundary only the caller can see, and a caller that wants them copied has
`append_scalar`. A child is therefore chunked on exactly the boundaries it was
appended on.
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>claude/builders-canonical-children-9ze0t6 Latest Branches
0%
codex/tiled-fixed-size-list -11%
claude/decimal-byte-parts-pr-p0ugog -31%
claude/chunked-canonical-via-builder-9ze0t6 © 2026 CodSpeed Technology 
