Remove some unsafe and allocations when creating PrimitiveArrays from Vec and from_trusted_len_iter#9299
Conversation
from_trusted_len_iterunsafe and allocations when creating PrimitiveArrays from Vec and from_trusted_len_iter
| }; | ||
| PrimitiveArray::from(data) | ||
| let nulls = | ||
| Some(NullBuffer::new(BooleanBuffer::new(null, 0, len))).filter(|n| n.null_count() > 0); |
There was a problem hiding this comment.
Note the
- no more
unsafe - Use avoid
vec![buffer]allocation
|
run benchmark arrow_statistics |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
run benchmark builder |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
There doesn't seem to be any measurable difference in the benchmarks |
Which issue does this PR close?
ArrayDatawith direct Array construction, when possible #9298Rationale for this change
While reviewing #9294 from @Dandandan I noticed some other places where we can avoid making ArrayData and thus save some allocations (and
unsafe)I don't expect this to make a huge performance difference, but every little allocation helps, and I think the change is justified simply from the perspective of avoiding some more
unsafeWhat changes are included in this PR?
Construct primitive arrays directly
Are these changes tested?
By existing CI
Are there any user-facing changes?