Apache Arrow

Apache Arrow Go 18.8.0 Release

2026-09-08T00:00:00-04:00

The Apache Arrow team is pleased to announce the v18.8.0 release of Apache Arrow Go. This minor release covers 283 commits from 16 distinct contributors.

Contributors

$ git shortlog -sn v18.7.0..v18.8.0
Minh Vu
Matt Topol
Colton Loftus
Digvijay
Tom Frank
Derek Perkins
Lucas Valente
Madan kumar
Neelesh Salian
Nick Ripley
Pedro Matias
Raúl Cumplido
Ruihao Chen
Tobias Pütz
Tomáš Karela Procházka
Willem Jan

Highlights

Arrow

Features

Added VariantGet for efficient path extraction from variant arrays #1206
Added list_element, cumulative_sum, cumulative_sum_checked, and dictionary_encode compute functions #1138, #1139, #1140
Added ValueAsAny for accessing Arrow array values as native Go values #1170
Strengthened IPC file validation for malformed block framing and lengths #1097

Bug Fixes

Fixed a memory leak when calling Resize(0) on non-empty builders #995
Preserved exact precision when casting strings to Decimal128 and Decimal256 scalars #1107

Performance Improvements

This is a performance-heavy release: 32 of the merged pull requests are dedicated optimizations. Unless noted otherwise, the figures below are the PR-reported benchmarks on an Apple M1 Pro, comparing the new path against the scalar baseline it replaces.

Expanded SIMD acceleration for compute arithmetic, comparisons, and filtering on ARM64 and AMD64. Arithmetic kernels run 4.1-4.4x faster on int64 and float64 adds, int64 array/array comparisons drop from 313.6us to 29.3us per op (10.7x), narrow 8- and 16-bit integer comparisons reach 12.8-15.1x, and null-free 32-bit fixed-width filters compress up to 11.8x faster on 1M-element inputs #1265, #1266, #1284, #1286, #1287, #1288
Building take indices from fragmented selection vectors is no longer quadratic. For a 1M-element alternating filter it drops from 4.31s to 515us, and a serial FilterRecordBatch over the same filter falls from 2.06s to 831us #1287
Added SIMD-accelerated packing of boolean slices and validity data into Arrow bitmaps: BooleanBuilder packing of 65,536 values improves roughly 4-5x (23.4us to 3.9us), batched validity packing is up to 4.4x faster, and ARM64 NEON bitmap operations gain 2.2-2.9x #1283, #1185, #1264, #1090
Batched run-end encoded appends turn a millisecond-scale operation into a microsecond-scale one for null and empty-value runs, cutting 1.16MB across 55 allocations per op down to 1.7KB across 19 #1214
Concatenation avoids temporary buffers and copies bitmaps directly: 65,536 int64 values spread over 8,192 chunks go from 571.7us to 214.5us (-62%) with allocations falling from 8,199 to 6 per op #1194, #1224
Serial and small-batch execution skips goroutine and span overhead: single-span kernel dispatch nearly halves latency at length 1 (342.0ns to 189.0ns), serial record filtering over 128 columns goes from 709.8us to 420.9us, and serial Take plus IPC encoder scratch reuse cut per-op allocations across the board #1233, #1245, #1226, #1221
CSV writing no longer materializes rows first, cutting a 1,000-row by 16-column write from 4.78ms to 0.84ms (-82%) and allocated bytes by 81% #1190

Parquet

Features

Added read and write support for GeoArrow extension types using Parquet geometry and geography logical types #960, #969
Added codec-aware dictionary cost fallback and per-column writer controls #1036

Bug Fixes

Correctly populated Bloom filters from dictionary-encoded values #1164
Rejected decimal values that overflow Parquet INT32 or INT64 columns instead of silently narrowing them #1122, #1161
Fixed reading fixed-size lists whose parent values are null #1099

Performance Improvements

Added SIMD BYTE_STREAM_SPLIT encoding on ARM64 and AMD64: 16.9x faster for 4-byte values (113,877ns to 6,745ns per op) and 7.6x for 8-byte values #1260
Boolean decoding now targets Arrow bitmaps directly. All-true RLE boolean pages of 1M values decode in 2.45us instead of 2.65ms, and dense boolean column reads through pqarrow drop 90% in time and 65% in heap use #1246, #1182
Batched encoding and decoding across the core codecs: plain BYTE_ARRAY encoding is 45-62% faster, RLE level encoding is 79% faster for all-defined levels, and DELTA_BINARY_PACKED decoding is 34-66% faster #1193, #1177, #1169
Dictionary encoding allocates far less: numeric insertion of 65,535 float32 values goes from 64,905 allocations per op to 13, and typed byte-array memo insertion cuts time 24% and allocations 50% (Apple M3 Max). Fixed-width dictionary materialization is also vectorized on AVX2 systems #1178, #1272, #1269
Reduced copying when writing binary and fixed-size binary Arrow values to Parquet: 16-byte FixedSizeBinary writes are 72% faster for required columns with statistics disabled, and 64K String writes go from 1.798ms to 1.260ms #1262, #1263
Compressors are reused across pages: random 64KiB gzip page encoding is 6.1x faster (94.7us to 15.6us) with allocations down from 15 to 1 per op, and Brotli encoding drops from about 22 allocations per op to 1 #1238, #1254
Serial reads skip goroutine orchestration, trimming 18% from single-column row-group reads #1235

Changelog

Selected Changes

A selection of the user-facing changes in this release. See the full changelog link below for all 285 merged pull requests.

Arrow

fix: remove superfluous struct field index compare by @Willem-J-an in #970
fix(ipc): reject truncated message frames by @fallintoplace in #991
fix: Fix arm64 assembly stack frame manipulation by @nsrip-dd in #984
fix(arrow/array): Resize(0) on non-empty builder causes memory leak by @serramatutu in #995
fix(arrow/csv): preserve decimal precision by @fallintoplace in #1022
refactor(arrow/avro): migrate from hamba/avro to twmb/avro by @prochac in #830
feat(flightsql): allow configuring the driver memory allocator by @fallintoplace in #1076
feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult by @ennuite in #732
feat(arrow/array): roll back JSON builder state after failed rows by @fallintoplace in #1113
feat(arrow/array): add ValueAsAny for native Go values by @dgvj-work in #1170
feat(extensions): add VariantGet for path extraction from variant arrays by @nssalian in #1206
fix(arrow/extensions): use canonical Variant extension name by @dgvj-work in #1241
fix(arrow/extensions): reject Null Variant typed_value by @dgvj-work in #1243
fix(arrow/ipc): validate file block framing and lengths by @fallintoplace in #1097
fix(arrow/ipc): avoid deadlock on compression errors by @fallintoplace in #1142
fix(arrow/scalar): preserve exact decimal string casts by @fallintoplace in #1107
fix(arrow/scalar): align timestamp timezone parsing by @fallintoplace in #1109
fix(arrow): validate precision in NewDecimalType by @fallintoplace in #1162
fix(arrow): reject duplicate union type codes by @fallintoplace in #1131
fix(arrow/csv): preserve first row during headerless inference by @fallintoplace in #1145
fix(arrow/csv): append null for later parse failures by @fallintoplace in #1149
fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array by @winklemad in #1219
fix(arrow/cdata): validate imported schema topology by @fallintoplace in #1046

Compute

feat(compute): add list_element by @fallintoplace in #1138
feat(compute): add cumulative_sum and cumulative_sum_checked by @fallintoplace in #1139
feat(compute): add dictionary_encode by @fallintoplace in #1140
fix(compute): round negative HalfToOdd ties correctly by @fallintoplace in #1094
fix(compute): handle odd temporal multiples by @fallintoplace in #1119
fix(compute): preserve caller context during execution by @fallintoplace in #1154
fix(compute): discard results after cancellation by @zeroshade in #1278
fix(compute/exprs): avoid empty field reference panic by @fallintoplace in #1130

Parquet

feat(parquet/schema): Wire up logical type functionality for writing geo types by @C-Loftus in #960
feat(parquet/pqarrow): Read support for extension types / geo by @C-Loftus in #969
feat(parquet): add WithDictionaryCostFallback writer property by @tom-eon in #1036
fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width by @tom-eon in #1027
fix(parquet/encoding): read ahead in the streaming value buffer by @joechenrh in #937
fix(parquet/pqarrow): prefer stored schema extension type on read by @twuebi in #1051
fix(parquet/pqarrow): require Variant value field when writing by @dgvj-work in #1242
fix(parquet): guard byte-stream-split encoder release against never-flushed buffer by @tom-eon in #1273
fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output by @zeroshade in #1256
fix(parquet): read null fixed-size-list parents by @fallintoplace in #1099
fix(parquet/pqarrow): reject decimal overflow in integer columns by @fallintoplace in #1122
fix(parquet/pqarrow): reject decimal overflow when writing integers by @fallintoplace in #1161
fix(parquet): build bloom filters from dictionary entries by @fallintoplace in #1164
fix(parquet): preserve dictionary index types by @fallintoplace in #1098
fix(parquet): restore Arrow duration units from schema metadata by @fallintoplace in #1095
fix(parquet/compress): honor LZ4 raw destination contract by @fallintoplace in #1147
fix(parquet): row-align V1 pages with offset indexes by @fallintoplace in #1075
fix(parquet/variant): validate metadata offset tables by @fallintoplace in #1063

Performance

perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull by @tom-eon in #1090
perf(bitutil): add ARM64 NEON bitmap operations by @fallintoplace in #1264
perf(arrow/array): pack validity values in batches by @fallintoplace in #1185
perf(arrow/array): SIMD-pack boolean slices into bitmaps by @fallintoplace in #1283
perf(arrow/array): batch run-end encoded appends by @fallintoplace in #1214
perf(arrow/array): avoid temporary buffers when concatenating by @fallintoplace in #1194
perf(arrow/array): concatenate bitmaps directly by @fallintoplace in #1224
perf(compute): add ARM64 NEON arithmetic kernels by @fallintoplace in #1265
perf(compute): add ARM64 NEON comparison kernels by @fallintoplace in #1266
perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers by @fallintoplace in #1288
perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters by @fallintoplace in #1286
perf(compute): vectorize mixed int32 filters with AVX2 by @fallintoplace in #1284
perf(compute): SIMD-compact fragmented filters into take indices by @fallintoplace in #1287
perf(compute): fast-path single-span execution by @fallintoplace in #1233
perf(arrow/compute): avoid goroutines for serial record filtering by @fallintoplace in #1245
perf(arrow/compute): avoid goroutines for serial take by @fallintoplace in #1226
perf(parquet): SIMD BYTE_STREAM_SPLIT encoding by @fallintoplace in #1260
perf(parquet): vectorize fixed-width dictionary materialization by @fallintoplace in #1269
perf(parquet): optimize numeric dictionary insertion by @fallintoplace in #1178
perf(parquet): use typed memo insertion for byte-array dictionaries by @derekperkins in #1272
perf(parquet): batch plain BYTE_ARRAY encoding by @fallintoplace in #1193
perf(parquet): encode RLE levels in batches by @fallintoplace in #1177
perf(parquet): batch delta binary packed decoding by @fallintoplace in #1169
perf(parquet): decode RLE booleans into bitmaps by @fallintoplace in #1246
perf(parquet/pqarrow): decode booleans directly to bitmaps by @fallintoplace in #1182
perf(parquet/pqarrow): write binary values from Arrow offsets by @fallintoplace in #1262
perf(parquet/pqarrow): write FixedSizeBinary values directly by @fallintoplace in #1263
perf(parquet/pqarrow): use synchronous path for serial reads by @fallintoplace in #1235
perf(parquet/compress): reuse gzip writers by @fallintoplace in #1238
perf(parquet/compress): reuse Brotli writers across encodes by @fallintoplace in #1254
perf(arrow/csv): avoid materializing rows before writing by @fallintoplace in #1190
perf(arrow/ipc): reuse record encoder scratch by @fallintoplace in #1221

New Contributors

@C-Loftus made their first contribution in #960
@tom-eon made their first contribution in #1027
@prochac made their first contribution in #830
@dgvj-work made their first contribution in #1170
@ennuite made their first contribution in #732
@derekperkins made their first contribution in #1272
@winklemad made their first contribution in #1219

Full Changelog: https://github.com/apache/arrow-go/compare/v18.7.0...v18.8.0

Apache Arrow nanoarrow 0.9.0 Release

2026-08-14T00:00:00-04:00

The Apache Arrow team is pleased to announce the 0.9.0 release of Apache Arrow nanoarrow. This release consists of 38 resolved GitHub issues from 5 contributors.

Release Highlights

In addition to a number of bugfixes and minor build system improvements, we added several new features in nanoarrow 0.9.0.

Dictionary decoding support in IPC reader
Reference-counted array/buffer support
LZ4 decompression support in R and Python bindings

See the Changelog for a detailed list of contributions to this release.

Features

Dictionary decode support

Whereas the nanoarrow IPC reader suppports most Arrow IPC features, dictionary support was a long requested gap in the reader functionality (mostly requested by users of the DuckDB nanoarrow extension, which uses nanoarrow's reader). Dictionary encoding is used to reduce the size of frequently repeated values and is the serialized equivalent of the "dictionary" data type that is exposed in most Arrow implementations.

In nanoarrow 0.9.0 built with the IPC feature enabled, streams that include the most common forms of dictionary encoding (i.e., dictionary replacement) should now work out of the box. This includes nested/complex dictionary types and dictionary replacement but does not include "delta" dictionaries (i.e., dictionaires that grow larger as more values are encountered in the encoded values).

In R this is accessible via read_nanoarrow(); in Python this is accessible via nanoarrow.ArrayStream.from_readable(); in C this is available via the higher level ArrowIpcArrayStreamReader API. Lower level users of the ArrowIpcDecoder will have to update existing usage to use the ArrowIpcDecoder...WithDictionaries() variants of some functions to support input with dictionary schemas or batches.

Reference-counted array/buffer support

In previous versions (since the introduction of the IPC reader), the ArrowIpcSharedBuffer has supported reading IPC streams and sharing an underlying set of data buffers for a group of arrays; however, this wasn't quite sufficient for the more complex case of decoding a dictionary and attaching cheaply-cloned shared "values" arrays for potentially many batches. Version 0.9.0 moves this functionality to the ArrowSharedBuffer and expands it to support moving all of an array's buffers into a shared state that can be more cheaply cloned.

LZ4 decompression support in R and Python

While LZ4 decompression support was has long been available via the pluggable decoder framework (and available since 0.8.0 as a built-in compile time option), reading IPC streams with LZ4 buffer compression was not possible in the R or Python bindings. In 0.9.0, the requisite configuration options were added such that the packages are built with LZ4 when it is available on the system.

import io
import nanoarrow as na
import pyarrow as pa

buf = io.BytesIO()
batch = pa.record_batch({"x": range(1000)})
with pa.ipc.new_stream(buf, batch.schema, options=pa.ipc.IpcWriteOptions(compression="lz4")) as w:
    w.write_batch(batch)

buf.seek(0)
na.ArrayStream.from_readable(buf).read_all()
# nanoarrow.Array>[1000]
# {'x': 0}
# {'x': 1}
# {'x': 2}
# {'x': 3}
# {'x': 4}
# {'x': 5}
# {'x': 6}
# {'x': 7}
# {'x': 8}
# {'x': 9}
# ...and 990 more items

library(nanoarrow)
library(reticulate)

# IPC Write with compression not available in arrow/R
pa <- reticulate::import("pyarrow")
io <- reticulate::import("io")

buf <- io$BytesIO()
batch <- arrow::record_batch(x = 1:1000)
writer <- pa$ipc$new_stream(buf, batch$schema, options = pa$ipc$IpcWriteOptions(compression = "lz4"))
writer$write_batch(batch)
writer$close()

nanoarrow::read_nanoarrow(as.raw(buf$getvalue())) |>
  tibble::as_tibble()
#> # A tibble: 1,000 × 1
#>        x
#>    
#>  1     1
#>  2     2
#>  3     3
#>  4     4
#>  5     5
#>  6     6
#>  7     7
#>  8     8
#>  9     9
#> 10    10
#> # ℹ 990 more rows

Contributors

This release consists of contributions from 5 contributors in addition to the invaluable advice and support of the Apache Arrow community.

$ git shortlog -sn apache-arrow-nanoarrow-0.9.0.dev..apache-arrow-nanoarrow-0.9.0
Dewey Dunnington
Andrew Kane
Bryce Mecum
Michael Osipov
Oliver Borchert

Apache Arrow 25.0.1 Release

2026-08-10T00:00:00-04:00

The Apache Arrow team is pleased to announce the 25.0.1 release. The 25.0.1 release is mostly a bugfix release that includes 9 resolved issues on 10 distinct commits from 6 distinct contributors.

See the Install Page to learn how to get the libraries for your platform.

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

C++ notes

Fix possible Segfault on SVE128 unpack (GH-50503), (GH-50605).
Bump bundled mimalloc to fix possible Segfault with bundled mimalloc (GH-50471), (GH-50428).

Python notes

Add performance improvement in order to solve Apache Spark's performance regression on their switch from regular Python UDFs to Arrow serialization by default (GH-50326).
Narrow the Feather deprecation added in 25.0.0 to legacy V1 only. pyarrow.feather V2 reads and writes no longer warn, while V1 reads and writes emit DeprecationWarning instead of FutureWarning (GH-50808).

Other modules and languages

No general changes were made to the other libraries or languages.

Apache Arrow ADBC 24 (Libraries) Release

2026-07-28T00:00:00-04:00

The Apache Arrow team is pleased to announce the version 24 release of the Apache Arrow ADBC libraries. This release includes 57 resolved issues and 142 merged pull requests from 28 distinct contributors.

This is a release of the libraries, which are at version 24. The API specification is versioned separately and is at version 1.1.0.

The subcomponents are versioned independently:

C/C++/GLib/Go/Python/Ruby: 1.12.0
C#: 0.24.0
Java: 0.24.0
JavaScript: 0.24.0
R: 0.24.0
Rust: 0.24.0

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Release Highlights

Note: we are planning to require C++20 starting from the next release. Also, we will drop support for Python 3.10 no earlier than the release after next i.e. no earlier than release 26. Release 26 is expected in about 3-4 months (October~November 2026), and the EOL for Python 3.10 is October 2026.

Breaking Changes and Deprecations

Development of the ADBC drivers for Apache DataFusion, BigQuery, Databricks, and Snowflake has moved to the ADBC Driver Foundry, an independent, community-maintained project separate from Apache Arrow. The drivers remain available and continue to be developed there. No further releases of those drivers will come from this project, but existing packages will remain available, possibly as archived listings. For background, see the ADBC documentation on driver development. Current driver locations and maintainer information are listed under Drivers.

Similarly, the experimental, incomplete support for Amazon Redshift in the PostgreSQL driver has been removed. A dedicated ADBC driver for Amazon Redshift is available from the ADBC Driver Foundry.

The ADBC drivers for Apache Arrow Flight SQL, PostgreSQL, and SQLite continue to be maintained and released by this project.

Java API definitions were narrowed so that close is only declared to throw AdbcException as a checked exception. (#4451)

The PostgreSQL driver now lazily initializes transactions to make it work better with connection pools. (#4424)

Documentation

The documentation has been overhauled and is now organized around three common tasks: finding and installing drivers, connecting through a client library, and developing a driver. (#4525)

The revised docs emphasize ADBC's cross-language driver model: drivers are typically distributed as shared libraries that can be used from any supported language or client.

A new Tools & Integrations page highlights tools and frameworks that integrate with ADBC. We welcome contributions to add more integrations.

A new Connection Profiles page shows how to use reusable profiles to configure ADBC connections.

The docs also describe how most drivers now recognize URIs with schemes matching their names, and how driver managers can use the scheme to resolve the driver when it is not otherwise specified.

Core APIs & Client Libraries

C# now has a native driver manager, allowing it to load drivers, manifests, and profiles (#4075, #4340, #4341, #4330). The core libraries are now compatible with AOT compilation, enabling drivers to be built as standalone shared libaries (#4243, #4318).

The Go database/sql adapter now supports converting more Arrow types to Go types (#4416).

The Java core APIs now support a "fluent" style ingest API (#4466). Also, support for dynamically loading drivers for use in Java has been greatly expanded and should now support all of the ADBC APIs and features expected of a client library (#4452, #4211, #4411, #4202, #4359, #4212, #4263, #4229, #4203, #4361, #4362, #4249, #4250, #4398, #4397, #4423, #4395, #4396, #4391).

The JavaScript client library no longer requires the driver parameter and can infer the driver to load based on the URI, or can accept a profile (#4357).

The R client library no longer requires the driver parameter and can infer the driver to load based on the URI, or can accept a profile (#4535).

Some tweaks have been made to the Rust core APIs to better support interop with dynamically loaded drivers and make certain conventions clearer (#4427, #4510, #4350, #4141, #4181, #4473, #4469).

Drivers

This project continues to maintain and release the ADBC drivers for Apache Arrow Flight SQL, PostgreSQL, and SQLite. As mentioned above, the DataFusion, BigQuery, Databricks, and Snowflake drivers are now maintained in the ADBC Driver Foundry, alongside many others.

The Flight SQL driver now recognizes URIs with the flightsql:// scheme (#4488). It also has more support for logging and OpenTelemetry tracing (#4322, #4486).

The PostgreSQL driver now uses libpq version 18.4 (up from 16.9) (#4566). Several bugs have been fixed around handling of the NUMERIC type, and it has been optimized on platforms where int128 is available (generally, platforms other than Windows) (#4536, #4499, #4523, #4498). GetObjects now populates the xdbc_type_name field (#4457). JSON columns are now returned with the arrow.json extension type (#4415), and ingesting into JSONB columns is now supported (#4505).

The SQLite driver now uses SQLite version 3.53.1 (up from 3.51.2) (#4566). It now recognizes URIs with the sqlite:// scheme (#4463).

Contributors

$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-23..apache-arrow-adbc-24
David Li
Bryce Mecum
Fredrik Fornwall
Curt Hagenlocher
eitsupi
Mandukhai Alimaa
davidhcoe
Matt Topol
takuya kodama
Ian Cook
복준수
Artur Rakhmatulin
Austin Bonander
Bruce Irschick
Daniel_McBride
Emil Sadek
Arnold Wakim
Aurélien Pupier
Dan Liu
Felipe Oliveira Carvalho
Kent Wu
Neal Richardson
Nir Portal
Pavel Agafonov
Rishav Rungta
Shubham Pandey
mete
xinyu.lin

Roadmap

We are working on the next revision of the API standard, focusing on missing features (primarily metadata/catalog data). We welcome anyone interested in contributing. Current progress can be found in the 1.2.0 specification milestone.

Getting Involved

We welcome questions and contributions from all interested. Issues can be filed on GitHub, and questions can be directed to GitHub or the Arrow mailing lists.

Apache Arrow Go 18.7.0 Release

2026-07-21T00:00:00-04:00

The Apache Arrow team is pleased to announce the v18.7.0 release of Apache Arrow Go. This minor release covers 101 commits from 23 distinct contributors.

Contributors

$ git shortlog -sn v18.6.0..v18.7.0
Minh Vu
Matt Topol
Fredrik Fornwall
Sai Asish Y
Jared Yu (余启正)
Neelesh Salian
Ondřej Pavela
abir
Alex Dubov
Bryce Mecum
David Li
David Zhao
Dima Kuznetsov
Lucas Valente
Mahdi Dibaiee
Matan Rosenberg
Patzifist
Ruihao Chen
Samuel Arnold
Tobias Pütz
Truffle
Varun Venkatesh
daniel-adam-tfs

Highlights

Arrow

Features

Support casts between binary/utf8 and binary_view/utf8_view #802
Added a Resize method to RecordBuilder #805
Support dictionary arrays for Filter and Take #716
Add an UnshredVariant method for collapsing a shredded variant array #859
Add implementation of TimestampWithOffset canonical extension type #558

Bug Fixes

Made C Data interface stream memory release deterministic #793
Preserve large integer precision in JSON decoding #816

Parquet

Features

Support writing LARGE_LIST arrays #838
Opt-in streaming reads for large data pages #880

Bug Fixes

Corrected a couple variant bugs #841, #840, #939
Properly align DataPageV2 pages to row boundaries in spaced writes #883

Performance Improvements

Added SIMD-accelerated byte-stream-split decoding #654
Avoid constructing ColunChunkMetaData in page index range determination #831
Improve BloomFilter memory recycling and pooling #864

Changelog

What's Changed

fix(release): keys download by @zeroshade in #781
parquet/compress: enable WithAllLitEntropyCompression(true) for zstd by @varun0630 in #779
release: fix syntax error in verify_rc.sh by @amoeba in #782
fix(parquet): align dictionary fallback with parquet-mr by @twuebi in #786
fix(arrow/cdata): make nativeCRecordBatchReader deterministic by @zeroshade in #793
fix(compute/kernels): map FSB byte-width to numeric memo type for is_in by @SAY-5 in #797
fix(arrow/flight): deliver response headers eagerly in streaming client middleware by @zeroshade in #801
fix(ipc): preserve map child field metadata by @dimakuz in #809
fix(avro): error on complex (non-nullable) unions instead of silently dropping (#777) by @SAY-5 in #808
fix(parquet/metadata): do not pre-set hasDistinctCount in stat factories by @SAY-5 in #807
feat(compute): support casts between binary/string and binary_view/string_view by @zeroshade in #802
fix(arrow/array): preserve large integer precision in JSON decoding by @zeroshade in #816
feat(arrow/array): add Resize method to RecordBuilder by @oakad in #805
perf(parquet/pqarrow): cap RecordReader batch size to actual row count by @paveon in #817
docs(parquet): correct DefaultDataPageSize comment from 1K to 1M by @zeroshade in #822
fix(arrow/array): silence copylocks warning in numeric_generic.go by @zeroshade in #823
feat(arrow/compute/exprs): support expr.IntervalYearToMonthLiteral in literalToDatum by @zeroshade in #825
fix(parquet): return error instead of panicking on first-write failure by @zeroshade in #824
[Parquet] Add SIMD-accelerated byte-stream-split decoding by @daniel-adam-tfs in #654
perf(parquet/metadata): avoid constructing ColumnChunkMetaData in page index range determination by @paveon in #831
feat: filter and take for dictionary arrays by @lesam in #716
fix(arrow/array): adding string boundary checking for (*String).Value by @happydave1 in #844
fix(parquet/variant): correct binary search bounds in ObjectValue.ValueByKey by @qzyu999 in #841
fix(parquet/variant): correct is_large bit position in valueSize for arrays by @qzyu999 in #840
feat(avro): support local-timestamp logical types by @abir-eon in #832
fix(avro): append raw bytes by @matan129 in #850
fix(arrow/array): use scale-aware ValueStr in decimal array String() by @zeroshade in #849
feat(parquet/pqarrow): support writing LARGE_LIST types by @lidavidm in #838
fix(arrow/scalar): implement Release()/Retain() on *scalar.Extension by @zeroshade in #851
docs(arrow/ipc): document that Writer and FileWriter are not concurrency safe by @zeroshade in #856
fix(parquet): add WriteBatchSpacedWithError to surface spaced-write failures by @zeroshade in #852
ci: pin Go to go.mod in RC verify so the matching toolchain is used by @zeroshade in #857
feat(arrow/extensions): add UnshredVariant to collapse a shredded variant array by @nssalian in #859
fix(arrow/compute): accept type max value in safe decimal-to-int cast by @zeroshade in #862
fix(array): validate negative bounds in NewSliceData by @fallintoplace in #870
fix(array): dictionary null index zeroing by @fallintoplace in #871
fix(array): struct stringer required child mask by @fallintoplace in #873
fix(array): empty chunked approximate equality by @fallintoplace in #874
fix(array): validate dense union offset lengths by @fallintoplace in #875
Use semantic type equality in AddColumn by @fallintoplace in #878
fix(array): clear dictionary data on reset by @fallintoplace in #876
Reject negative chunked slice bounds by @fallintoplace in #879
fix(array): array data hashing coverage by @fallintoplace in #872
Support dense union slice offsets by @fallintoplace in #877
Support sparse union sliced type ids by @fallintoplace in #881
fix(build): correct phony target declaration by @fallintoplace in #890
fix(parquet/compress): Gzip CompressBound by @fallintoplace in #886
fix(parquet): free excluded stats bytes in ApplyStatSizeLimits by @mdibaiee in #885
fix(tmpl): preserve escaped JSON string bytes by @fallintoplace in #889
fix(parquet/compress): Brotli CompressBound by @fallintoplace in #884
fix(parquet): align DataPageV2 pages to row boundaries in spaced writes by @truffle-dev in #883
fix(array): guard BinaryViewBuilder size checks by @fallintoplace in #888
fix: clarify StringViewBuilder UnmarshalJSON error message by @fallintoplace in #894
fix: clarify StringViewBuilder.UnmarshalOne expected type by @fallintoplace in #897
fix: guard multiBufferBuilder UnsafeAppend against short copy by @fallintoplace in #896
fix(array): make SetNull idempotent and track null count by @fallintoplace in #905
fix(ci): accept --json and pin gobenchdata by @fallintoplace in #906
build: verify Apache RAT jar checksum before execution by @fallintoplace in #908
fix(parquet): read full bloom filter buffers by @fallintoplace in #892
fix(docs): fix CPU markdown link in README by @fallintoplace in #907
fix(parquet): validate DataPageV2 decoded length by @fallintoplace in #911
fix(arreflect): validate primitive conversion assignment overflow checks by @fallintoplace in #903
fix(parquet): clamp repLevels in byte-array DataPageV2 row-boundary alignment by @zeroshade in #919
fix(parquet/metadata): fix BloomFilter memory recycling, doc contract, and benchmark by @Patzifist in #864
fix(flight/flightsql): recover real error when prepared DoPut stream closes early by @fornwall in #920
fix: guard protobuf oneof/union reflection on nil oneof values by @fallintoplace in #910
fix(parquet): copy ByteArray statistics min/max to prevent use-after-free by @zeroshade in #917
fix(array): bounds check FixedSizeBinary.Value by @fallintoplace in #923
fix(array): reject unsupported type IDs by @fallintoplace in #925
fix(array): validate JSON start offsets by @fallintoplace in #929
fix(scalar): validate dictionary indices before conversion by @fallintoplace in #927
fix(array): retain dictionary before replacement by @fallintoplace in #926
feat(parquet): opt-in streaming reads for large data pages by @joechenrh in #880
fix(parquet): propagate row-group and page-index write errors by @fallintoplace in #921
fix(array): honor struct offsets in constructors and validity masks by @fallintoplace in #922
fix(parquet): make column reader cleanup idempotent by @fallintoplace in #924
fix(scalar): hash list values logically by @fallintoplace in #928
fix(array): recursively validate nested arrays by @fallintoplace in #930
feat(parquet/variant): add Metadata.SizeBytes to split a concatenated variant by @nssalian in #939
fix(cdata): validate imported array sizes by @fallintoplace in #931
fix(avro): decode nullable logical timestamps by @abir-eon in #935
fix(flight/flightsql/example): support dictionary-encoded parameter columns by @fornwall in #936
fix(arrow/scalar): reject negative indices in GetScalar by @fornwall in #940
fix(flight): reject malformed unary bearer headers by @fallintoplace in #941
fix(flight): reject Basic credentials without a separator by @fallintoplace in #942
fix(memory): keep allocator ownership across Buffer.Reset by @fallintoplace in #947
fix(parquet): limit page allocations by @fallintoplace in #953
fix(parquet): return decompression errors by @fallintoplace in #954
fix(parquet): return errors for malformed encrypted payloads by @fallintoplace in #945
fix: repair failing benchmarks and stop bench.sh masking failures by @zeroshade in #938
fix(parquet): handle short input in GetBatchBools by @fallintoplace in #946
fix(arrow): validate timestamp timezones by @fallintoplace in #952
fix(ipc): validate message framing bounds by @fallintoplace in #955
fix(parquet): reject truncated bit-packed batches by @fallintoplace in #943
fix(compute): handle comparison output offsets by @fallintoplace in #956
fix(arrow/array): avoid allocating for ReserveData(0) by @fallintoplace in #895
fix(compute): release temporary arrays in binary view casts by @fallintoplace in #898
fix(arrow/array): validate map builder entry lengths by @fallintoplace in #957
Add TimestampWithOffset canonical extension type by @serramatutu in #558
chore: bump version number by @zeroshade in #959

New Contributors

@varun0630 made their first contribution in #779
@twuebi made their first contribution in #786
@SAY-5 made their first contribution in #797
@oakad made their first contribution in #805
@paveon made their first contribution in #817
@lesam made their first contribution in #716
@happydave1 made their first contribution in #844
@qzyu999 made their first contribution in #841
@abir-eon made their first contribution in #832
@matan129 made their first contribution in #850
@nssalian made their first contribution in #859
@fallintoplace made their first contribution in #870
@mdibaiee made their first contribution in #885
@truffle-dev made their first contribution in #883
@Patzifist made their first contribution in #864
@fornwall made their first contribution in #920

Full Changelog: https://github.com/apache/arrow-go/compare/v18.6.0...v18.7.0

Apache Arrow 25.0.0 Release

2026-07-10T00:00:00-04:00

The Apache Arrow team is pleased to announce the 25.0.0 release. This release covers over 3 months of development work and includes 222 resolved issues on 268 distinct commits from 66 distinct contributors. See the Install Page to learn how to get the libraries for your platform.

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Format Notes

We clarified that variadic buffers exported over the C Data Interface could be null (GH-50255). Consumers of the C Data Interface must be ready to handle them.

Arrow Flight RPC Notes

The Flight SQL protocol was amended to let servers explicitly inform clients whether a prepared statement contains a result set or not (GH-49497).

Progress was made on the ODBC driver for Flight SQL, but we are not yet distributing packages for end users at this time.

C++ Notes

CSV

A new option default_column_type disables type inference for all columns, including those not listed in the column_types mapping (GH-47663).

Compute

A new hypot compute function calculates Euclidean norms without the avoidable overflow of a naive implementation (GH-50198).

Comparison functions have been upgraded to support StringView and BinaryView inputs (GH-49964).

Sort functions and their siblings (rank, select-k) now allow configuring per-key null placement, so as to emulate SQL constructs such as ORDER BY i NULLS FIRST, j NULLS LAST (GH-46926).

Rank functions now correctly distinguish NaNs from null values in floating-point arrays (GH-45193).

The count function now accounts for logical nulls in run-end-encoded arrays (GH-49908).

File Systems

The FileSystemFactory interface, used for dynamically-initialized filesystem implementations, now allows passing a set of key-value pairs in addition to a URI. This allows to pass sensitive initialization data, such as credentials, without leaking them in the URI (GH-50044).

IPC

We made the IPC reader stricter in a number of places, which could reject invalid IPC streams or files that would previously appear to read successfully (#49897, #50235).

Parquet

When writing a Parquet file with bloom filters enabled, bloom filters are automatically "folded" so as to match the configured fpp (the max false positive rate) according to the actual cardinality of the data used for the filter. This can provide size savings, especially with the conservative default cardinality estimate (GH-50008).

Bloom filters can be faster on some platforms thanks to vectorization (GH-50030).

It is now possible to read and write ListView data from/to Parquet (GH-50160).

Miscellaneous C++ changes

On ARM64 platforms, Arrow C++ now supports dynamically dispatching to SVE-optimized routines on compatible CPUs (GH-49756). Previously, dynamic dispatch was only supported on x86 platforms.

Runtime CPU detection now uses xsimd instead of home-grown detection functions (GH-49940).

The ChunkedArray class has a new method ComputeLogicalNullCount, mirroring the existing methods of the same name on Array and ArrayData classes (GH-50261).

The Table class has a new method ToTensor complementing the existing method of the same on the RecordBatch class (GH-41870). Both methods convert from the columnar format to a contiguous two-dimensional array.

Linux Packaging Notes

The release dropped support for Debian bookworm GH-50200 due to the distribution reaching End Of Life.

We added Reproducible Builds support for the Debian Linux Packages GH-49988.

Python Notes

Compatibility notes

Feather reader and writer is deprecated in favour of the Arrow IPC API GH-49232.

New features

hypot compute kernel is added to Arrow compute module and accessible in PyArrow GH-50197.
pa.OSFile now accepts open file descriptor (int parameter) besides the str path GH-49751.
Conversion from a list of individual numpy.ndarrays to a FixedShapeTensor is added GH-49644.
create_encryption_properties and create_decryption_properties methods are added to the parquet API using Arrow C++ FileEncryptionPropertiesBuilder and FileDecryptionPropertiesBuilder GH-47435.
Conversion of Table to Tensor has been implemented in Arrow C++ and can also be used in Python bindings GH-40062.
default_column_type option is added to csv.ConvertOptions which sets a default column type for all columns and can be combined with column_types GH-22232.

Other improvements

Extension types are supported in pyarrow.parquet.read_schema GH-48254
Default values for Parquet pre_buffer are made consistent GH-49923.

Relevant bug fixes

count compute kernel bug for sliced union arrays is fixed GH-50113.
hash_any/hash_all compute kernel bug is fixed for sliced boolean arrays GH-50043.
Table.from_pylist on ExtensionType column with list_ storage crash when values exceed int32 offsets is fixed GH-50012.
Bug causing Use-After-Free on PyList_SetItem in SparseCSFTensorToNdarray is fixed GH-49917.
Timezone drop when converting tz-aware pandas Categorical is fixed GH-49875.
_export_to_c segmentation fault for binary_view array is fixed by fixing cast kernels so all-inline view arrays do not keep a null variadic buffer slot GH-49740.
replace_with_mask crash when null type inputs are used is fixed GH-47447.
Segmentation fault when using sort_indices for temporal types is fixed GH-47252
Index level is bumped if pandas dataframe already contains __index_level_i__ column GH-46179.
Special handling for single-file paths passed to ParquetDataset constructor is restored, fixing merge error in pyarrow.parquet.read_table GH-43574.

Other

Annotations are withhold from Python wheels until they are complete GH-49831.
PyBuffer and NumPyBuffer destructors are protected against interpreter finalization GH-49942.
Documentation updates in GH-50227 and GH-20403.
Tests for regular replace_with_mask kernel usage are added GH-50072.
Hypothesis timezones test strategy now includes fixed offsets GH-31318.

R Notes

Breaking changes

Arrow uint64 types are now always converted to R double (numeric) vectors, regardless of the values. Previously, small uint64 values were converted to R integer, which could cause inconsistent types within list columns when different list elements had different value ranges (#50339).

New features

Field objects now support field-level metadata via $metadata and $with_metadata() (@max-romagnoli, #33390).
Parquet files now support list-columns of ordered factors (ordered dictionaries) (#49689).

Minor improvements and fixes

Array$create() now gives a clearer error message when given a POSIXct object with an invalid timezone (#40886).
Dictionary arrays with large_string value types now convert correctly to R factors (#39603).
open_dataset() now gives a clearer error message when providing a mix of readr and Arrow options (@Rich-T-kid, #33420).
read_parquet() no longer triggers a C++ alignment warning from the Acero source node (#46178).
Schema metadata partial matching on $metadata$r no longer errors when other metadata keys start with “r” (#50163).
to_arrow() now preserves group_by() when converting from a dbplyr lazy table (#40640).
write_parquet() now correctly validates that max_rows_per_group is a positive number (#40742).
Stale S3 connections no longer cause a segfault during garbage collection (#50009).
Spurious “Invalid metadata$r” warnings are no longer emitted when reading files with custom schema metadata (#48712).

Installation

The R package now builds under r-universe/r-wasm (#49981).

Ruby and C GLib Notes

Added fallback data type for unknown extension type: GH-49969

Ruby

Added RecordBatch#merge: GH-50175
Ensuring zero-initializing all rb_memory_view_t members for rb_memory_view_get(): GH-50234

C GLib

No C GLib only notes.

Java, JavaScript, Go, .NET, Swift and Rust Notes

The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate repositories outside the main Arrow monorepo.

For notes on the latest release of the Java implementation, see the latest Arrow Java changelog.
For notes on the latest release of the JavaScript implementation, see the latest Arrow JavaScript changelog.
For notes on the latest release of the Rust implementation see the latest Arrow Rust changelog.
For notes on the latest release of the Go implementation, see the latest Arrow Go changelog.
For notes on the latest release of the .NET implementation, see the latest Arrow .NET changelog.
For notes on the latest release of the Swift implementation, see the latest Arrow Swift changelog.

Apache Arrow Go 18.6.0 Release

2026-04-28T00:00:00-04:00

The Apache Arrow team is pleased to announce the v18.6.0 release of Apache Arrow Go. This minor release covers 46 commits from 16 distinct contributors.

Contributors

$ git shortlog -sn v18.5.2..v18.6.0
Matt Topol
Sebastiaan van Stijn
Dima Kuznetsov
Willem Jan
Alex
Alex Normand
Andrei Tserakhau
Ben Bellick
David Li
Harrison Crosse
Karen Li
Lucas Valente
junyan-ling
starpact
wjywbs
wwarner-inf

Highlights

Arrow

Features

compute package now has Sorting functions #749
Brand new array/arreflect package for round-trip reflection between Arrow and Go types/structs #771

Bug Fixes

Route QueryContext for flightsql through active transactions #692
Correctly set nullbility for Avro list type #709
Fix data race and memory leak in is_in kernel #712
Fix cdata handling colons in values #761

Performance Improvements

Improved take kernel performance (20-30% gains for 99% of cases) #702
Optimize the ARM64 NEON min/max assembly #748

Parquet

Bug Fixes

Fixed Decimal256 sign extension #711
Strip the repetition_type from the root SchemaElement during serialization #723
Normalized the element name in stored ARROW:schema #746

Performance Improvements

Avoid double bool bitmap conversion #707
Improve zstd pool memory usage (14x less memory!) #717
Optimized stats and bloom filters for boolean columns (76% less memory, more than twice as fast) #715
Vectorized bool unpack (~4x throughput) #735, #731
Eliminated per-value allocation in delta bit-pack decoder (>4x faster decoding) #730

New Contributors

@junyan-ling made their first contribution in #689
@wjywbs made their first contribution in #692
@dimakuz made their first contribution in #711
@starpact made their first contribution in #708
@laskoviymishka made their first contribution in #712
@hcrosse made their first contribution in #723
@wwarner-inf made their first contribution in #726
@alexandre-normand made their first contribution in #728
@benbellick made their first contribution in #754
@thaJeztah made their first contribution in #762
@kli19 made their first contribution in #757
@serramatutu made their first contribution in #758

Full Changelog: https://github.com/apache/arrow-go/compare/v18.5.2...v18.6.0

Apache Arrow 24.0.0 Release

2026-04-21T00:00:00-04:00

The Apache Arrow team is pleased to announce the 24.0.0 release. This release covers over 3 months of development work and includes 259 resolved issues on 325 distinct commits from 57 distinct contributors. See the Install Page to learn how to get the libraries for your platform.

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Community

We recently published our Community Highlights for 2025, check those out.

Thanks everyone for your contributions and participation in the project!

Format Notes

We have written a project-wide Security Model outlining what users should expect when dealing with Arrow data, especially coming from untrusted sources GH-48868.

Arrow Flight RPC Notes

The ODBC driver is still a work-in-progress. The driver now builds on Linux, but currently no builds are distributed (for any platform) (GH-49463).

In C++, we have refactored serialization/deserialization to make low-level functionality accessible for advanced usage (GH-49548).

C++ Notes

In addition to the aforementioned project-wide Security Model, we have written a specific Security Model for Arrow C++ covering more concrete topics such as API usage and parameter validity GH-49274.

Compute

Extension Types

The canonical type VariableShapeTensor was finally implemented GH-38007.

Parquet

Breaking change: The Arrow extension type name for Parquet Variant columns used to be parquet.variant but has been changed to arrow.parquet.variant GH-49081.

While Parquet C++ could only read unencrypted bloom filters, it now supports reading encrypted bloom filters as well GH-48334. In addition, it can also write bloom filters, though only unencrypted GH-34785.

An ambitious rewrite of the bit-unpacking utilities and optimizations has led to significant performance improvements on reading some Parquet columns, up to 50% faster in some cases GH-48277. This rewrite is described in more detail in an accompanying blog post.

The performance of reading DELTA_BINARY_PACKED-encoded integers has been improved in some favorable cases GH-49266.

Miscellaneous C++ changes

We have migrated to C++20 std::span, removing our home-grown implementation in arrow::util::span GH-48588.

A bunch of previously deprecated APIs have been removed GH-49356.

Linux Packaging Notes

Added support for Ubuntu 26.04, the next LTS GH-49341

MATLAB Notes

No major notes for this release on MATLAB.

Python Notes

Compatibility notes

pyarrow.gandiva is deprecated and will be removed in a future version GH-49227

New features

Type annotations work is starting to be included (GH-49102 and GH-49452)
Basic arithmetic on arrays and scalars is now supported GH-32007
Options to control writing of Parquet Bloom filters are added to parquet.write_table GH-49376
OpenTelemetry is enabled in PyArrow wheels GH-49382
AzureFileSystem is now included in the Windows wheels GH-44655

Other improvements

Scikit-build-core is now used as the PyArrow build system GH-36411
UUID objects are now inferred automatically in pa.scalar() and pa.array() without the need to specify the type explicitly GH-48241
Constructing an extension array via pa.array() from a list of extension-type scalars is now supported GH-48470
There have been some improvements in the documentation (GH-49278, GH-49269 and GH-28859)
CSV and JSON options have improved repr/str methods GH-47389

Relevant bug fixes

SparseCOOTensor.__repr__ missing f-string prefix is now fixed GH-49108
Pickling SubTreeFileSystem(base_path, AzureFileSystem(...)) is fixed GH-49078
Casting from StringArray to pandas 3.* when element is None is fixed GH-49002
Dictionary key order is now preserved when inferring struct type GH-40053
Duplicate csv header when table batches start with empty is now fixed GH-36889

R Notes

New Features

A number of new dplyr bindings GH-49533, GH-49256, GH-49535 and GH-49534

Compatibility notes

Arrow no longer builds with GCS enabled on CRAN to avoid failures in their build systems. If you would like a full-featured build of Arrow, we recommend installing from R-universe; see the Using cloud storage article in the docs for more information. GH-49067

Relevant bug fixes

to_arrow() now retains grouping GH-40640

Ruby and C GLib Notes

Fixed GC related problems.
GArrowListArray: Added support for returning offset buffer.
GArrowLargeListArray: Added support for returning offset buffer.
GArrowUnionArray: Added support for returning fields.
Deprecated Feather features.

Ruby

We've added pure Ruby Apache Arrow writer implementation to the red-arrow-format gem.

We've marked pure Ruby Apache Arrow reader implementation in the red-arrow-formatgem as stable because it passes integration tests with other implementations. But it still has some missing features.

The red-arrow gem:

Add support for converting to raw Ruby objects of the following arrays:
- Arrow::LargeBinaryArray
- Arrow::LargeUTF8Array
- Arrow::LargeListArray
- Arrow::FixedSizeListArray
- Arrow::DurationArray
- Arrow::DictionaryArray with Arrow::LargeBinaryArray or Arrow::LargeUTF8Array

C GLib

No C GLib only notes.

Java, JavaScript, Go, .NET, Swift and Rust Notes

The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate repositories outside the main Arrow monorepo.

For notes on the latest release of the Java implementation, see the latest Arrow Java changelog.
For notes on the latest release of the JavaScript implementation, see the latest Arrow JavaScript changelog.
For notes on the latest release of the Rust implementation see the latest Arrow Rust changelog.
For notes on the latest release of the Go implementation, see the latest Arrow Go changelog.
For notes on the latest release of the .NET implementation, see the latest Arrow .NET changelog.
For notes on the latest release of the Swift implementation, see the latest Arrow Swift changelog.

Apache Arrow ADBC 23 (Libraries) Release

2026-04-07T00:00:00-04:00

The Apache Arrow team is pleased to announce the version 23 release of the Apache Arrow ADBC libraries. This release includes 41 resolved issues from 20 distinct contributors.

This is a release of the libraries, which are at version 23. The API specification is versioned separately and is at version 1.1.0.

The subcomponents are versioned independently:

C/C++/GLib/Go/Python/Ruby: 1.11.0
C#: 0.23.0
Java: 0.23.0
R: 0.23.0
Rust: 0.23.0

The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog.

Release Highlights

A breaking change has been made to the Rust APIs (pre-1.0): returned RecordBatchReaders are now type-erased and boxed for caller flexibility; this also fixes the returned reader lifetime accidentally being tied to input argument lifetimes (#3904).

A driver manager for Node.js is now available from NPM (#4046, #4091, #4116, #4125, etc.).

The C++ and Rust driver managers now support connection profiles (#3876, #3973, #4080, #4083 etc.). (Note that other bindings that use the C++ driver manager, including GLib/Ruby, Go, Java, Python, R, and so on, inherit this support.)

The Go APIs have added interfaces that always take a context.Context for consistency, and to make sure context like telemetry traces propagate properly (#4009).

The Python driver manager has added specific parameters for using connection profiles as well (#4078, #4118). Also, non-string option values are directly accepted for convenience (#4088). adbc_get_statistics has been added (#4129).

The JNI bindings (allowing use of C/C++/Go/Rust/etc. drivers from Java) now support more functions (GetObjects, GetInfo, ExecuteSchema, etc.) (#3966, #3972, #4056).

Packages are now being uploaded to Homebrew (#4131).

Python wheels now require manylinux_2_28, up from manylinux2010, following PyArrow (#4146). On macOS, macOS 12 is now the minimum version due to upgrading to Go 1.25+ (including on conda-forge, where the packages previously pinned Go 1.24 to avoid this).

The PostgreSQL driver tries to reconcile Arrow NA arrays with PostgreSQL types when binding (#4098). Also, a bug in conversion from Arrow decimals to PostgreSQL numerics has been fixed (#3787).

The SQLite driver now enables various optional features, like math functions (#4147).

Contributors

$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-22..apache-arrow-adbc-23
David Li
Kent Wu
Matt Topol
eitsupi
Bryce Mecum
Bruce Irschick
Mandukhai Alimaa
Emil Sadek
Tornike Gurgenidze
Dewey Dunnington
Felipe Oliveira Carvalho
eric-wang-1990
Curt Hagenlocher
Ian Cook
Madhavendra Rathore
Mila Page
Pavel Agafonov
Roshan Banisetti
davidhcoe
oglego

Roadmap

Getting Involved

We welcome questions and contributions from all interested. Issues can be filed on GitHub, and questions can be directed to GitHub or the Arrow mailing lists.

Community Highlights 2025

2026-03-19T00:00:00-04:00

As you may have read in a previous blog post ¹, the Apache Arrow project recently turned 10 years old. We are grateful to everyone who helped us achieve this milestone, and we wanted to celebrate the community's accomplishments, by publishing our community highlights from 2025.

We were inspired by the research by Dr Cat Hicks et al ², who found that concrete evidence of progress and accomplishments is instrumental to motivation and collaboration in developer teams. We think the same should hold for open source.

New contributors

It has been great to see many new contributors joining the project in the past year, with over 300 such individuals observed across the main Apache Arrow language implementations.

Number of new contributors per repository.
Repository/Implementation	Number of new contributors
arrow	125
arrow-rs	132
arrow-java	28
arrow-go	35

Worth highlighting is alinaliBQ who has been very active on the C++ Flight SQL ODBC Driver work together with justing-bq.

AntoinePrv has done a huge amount of work on the C++ Parquet implementation and andishgar in the C++ Statistics area.

rmnskb got involved with PyArrow in EuroPython sprints and has contributed multiple PRs since then. On the same event paddyroddy also started with his first contribution and helped on the Python packaging side further on.

sdf-jkl, liamzwbao, friendlymatthew, and klion26 helped drive early Variant functionality in the Rust Parquet implementation and contributed a number of follow-up improvements.

jecsand838 drove major improvements to the Rust arrow-avro crate, work highlighted in the Introducing Arrow Avro blog post.

Notable New Contributors in apache/arrow for 2025.
Author	Number of prs	Number of line changes (+ and -)
alinaliBQ	36	15754
andishgar	19	2926
AntoinePrv	8	79257
rmnskb	7	550
justing-bq	4	12607

Notable New Contributors in apache/arrow-rs for 2025.
Author	Number of prs	Number of line changes (+ and -)
scovich	50	21006
jecsand838	38	26753
friendlymatthew	33	7203
sdf-jkl	4	388
rambleraptor	4	333

Notable New Contributors in apache/arrow-go for 2025.
Author	Number of prs	Number of line changes (+ and -)
Mandukhai-Alimaa	6	1392
hamilton-earthscope	5	2998

Release, Packaging and CI

A lot of work has been done around the Continuous Integration and Developer Tools area. Ensuring a project with the reach of Arrow is properly working requires validation on a huge matrix of operating systems, architectures, libraries, versions. Needless to say that maintenance work has tremendous importance for the health of the project and the positive contributor experience.

The most active contributors in the main repository are the ones contributing heavily on those areas while also providing the most review capacity. Shout out to kou and raulcd for taking such good care of the project and devoting countless hours so that everything runs smoothly.

Notable contributions worth mentioning are enhanced release automation and reproducible builds for sources, migrating remaining AppVeyor and Azure jobs to GitHub actions, improving dev experience with more pre-commit checks instead of custom made linting tools.

Moving some implementations out of the main repository (apache/arrow on GitHub) helped with easier releases and maintenance of the main repository and also of separate language implementations. The current apache/arrow repo now holds the format specification, C++ implementation together with all the bindings to it (Python, R, Ruby and C GLib). Other languages now live in their own apache/ repos namely apache/arrow-java, apache/arrow-js, apache/arrow-rs, apache/arrow-go, apache/arrow-nanoarrow, apache/arrow-dotnet and apache/arrow-swift.

Notable Contributors in apache/arrow for 2025.
Author	Number of prs	Number of line changes (+ and -)
kou	221	141015
AntoinePrv	8	79257
raulcd	110	46645
pitrou	101	36585
jbonofre	1	20061

Notable Components in apache/arrow for 2025.
Component label	Number of merged prs	Number of line changes (+ and -)
Parquet	100	103828
C++	387	82744
FlightRPC	43	52659
CI	237	42249
Ruby	74	20676

Migration of infrastructure from Voltron Data

As Voltron Data has wound down its operations in 2025, the Arrow project had to migrate benchmarking infrastructure and nightly report from Voltron-managed services to an Arrow-managed AWS account. This work has been driven by rok.

Closing of Stale issues

thisisnic was working on closing of stale issues in the apache/arrow repository which helped surfacing important issues that were overlooked or forgotten.

Code contributions

C++ implementation

Community support for maintenance and development of the Acero C++ is continuing with multiple bigger contributions in 2025 done by pitrou and zanmato1984.

Many kernels have been moved from the integrated compute module into a separate, optional package for improvement of modularity and distribution size when optional compute functionality is not being used. The work has been done by raulcd.

Arrow C++ Parquet implementation

There have been multiple contributions to fix and improve fuzzing support for Parquet. Fuzzing work is led by pitrou who is also one of the most active members of the community guiding other developers and supporting us with abundant review capacity.

Multiple newer types have also been supported in the last year, namely: VARIANT, UUID, GEOMETRY and GEOGRAPHY contributed by neilechao and paleolimbot.

An important feature added has also been Content-Defined Chunking which improves deduplication of Parquet files with mostly identical contents, by choosing data page boundaries based on actual contents rather than a number of values ³. This work has been done by kszucs.

There have been improvements in the Parquet encryption support for most of the releases in the last year. These efforts have been driven mostly by EnricoMi, pitrou, adamreeve and kapoisu.

PyArrow

A lot of work has been put into adding type annotations. It all started in July at EuroPython sprints and the code is now ready to be reviewed and merged. Some more review capacity will be needed to get this over the finish line. The work has been championed by rok.

Rust

Arrow Rust community invested heavily in the Rust parquet reader for which they created several blog posts ⁴, ⁵. The work has been championed by alamb and etseidl.

Notable Components in apache/arrow-rs for 2025.
component	merged_prs	line_changes
parquet	333	140958
arrow	436	76590
parquet-variant	125	41832
api-change	59	33938
arrow-avro	48	29487

Java

The biggest changes in apache/arrow-java for 2025 have been connected to Flight and Avro components plus Sphinx support due to the Java implementation being moved into a separate Apache repository. Contributors involved in the above are lidavidm and martin-traverse.

Go

There has been a lot of work related to new variant type in the Parquet implementation done in apache/arrow-go all by zeroshade.

Noticeable emphasis was also visible on performance-focused PRs leading to the addition of row seeking, bloom filter reading/writing, and reduction of allocations in the Parquet library along with significant optimization work in the compute.Take kernels. Shout out to pixelherodev and hamilton-earthscope for the emphasis they placed on improving performance.

Notable Components in apache/arrow-go for 2025.
component	merged_prs	line_changes
parquet	34	27056
arrow	33	14235

Nanoarrow

Bigger work in nanoarrow include Decimal32/64 and ListView/LargeListView support, LZ4 and ZSTD decompression in the IPC reader, and broader packaging via Conan, Homebrew, and vcpkg. Contributors driving most above are paleolimbot and WillAyd.

Arrow Summit 25

One last thing to highlight would be our first Arrow Summit 25 that was held in Paris in October 2025. The event was a great success and it brought users, contributors and maintainers together. It definitely was a highlight of the year for many of us. Thanks to raulcd and pitrou for organizing the event.

Thank you!

We would like to thank every single contributor to Apache Arrow for being a part of this great community and project! Hope this blog post helps to validate all the work you have done and motivates us to continue collaborating and growing together!

The Notebooks with the analysis for this blog post can be found in ⁶.

Note not all language implementations are mentioned. Some due to being moved into a separate repository in 2025 resulting in missing information for large amount of merged pull requests. Others due to having lower number of bigger contributions in the past year.