Skip to content

Apache Paimon Rust

The Rust implementation of Apache Paimon — a streaming data lake platform with high-speed data ingestion, changelog tracking, and efficient real-time analytics.

Overview

Apache Paimon Rust provides native Rust libraries for reading and writing Paimon tables, enabling high-performance data lake access from the Rust ecosystem.

Key features:

  • Native Rust reader for Paimon table format
  • Support for local filesystem, S3, OSS, COS, Azure, OBS, GCS, and HDFS storage backends
  • REST Catalog integration
  • Apache DataFusion integration for SQL queries

Performance

In the TPC-DS SF100 benchmark, Paimon Rust completes the 96-query workload within 9.3% of DataFusion's native Parquet reader. The gap narrows to 3.1% for queries whose native-Parquet median is at least five seconds, while Paimon's ZSTD data files use 23.25% less physical storage than the SNAPPY source files.

Read the full TPC-DS SF100 performance report

Status

The project is under active development (0.3.0 in development).

License

Apache Paimon Rust is licensed under the Apache License 2.0.