The Linux Foundation Projects
Delta Lake
logo

Delta Lake Integrations

Use the following frameworks, Delta Sharing clients, managed services, and/or community integrations
for Delta Lake and Delta Sharing.

UniForm

Delta Universal Format (UniForm)
allows you to read Delta tables with Iceberg and Hudi clients

Requirements Enable UniForm

Frameworks

Use the following frameworks and languages including but not limited to Apache Flink, Apache Spark, Trino, and Rust.

Image

Apache Druid

docs source code

Druid

This connector allows Apache Druid to read from Delta Lake.

Image

Apache Flink

docs source code

Flinkstandalone

This connector allows Apache Flink to write to Delta Lake.

Image

Apache Hive

docs source code

Hivestandalone

This connector allows Apache Hive to read from Delta Lake.

Image

Apache Pulsar

docs source code

Pulsarcommunity

This connector allows Apache Pulsar to read from and write to Delta Lake.

Image

Apache Spark™

docs source code

Spark

This connector allows Apache Spark™ to read from and write to Delta Lake.

Image

ClickHouse

docs source code

ClickHouse

ClickHouse is a column-oriented database that allows users to run SQL queries on Delta Lake tables. This provides a read-only integration with existing Delta Lake tables in Amazon S3.

Image

Dagster

docs source code

DagsterPython

Use the Delta Lake IO Manager to read from and write to Delta Lake in your Dagster orchestration pipelines.

Image

Delta Rust API

docs source code

RustPython

This library allows Rust (with Python bindings) low level access to Delta tables and is intended to be used with data processing frameworks like datafusion, ballista, rust-dataframe, vega, etc.

Image

Delta Standalone

docs source code

ScalaJavastandalone

This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

Image

FINOS Legend

docs source code

FINOSLegend

An extension to the FINOS Legend framework for Apache Spark™ / Delta Lake based environment, combining best of open data standards with open source technologiesThis connector allows Trino to read from and write to Delta Lake.

Image

Hopsworks

docs source code

HopsworksPython

Hopsworks Feature Store stores, manages, and serves feature data in Delta Lake.

Image

Kafka Delta Ingest

docs source code

KafkaRust

This project builds a highly efficient daemon for streaming data through Apache Kafka into Delta Lake.

Image

PrestoDB

docs source code

PrestoDBstandalone

This connector allows PrestoDB to read from Delta Lake.

Image

RisingWave

docs source code

RisingWavestandalone

This connector allows RisingWave to write to Delta Lake.

Image

SQL Delta Import

docs source code

SQLJDBC

This utility is for importing data from a JDBC source into a Delta Lake table.

Image

StarRocks

docs source code

StarRocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. StarRocks has the ability to read from Delta Lake.

Image

Trino

docs source code

Trino

This connector allows Trino to read from and write to Delta Lake.

Sharing

Use the following clients that integrate with Delta Sharing from C++ to Rust.

Image

C++

docs source code

C++Delta Sharingcommunity

This connector allows a C++ client to read from Delta Sharing endpoint.

Image

Excel

docs

ExcelDelta Sharing

This connector allows a Excel client to read from Delta Sharing endpoint.

Image

Go

source code

GoDelta Sharingcommunity

This connector allows a Go client to read from Delta Sharing endpoint.

Image

Java

source code

JavaDelta Sharingcommunity

This connector allows a Java client to read from Delta Sharing endpoint.

Image

Kotosiro Sharing

source code

KotosiroDelta SharingRust

A Minimalistic Rust Implementation of Delta Sharing Server.

Image

MLflow

docs source code

MLflowDelta Sharingcommunity

Perform model exchange via Delta Sharing and MLflow

Image

node.js

docs source code

node.jsDelta Sharingcommunity

This connector allows node.js to read from Delta Sharing endpoint.

Image

Oracle

OracleDelta Sharing

This connector allows for Delta Sharing with Oracle Autonomous Database Data Studio.

Image

Power BI

docs

PowerBIDelta Sharing

This connector allows Power BI to read from Delta Sharing endpoint.

Image

R

source code

RDelta Sharingcommunity

This connector allows a R client to read from Delta Sharing endpoint.

Image

Rust

docs source code

RustDelta Sharingcommunity

This connector allows a Rust client to read from Delta Sharing endpoint.

Image

Terminal

source code

TerminalDelta Sharingcommunity

Terminal application for browsing Delta Sharing Metadata.

Services

Use the managed services of your choice that integrate with Delta Lake.

Image

Athena

docs

AthenaAWS

This utility allows Athena to natively read from Delta Lake starting with Athena SQL 3.0

Image

AWS EMR

docs

AWSEMR

Starting with Amazon EMR release 6.9.0, you can use Apache Spark 3.x on Amazon EMR clusters with Delta Lake tables.

Image

AWS Glue

docs

AWSGlue

AWS Glue 3.0 and later supports the Linux Foundation Delta Lake framework

Image

aws-pandas-sdk

docs source code

pandasAWSawswrangleraws-pandas-sdk

pandas on AWS - Easy integration with AWS services including optional dependency with Delta Lake

Image

Azure Stream Analytics

docs

AzureASAStream Analytics

Azure Stream Analytics provides native write support for Delta Lake

Image

BigQuery

docs

BigQuery’s native Delta Lake support enables seamless delivery of data for downstream applications.

Image

Databricks

docs

DatabricksAzureGCPAWS

Delta Lake is included within Databricks allowing it to read from and write to Delta Lake.

Image

Microsoft Fabric

docs

In order to achieve seamless data access across all compute engines in Microsoft Fabric, Delta Lake is chosen as the unified table format.

Image

Power BI

docs source code

PowerBIcommunity

This connector allows Power BI to read from Delta Lake.

Image

Redshift

docs source code

RedshiftAWSmanifest

This utility allows AWS Redshift to read from Delta Lake using a manifest file.

Image

Snowflake (Beta)

docs

Snowflake

This preview allows Snowflake to read from Delta Lake via an external table.

Image

Starburst

docs

StarburstAzureGCPAWS

The Starburst Delta Lake connector is an extended version of the Trino/Delta Lake connector with configuration and usage identical.

Image

StarTree

docs

StarTreePinot

StarTree Cloud includes the Apache Pinot / Delta Lake connector.

Community

Try out the following community integrations with Delta Lake.

Image

Apache Beam

docs source code

Beamstandalonecommunity

This connector allows Apache Beam to read from Delta Lake.

Image

Athena Query Federation (Beta)

docs source code

AWSAthenastandalonecommunity

This connector allows AWS Athena to read from Delta Lake.

Image

Beam Delta Lake

docs source code

Beamstandalonecommunity

With DataLakeIO, data from Apache Beam's pipelines can be read from and written to Delta Lake

Image

Ceph

source code

Cephcommunity

This connector allows you to read and write from Delta tables on Ceph storage.

Image

dlt | SparkR

docs source code

SparkRcommunity

This package allows SparkR to read from and write to Delta Lake.

Image

DataHub

source code

DataHubcommunity

This connector allows DataHub to extract Delta Lake metadata.

Image

Datastream Connector

source code

GCSDatastreambadal.iocommunity

As Datastream streams changes to files to Google Cloud Storage, this connector streams these files and writes the changes to Delta Lake.

Image

MinIO

docs source code

MinIOcommunity

This connector allows you to read and write from Delta tables on MinIO storage.