IndexLake is an experimental table format with extensible index and inline table support.
- Extensible Index System: Pluggable index types (BM25, B-Tree, HNSW, RaBitQ, R*-tree)
- Inline Tables: Store small datasets directly within the catalog
- ACID Transaction: Support transaction through sql catalog
- Flexible Catalog: PostgreSQL and SQLite catalog support
- Flexible Storage: Local filesystem and S3-compatible storage backends
- DataFusion Integration: Native Apache DataFusion support for SQL query workloads
IndexLake
├── Core Library (indexlake) # Core table format and APIs
├── Catalogs
│ ├── PostgreSQL # Postgres catalog
│ └── SQLite # SQLite catalog
├── Indexes
│ ├── BM25 # Full-text search index
│ ├── B-Tree # Traditional B-Tree index
│ ├── HNSW # Hierarchical navigable small world graph
│ ├── RaBitQ # Randomized binary and quantization index
│ └── R*-tree # Spatial index for multidimensional data
├── Storages
│ ├── FS # Local filesystem storage
│ └── S3 # S3-compatible object storage
└── Integrations
└── DataFusion # Apache DataFusion SQL engine integration
