LanceDB

@lancedb

The multimodal lakehouse for AI, accelerating large-scale data curation and feature engineering so teams can build better models faster.

San Francisco, CA

lancedb.com

Replies Reposts Media

Pinned
LanceDB
@lancedb
Aug 11
Every generative model is a kind of dream, rendered from what it has seen. Introducing Reverie, LanceDB’s summit on the data systems those dreams are made on. Nov 5 in SF, with speakers from @nvidia, @runwayml, @LumaLabsAI, @AppliedInt, Cruise by @GM, @Adobe, @ExaAILabs + more.
7
LanceDB
@lancedb
Sep 22
Vector search that works at 10M vectors shouldn’t need a redesign at 10B. LanceDB hit 18.05ms p50 / 21.61ms p99 across 10B vectors, while scaling indexing and letting each query tune precision. New deep dive from @Yah01_
lancedb.com
From 10 Million to 10 Billion: Vector Search Built to Grow
LanceDB lets you tune search precision at query time, balance memory and disk caching, and keep new data searchable as your table grows.
1
LanceDB
@lancedb
Sep 21
A great blog written by @loldedxd & @ariG23498 🤗👏 huggingface.co/blog/ariG23498… funes, by @huggingface, turns past agent sessions into memory your agents can actually use. It indexes Claude Code, Codex, pi, and Hermes traces into one local Lance dataset, then gives the agent
funes: Local Memory for Coding Agents, Built on Lance
From huggingface.co
11
LanceDB
@lancedb
Sep 17
We’re starting a new series called Feature of the Week 🎉 This week’s feature by @Yah01_: Index prewarm now reads in parallel bytes windows. Loading a billion-vector index into memory used to take 95 minutes, now it takes 3.
00:00
12
LanceDB
@lancedb
Sep 16
Raw text → curated data → 2.43B tokens → trained GPT-2 in 25 minutes on 8 H100s. @loldedxd built the entire pretraining pipeline on one LanceDB table, streaming + globally shuffling directly from S3 at 3.16M tokens/sec. Full global shuffle, not a pre-shuffled copy.
lancedb.com
A Practical LLM Pretraining Pipeline with LanceDB
Training a GPT-2-like model from raw text in 25 minutes with LanceDB, using one table for curation, tokenization, shuffling, packing, training, and analysis.