Log inSign up
user avatar

Nathan VanBenschoten

@natevanben
chief architect @turbopuffer, previously distributed systems @CockroachDB
New York, NY
Joined April 2013
131
Following
1,043
Followers
RepliesMedia
  • user avatar
    Nathan VanBenschoten
    @natevanben
    Jan 22
    We've been pushing search to uncomfortable scales @turbopuffer: 100B vector indexes To get there, we had to rethink how search algos map onto hardware. The key insight was balancing bandwidth demands across the memory hierarchy I wrote about the approach and the math behind it
  • user avatar
    Nathan VanBenschoten
    @natevanben
    Jul 4, 2025
    Compare-and-swap has an infinite consensus number. It’s universal – powerful enough to solve any consensus problem. That’s why I’m excited about what we built here. It’s compare-and-swap all the way down.
  • user avatar
    Nathan VanBenschoten
    @natevanben
    Apr 1, 2025
    Week one hacking on search @turbopuffer and I’m shocked: the write:read ratio is way higher than OLTP. Ingesting and indexing a constant stream of massive batches is heavy on writes. Reads are less frequent—they just need to be fast.
  • user avatar
    Nathan VanBenschoten
    @natevanben
    Jan 6, 2025
    Sign you’ve spent too much time recently with weak iso levels: when trying to recall why consensus requires two durable writes, the prepare phase of paxos starts to look suspiciously like an optimistic, quorum-replicated SELECT FOR UPDATE lock to avoid lost updates (break safety)
  • user avatar
    Nathan VanBenschoten
    @natevanben
    Mar 13, 2022
    Great thread. Geo latency necessitates a reimagining of infrastructure. Compromising on survivability/correctness of state isn't the answer. My team @CockroachDB spent 2 years addressing this head-on by letting apps express access locality in their schema cockroachlabs.com/blog/deploy-a-…
    user avatar
    Pekka Enberg
    @penberg
    Mar 13, 2022
    Latency is increasingly important across variety of use cases. For example, in e-commerce, going beyond single digit millisecond latencies is believed to hurt revenue. But how is this need for speed driving infrastructure changes today? Here's one answer. 🧵 1/