Eligible Leader Replicas (ELR) in Kafka 4.1: What You Need to Know

Fri, 24 Jul 2026 14:17:00 +0000
Kafka, Apache Kafka, Data Streaming

Distributed systems are full of edge cases, and Kafka is no exception - there was an extremely rare scenario where even committed messages could be lost. Motivated by this, KIP-966 introduced Eligible Leader Replicas (ELR) - a mechanism that tackles leader eligibility by separating “in-sync replicas” from “replicas that are safe to lead”. Let’s dive into it!

How Kafka Can Lose Committed Data

This blog assumes you have a basic knowledge of Kafka and understand Kafka’s durability guarantee. If not, check one of my previous blogs - What I learned about Kafka CCDAK certification - it breaks down why replication is critical in Kafka’s Architecture, and what producer parameter acks=all ensures.

KIP-966 fixes an extreme edge case that could actually violate Kafka's durability guarantees. Even with the proper configuration and the strictest producer setting, acks=all, if an unclean shutdown collides with the last replica standing issue, committed messages can still be lost. Even though the odds of hitting this exact sequence are incredibly low, the risk was high enough to enhance the Kafka failover logic. To better understand the motivation behind KIP-966, let’s check the details of the following scenarios:

Unclean shutdown scenario

When a broker shuts down gracefully, Kafka flushes its state by writing any buffered data from memory and the operating system's page cache to disk and closes all resources cleanly. On the other hand, an unclean shutdown occurs when a broker is abruptly terminated, e.g., by cutting electrical power or because of a server machine failure. In such situations, some records may exist in the Kafka operating system's page cache but may not yet be persisted to disk. Kafka doesn’t flush to disk before marking a message as committed - it would be extremely expensive - instead, Kafka does it asynchronously in batches. There are configuration options such as log.flush.interval.messages or log.flush.interval.ms, but the default recommendation is to leave them unset and allow the operating system’s background flush capabilities, as this is more efficient.

Because of that, during an unclean shutdown and after eventual recovery, the broker can come back with a shorter log than it was before the crash.

Under normal operating conditions - where the broker has other healthy, in-sync replicas (ISR) available - Kafka handles this seamlessly by promoting one of the remaining healthy, in-sync replicas (ISR) to the leader. When the crashed broker eventually rejoins the cluster, it gets automatically synced with its log segments to match the current leader's state.

Last Replica Standing issue

During an unclean shutdown, there is an extreme edge case where we can lose committed data. It’s during the Last Replica Standing (LRS) state - when the broker that suffered the unclean shutdown hosts the only replica left in the ISR. This may happen when other replicas experience, e.g., network issues, one by one, in a very short period.

You might think “When using acks=all, we would also definitely set the min.insync.replicas parameter - if we will have only 1 last replica...