Skip to content

[fix][broker] Fix delayed message index data loss when trimming overlapping bucket snapshots - #26240

Merged
lhotari merged 1 commit into
apache:masterfrom
void-ptr974:fix/bucket-delayed-trim-full-range
Jul 25, 2026
Merged

[fix][broker] Fix delayed message index data loss when trimming overlapping bucket snapshots#26240
lhotari merged 1 commit into
apache:masterfrom
void-ptr974:fix/bucket-delayed-trim-full-range

Conversation

@void-ptr974

Copy link
Copy Markdown
Contributor

Motivation

When bucket trimming selects an immutable bucket that overlaps the first active ledger, RangeMap returns a clipped range key. The delete path still deletes the complete bucket snapshot, so the delayed-delivery index cannot be recovered after a broker restart.

Modifications

  • Select only buckets whose complete immutable range ends before the first active ledger.
  • Add a regression test for a bucket that overlaps the first active ledger.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

  • Ran the bucket trimming tests covering complete deletion, delete failure, no orphaned buckets, and overlapping buckets.
  • Ran pulsar-broker checkstyleMain and checkstyleTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

@void-ptr974 void-ptr974 changed the title [fix][broker] Fix delayed-delivery index data loss when trimming overlapping buckets [fix][broker] Prevent BucketDelayedDeliveryTracker from deleting overlapping bucket snapshots Jul 25, 2026
@void-ptr974 void-ptr974 changed the title [fix][broker] Prevent BucketDelayedDeliveryTracker from deleting overlapping bucket snapshots [fix][broker] Fix deletion of overlapping bucket snapshots in BucketDelayedDeliveryTracker Jul 25, 2026
@void-ptr974 void-ptr974 changed the title [fix][broker] Fix deletion of overlapping bucket snapshots in BucketDelayedDeliveryTracker [fix][broker] Fix delayed message index data loss when trimming overlapping bucket snapshots Jul 25, 2026
@lhotari lhotari added this to the 5.0.0-M2 milestone Jul 25, 2026
@lhotari
lhotari requested review from dao-jun and nodece July 25, 2026 11:13
@lhotari
lhotari merged commit c5cb6fc into apache:master Jul 25, 2026
82 of 85 checks passed
lhotari pushed a commit that referenced this pull request Jul 25, 2026
…apping bucket snapshots (#26240)

(cherry picked from commit c5cb6fc)
lhotari pushed a commit that referenced this pull request Jul 25, 2026
…apping bucket snapshots (#26240)

(cherry picked from commit c5cb6fc)
sandeep-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 31, 2026
sandeep-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 31, 2026
sandeep-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 31, 2026
nodece pushed a commit to ascentstream/pulsar that referenced this pull request Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants