Skip to content

[improve][client] Clean up unacked message tracker when topics are removed in multi-topic consumers - #25923

Merged
lhotari merged 1 commit into
apache:masterfrom
Dream95:improve_multi-pattern-consumer-unacked-cleanup
Jun 5, 2026
Merged

[improve][client] Clean up unacked message tracker when topics are removed in multi-topic consumers#25923
lhotari merged 1 commit into
apache:masterfrom
Dream95:improve_multi-pattern-consumer-unacked-cleanup

Conversation

@Dream95

@Dream95 Dream95 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Motivation

Follow-up to [#25921]. That PR fixed incorrect unacked-message removal on explicit unsubscribe by comparing base partitioned topic names instead of substring matching. This change extends the same cleanup to other paths where a multi-topic or pattern consumer drops a topic without calling unsubscribe, so timed-out unacked entries are not left behind.

In production, topics can also disappear from a consumer when:

  • Partition metadata shrinks — onTopicsExtended closes partition consumers when a partitioned topic is deleted or loses partitions (autoUpdatePartitions).
  • Pattern subscription — PatternMultiTopicsConsumerImpl removes a grouped partitioned topic after all its partition consumers are gone.

Those paths closed partition consumers but did not clear matching entries from the unacked tracker.

Modifications

  • Extract removeTopicMessagesFromUnackedTracker(String topicName) in MultiTopicsConsumerImpl and reuse it from unsubscribeAsync .
  • After partition consumers are closed in onTopicsExtended when partition count goes to zero, call removeTopicMessagesFromUnackedTracker for the logical topic name.
  • In PatternMultiTopicsConsumerImpl, when all partitions of a grouped topic are removed, call the same helper before dropping the topic from partitionedTopics.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change added tests and can be verified as follows:

./gradlew :pulsar-client-original:test --tests "org.apache.pulsar.client.impl.MultiTopicsConsumerImplTest.testOnTopicsExtendedRemovedTopicCleansUnackedMessages"
./gradlew :pulsar-client-original:test --tests "org.apache.pulsar.client.impl.PatternMultiTopicsConsumerImplTest#testOnTopicsRemovedCleansUnackedMessagesForRemovedPartitionedTopic"

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Matching PR in forked repository

PR in forked repository: Dream95#13

…n multi-topic consumers

Signed-off-by: Dream95 <zhou_8621@163.com>
@Dream95 Dream95 changed the title [improve][client] Clean up unacked messages when topics are removed in multi-topic consumers [improve][client] Clean up unacked message tracker when topics are removed in multi-topic consumers Jun 4, 2026
@lhotari
lhotari merged commit 26cf550 into apache:master Jun 5, 2026
44 of 46 checks passed
@lhotari lhotari added this to the 5.0.0-M1 milestone Jun 5, 2026
lhotari pushed a commit that referenced this pull request Jun 5, 2026
…moved in multi-topic consumers (#25923)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 26cf550)
lhotari pushed a commit that referenced this pull request Jun 5, 2026
…moved in multi-topic consumers (#25923)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 26cf550)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 8, 2026
…moved in multi-topic consumers (apache#25923)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 26cf550)
(cherry picked from commit 8ead3bd)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 9, 2026
…moved in multi-topic consumers (apache#25923)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 26cf550)
(cherry picked from commit 8ead3bd)
@Dream95
Dream95 deleted the improve_multi-pattern-consumer-unacked-cleanup branch July 22, 2026 14:28
nodece pushed a commit to ascentstream/pulsar that referenced this pull request Aug 28, 2026
…moved in multi-topic consumers (apache#25923)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 26cf550)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants