Skip to content

[fix][broker] Use effective offload policies for extra configs - #25781

Merged
coderzc merged 4 commits into
apache:masterfrom
coderzc:codex/fix-offload-extra-config-merge-2
May 18, 2026
Merged

[fix][broker] Use effective offload policies for extra configs#25781
coderzc merged 4 commits into
apache:masterfrom
coderzc:codex/fix-offload-extra-config-merge-2

Conversation

@coderzc

@coderzc coderzc commented May 15, 2026

Copy link
Copy Markdown
Member

Motivation

#25736 made broker-level managedLedgerOffloadExtraConfig* entries available to OffloadPoliciesImpl.mergeConfiguration(...). However, managedLedgerExtraConfigurations defaults to an empty map, so a namespace or topic offload policy with no extra config still stops fallback to broker-level extra config.

This can produce runtime offload policies that inherit the broker driver/bucket/region but lose broker-level extra config such as a tiered-storage bucket prefix. The namespace offloaded-ledger scanner also used raw namespace offload policies directly instead of the effective merged policy used by topic loading.

Modifications

  • Merge managedLedgerExtraConfigurations as a map from broker -> namespace -> topic.
  • Treat null or empty higher-level extra config maps as unset so broker-level entries are inherited.
  • Preserve higher-level key overrides, including an explicit empty-string value to clear a broker-level key.
  • Use effective merged namespace offload policies when scanning offloaded ledgers.
  • Add regression coverage for empty namespace-level extra config inheriting broker-level extra config and for scan using the merged policy.

Verifying this change

  • ./gradlew :pulsar-common:test --tests org.apache.pulsar.common.policies.data.OffloadPoliciesTest
  • ./gradlew :pulsar-broker:test --tests org.apache.pulsar.broker.admin.AdminApiOffloadTest.testScanOffloadedLedgersUsesMergedNamespaceOffloadPolicies

@coderzc coderzc changed the title [fix][broker] Inherit broker offload extra configs from empty policies [fix][broker] Use effective offload policies for extra configs May 15, 2026
@coderzc coderzc added type/bug The PR fixed a bug or issue reported a bug area/broker labels May 15, 2026
@coderzc coderzc self-assigned this May 15, 2026
@coderzc
coderzc requested a review from codelipenghui May 15, 2026 10:48
@coderzc
coderzc merged commit d22970d into apache:master May 18, 2026
43 checks passed
coderzc added a commit that referenced this pull request May 18, 2026
coderzc added a commit that referenced this pull request May 18, 2026
nodece pushed a commit to ascentstream/pulsar that referenced this pull request May 27, 2026
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 9, 2026
@lhotari lhotari added this to the 5.0.0-M1 milestone Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants