[core] Reduce memory usage of CPD's MatchCollector

[lukasgraef]

Describe the PR

When analyzing the heap dump created by running CPD against https://github.com/alsa-project/alsa-firmware/blob/master/emu/emu0404_netlist.h (mentioned in #6145) with Eclipse's Memory Analzyer, there's a clear culprit for the huge memory consumption in CPD: net.sourceforge.pmd.cpd.MatchCollector#tokenMatchSets

---

tokenMatchSets is a Map that stores lots of Set<Integer> as its values, which can take up most of the available heap space. There's a drop-in replacement for using Set<Integer> here, that takes up significantly less memory: java.util.Bitset.

When using PMD 7.17, I wasn't able to run CPD on emu0404_netlist.h with --minimum-tokens 100, even with a max heap size of 8GB.

Using my locally built SNAPSHOT with BitSet, I was able to analyze emu0404_netlist.h, while only consuming about ~750MB of memory.

Which means in this scenario, this simple replacement reduces the memory footprint of CPD to less than a tenth compared to 7.17.

Related issues

• Alleviates the memory problems described in [core] CPD throws java.lang.OutOfMemoryError: Java heap space #6145

Ready?

• Added unit tests for fixed bug/feature
• Passing all unit tests
• Complete build ./mvnw clean verify passes (checked automatically by github actions)
• Added (in-code) documentation (if needed)

[pmd-actions-helper]

Documentation Preview

No regression tested rules have been changed.

(comment created at 2025-10-29 12:01:44+00:00 for 4346bca)

[oowekyala]

I could reproduce your results on my machine. I was skeptical at first because I thought BitSet would grow too large if the integers that need to be stored are very large, but that doesn't look to be the case.

Thanks for suggesting this change! This is a great improvement

[adangel]

Thanks!

I now finished my own testing (#6145 (comment)).

BitSet seems to use slightly more memory when using a "normal" project with not so much overlapping duplicates (case 1). For the single file emu0404_netlist.h (case 2) on the other hand, it uses less, so that it passes (I can confirm your ~750MB, you probably used java 25?).
It's still not enough to be able to analyze the whole project (case 3), so we'll need more changes in this area. The dedup solution looks promising combined with BitSet.

So, overall, it's an improvement and I'm going to merge it for 7.18.0.