patch-from in single-thread mode performs worse than multithreaded

[RubenKelevra]

Describe the bug
The man page states:

Note: for all levels, you can use --patch-from in --single-thread mode to improve compression ratio at the cost of speed.

I tried that. This is not true.

To Reproduce
Steps to reproduce the behavior:

1. use --patch-from with -T8
2. use --patch-from with --single-thread
3. see --single-thread perform worse

Expected behavior
--single-thread should not perform worse.

Screenshots and charts

❯ zstd --long -T8 --progress --ultra -22 --patch-from archlinux-2024.06.01-x86_64.iso archlinux-2024.07.01-x86_64.iso
archlinux-2024.07.01-x86_64.iso : 25.03%   (  1.09 GiB =>    280 MiB, archlinux-2024.07.01-x86_64.iso.zst) 

5m 53s
❯ zstd --long --single-thread --progress --ultra -22 --patch-from archlinux-2024.06.01-x86_64.iso archlinux-2024.07.01-x86_64.iso 
archlinux-2024.07.01-x86_64.iso : 46.21%   (  1.09 GiB =>    517 MiB, archlinux-2024.07.01-x86_64.iso.zst) 

7m 9s

Desktop (please complete the following information):

• OS: CachyOS
• Version 1.5.6
• Other relevant hardware specs: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz

[Cyan4973]

I confirm the effect, measuring it on the exact same set of samples.

Understanding why it happens will be a bit tricky.

To begin with, I'm not sure why the doc states that --single-thread is always better.
It might have been true at the time the doc was written, but the code base has evolved since, and nowadays I would not expect a large difference in favor of --single-thread.

For example, it could be that some improvement brought later to the --long or --patch-from mode happens to be reliant on the MT code path, because it is more convenient to bundle several operations into a processing pipeline (I have #2341 in mind typically).

Anyway, the reason is not clear at this point.
The current mitigation is pretty simple (do not use --single-thread), and we should probably consider removing mentions of --single-thread superiority on short term from the doc since it doesn't seem to be true anymore (I note it seems to remain true for lower compression levels <= 15, though the benefit is small. Then, starting level 16, --single-thread becomes detrimental, and coincidentally, it is also the level at which #2341 starts to become active).

[RubenKelevra]

and we should probably consider removing mentions of --single-thread superiority on short term from the doc

PR attached :)

#4094

[RubenKelevra]

Changing headline, due to #4094 was merged, so it's now no longer unexpected.

[RubenKelevra]

@Cyan4973 technically the issue is not solved, just the documentation is updated, so users don't expect the single-thread performance to be better than the multi-thread performance.

But it still breaks the user expectations, as the single thread performance is so much worse than the multi-thread performance.

[Cyan4973]

There is not even the beginning of a plan on how to achieve compression parity, let alone make the single-thread mode compress better.
This is not something that is expected to change in any foreseeable future.
So the best we can do at this stage is document.

[RubenKelevra]

Ah okay. I thought it makes sense to keep this open as a tracking issue.

[Cyan4973]

It would make sense if there was at least a chance for this status to evolve in the future.
But at this point, it seems not.