locale, factor, shuf: Example of deferred locale loading for clap#10911

ChrisDryden · 2026-02-13T13:51:53Z

A few month ago I attempted the same thing but now I've got a much better understanding of the project to attempt it again. The change here is that there is a parse_deferred handler now that attempts to parse the clap arguments and it it fails it will reattempt it with the locale files loaded. This combined with changes in the default locale initialization to take the util_name at the beginning from the arguments allows us to not have to load the locale file until the translation is actually required.

This requires migrating each utility to this new format, but I already started the process for factor and shuf to start.

This change is also required for the ls/stat-free-symlink gnu test and it should make the valgrind tests much more stable since it runs twice as fast in local testing and less likely to timeout.

ChrisDryden · 2026-02-13T13:59:46Z

Whoops didn't realize, uudoc calls uuapp directly, should be a simple fix to swap the names of the two uuapp functions and it should work backwards compatible. Just waiting to see the perf numbers and the gnu test results

github-actions · 2026-02-13T14:06:21Z

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Congrats! The gnu test tests/pr/bounded-memory is no longer failing!
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.
Congrats! The gnu test tests/unexpand/bounded-memory is now passing!

codspeed-hq · 2026-02-13T14:07:29Z

Merging this PR will degrade performance by 4.94%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 121 improved benchmarks
❌ 4 regressed benchmarks
✅ 158 untouched benchmarks
⏩ 38 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`wc_bytes_synthetic[500]`	194.7 µs	86.9 µs	×2.2
⚡	Simulation	`wc_chars_large_line_count[100000]`	1,023 µs	912.4 µs	+12.12%
⚡	Simulation	`wc_lines_variable_length[(50, 500)]`	3.5 ms	3.3 ms	+3.19%
⚡	Simulation	`wc_lines_extreme_line_lengths[(100000, 200)]`	1.6 ms	1.5 ms	+6.32%
⚡	Simulation	`wc_lines_large_line_count[500000]`	2.9 ms	2.8 ms	+3.94%
⚡	Simulation	`b64_decode_synthetic`	171.7 µs	60.1 µs	×2.9
⚡	Simulation	`b64_decode_ignore_garbage_synthetic`	168.7 µs	62 µs	×2.7
⚡	Simulation	`b64_encode_synthetic`	167.7 µs	58.6 µs	×2.9
⚡	Simulation	`cksum_blake3`	216.9 µs	113.3 µs	+91.43%
❌	Simulation	`sort_ascii_c_locale`	18.2 ms	18.8 ms	-3.16%
⚡	Simulation	`sort_ascii_utf8_locale`	18 ms	17.2 ms	+4.47%
⚡	Simulation	`sort_long_line[160000]`	1.4 ms	1.3 ms	+6.8%
⚡	Memory	`b64_decode_synthetic`	46.2 KB	12.1 KB	×3.8
⚡	Memory	`b64_encode_synthetic`	46.2 KB	8.6 KB	×5.4
⚡	Memory	`b64_decode_ignore_garbage_synthetic`	46.2 KB	12.1 KB	×3.8
⚡	Memory	`file_tz_abbreviations`	333.4 KB	288.4 KB	+15.59%
⚡	Memory	`complex_relative_date`	46.1 KB	16.9 KB	×2.7
⚡	Memory	`single_date_now`	46.1 KB	15.6 KB	×3
⚡	Memory	`file_custom_format`	46.2 KB	20.1 KB	×2.3
⚡	Memory	`file_iso_dates`	46.2 KB	19.5 KB	×2.4
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

_{Comparing ChrisDryden:deferred-locale-loading (7e70533) with main (dec633c)}

38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

ChrisDryden · 2026-02-13T14:21:38Z

This should also unblock us from doing translations in all of the shared libraries because there will no longer be the performance penalty hit from adding a bunch of translations into the shared locale file

github-actions · 2026-02-13T14:26:16Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/factor/t33 is no longer failing!
Note: The gnu test tests/cut/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.
Congrats! The gnu test tests/printf/printf-surprise is now passing!

sylvestre · 2026-02-15T09:55:24Z

src/uucore/src/lib/lib.rs

 ///     .help_template(localized_help_template("myutil"));
 /// ```
-pub fn localized_help_template(util_name: &str) -> clap::builder::StyledStr {
+pub fn localized_help_template(_util_name: &str) -> clap::builder::StyledStr {


maybe remove _util_name if we don't use it anymore?

sylvestre · 2026-02-15T09:57:35Z

src/uucore/src/lib/mods/clap_localization.rs

+    let name = cmd.get_name().to_string();
+
+    cmd = cmd
+        .about(crate::locale::get_message(&format!("{name}-about")))


i wonder if we follow the same standard everywhere ?

lordeji · 2026-02-15T20:14:16Z

Hi ! I've also tried to work a bit on the localization system and your defer implementation is really game changer.

I just have a few remarks :

I've observed that each uupackage have some differences in their localization system. Some manually set help and version some don't. Some use uucore::clap_localization::configure_localized_command while others manually set .help_template(uucore::localized_help_template(uucore::util_name())).
This system would need to reformat all uupackages and if you need help I can work with you for this tedious task.
In localize_command(), you're doing a no-op when no argument localization ID is found but I think that it's too much forgettable. Maybe forcing the existence of every arg ID, even when it's an empty string, would lead to an easier way to convert flt files and also making developpers know how they should format the flt file and "enforce" this without relying on documentation that people won't really read. Seems more draconian but easier on the long run.
Why using crate::locale::get_message instead of translate! ?

oech3 · 2026-02-16T07:33:44Z

Would you add dd too? It looks related with perf regression with mimalloc (removing localization fixed it).

lordeji · 2026-02-27T23:47:05Z

@ChrisDryden I'm really sorry if that's impolite, but it seems that you're not working on this and the defer feature would help me a lot and, if you're okay, I would gladly do the implementation and submit a PR.

ChrisDryden · 2026-02-27T23:48:09Z

No worries please go ahead

oech3 · 2026-02-28T00:01:23Z

ANy chance to do this with ~20 lines diff?

lordeji · 2026-02-28T00:10:43Z

@oech3 For each commit or the whole PR ?

If it's for the whole PR, that seems impossible with the whole refactor of the locale init system...

oech3 · 2026-02-28T00:13:56Z

that seems impossible with the whole refactor of the locale init system...

I see.

lordeji · 2026-03-03T20:35:33Z

@oech3 @ChrisDryden Hi, I said I would work on this feature but I was not able to work on it and would be able to work untile the 12th-15th of march.

I'm letting you know just in case someone else want to work on it.

Really sorry about it, have a nice day !

NB : @oech3 I've looked around a bit and I think that we can divide the PR in 2. The first one should defer the setup of the Localizer to the first one to call translate! and then a second PR should defer clap localization.
The first one would not change the performance because clap is still localizing but it's cleaner and the two systems are distinct.

lordeji · 2026-03-15T20:20:16Z

@oech3 @ChrisDryden I'm really sorry but my health is getting worse. I will not be able to program for an (probably very long) indefinite amount of time.
I'm therefore not working on it and the PR is left for everyone to work on.

Wishing you all the best.

oech3 · 2026-03-18T09:28:39Z

If I am undrstanding correctly, separating uu_app for uudoc and utils are very easy. OK? I might try.

ChrisDryden added 2 commits February 13, 2026 13:45

uucore: remove redundant setup_localization call in help template

0bf31e2

uucore: defer locale loading until first translated message is needed

c7f195d

shuf,factor: restore localized uu_app() for uudoc compatibility

7e70533

ChrisDryden mentioned this pull request Feb 13, 2026

uucore: remove redundant setup_localization call in help template #10908

Closed

sylvestre reviewed Feb 15, 2026

View reviewed changes

lordeji mentioned this pull request Feb 27, 2026

dedup high-cost localization setup #11147

Merged

Uh oh!

Conversation

ChrisDryden commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ChrisDryden commented Feb 13, 2026

Uh oh!

github-actions bot commented Feb 13, 2026

Uh oh!

codspeed-hq bot commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 4.94%

Performance Changes

Footnotes

Uh oh!

ChrisDryden commented Feb 13, 2026

Uh oh!

github-actions bot commented Feb 13, 2026

Uh oh!

sylvestre Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

sylvestre Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

lordeji commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oech3 commented Feb 16, 2026

Uh oh!

lordeji commented Feb 27, 2026

Uh oh!

ChrisDryden commented Feb 27, 2026

Uh oh!

oech3 commented Feb 28, 2026

Uh oh!

lordeji commented Feb 28, 2026

Uh oh!

oech3 commented Feb 28, 2026

Uh oh!

lordeji commented Mar 3, 2026

Uh oh!

lordeji commented Mar 15, 2026

Uh oh!

oech3 commented Mar 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ChrisDryden commented Feb 13, 2026 •

edited

Loading

codspeed-hq bot commented Feb 13, 2026 •

edited

Loading

lordeji commented Feb 15, 2026 •

edited

Loading