Don't escape U+FF9E and U+FF9F in `escape_debug_ext` - #158057

Jules-Bertholet · 2026-06-18T01:51:36Z

The two characters ﾞ U+FF9E HALFWIDTH KATAKANA VOICED SOUND MARK and ﾟ U+FF9F HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK are the odd ones out when it comes to grapheme extenders, in that graphically they are not combining at all. See microsoft/terminal#18087 for more background on these characters.

I think this needs FCP? It affects char::escape_debug, and various Debug impls for characters and strings.

@rustbot labels T-libs-api needs-fcp A-Unicode

rustbot · 2026-06-18T01:51:42Z

r? @Darksonn

rustbot has assigned @Darksonn.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: libs
libs expanded to 11 candidates
Random selection from 6 candidates

Darksonn · 2026-06-18T06:29:43Z

            _ if self.is_control()
                || self.is_private_use()
                || self.is_whitespace()
                || args.escape_grapheme_extender && self.is_grapheme_extender()
                || self.is_default_ignorable()
                || self.is_format_control()
                || self.is_unassigned() =>
            {
                EscapeDebug::unicode(self)


Which category does it fall into today? What sort of other things are in that category? It seems like a really weird exception, and I can't help but wonder whether it's one of these methods that should be fixed instead.

View changes since the review

They fall under is_grapheme_extender(). https://www.unicode.org/versions/latest/core-spec/chapter-3/#G41165 is the description of this category in the Unicode standard (which explicitly special-cases these two characters). https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3AGrapheme_Extend%3A%5D is the full list of characters in the category

If I set escape_grapheme_extender, then I would expect it to escape all grapheme extenders.

I added a note to the field's doc comment.

Darksonn · 2026-06-18T06:31:22Z

Could you give an example of a string whose Debug impl changes? Perhaps add it as a test.

Jules-Bertholet · 2026-06-18T13:53:13Z

An example string that would be affected: "ﾍﾟﾝｷﾞﾝ" (Japanese "penguin" in half-width katakana) would previously debug-print as "ﾍ\u{ff9f}ﾝｷ\u{ff9e}ﾝ", but after this PR will print as itself.

Jules-Bertholet

I added a test.

View changes since this review

Darksonn

Well, with my somewhat limited but non-zero knowledge of japanese, I do agree that ﾍﾟﾝｷﾞﾝ should ideally be printed without escaping, and I also understand why it makes sense for them to be considered "part" of the current character even if they do not stack on top of the previous character.

So I'm going to review this PR with the assumption that we want to make this change now, but I do agree that an FCP would be ideal for this behavior change.

View changes since this review

Darksonn · 2026-06-19T20:22:31Z

cc @Amanieu can you help with whether FCP is needed? Thanks!

Amanieu · 2026-06-19T21:26:11Z

When in doubt, just add the label to nominate for discussion in the next libs-api meeting.

nia-e · 2026-06-23T17:05:35Z

cc @Manishearth as our Unicode Knower, if this seems reasonable to you we're okay shipping this

joshtriplett · 2026-06-23T17:06:08Z

@rfcbot merge libs-api

@rfcbot concern wait-for-manish-feedback

rust-rfcbot · 2026-06-23T17:06:24Z

@joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

~~wait-for-manish-feedback~~ resolved by Don't escape U+FF9E and U+FF9F in escape_debug_ext #158057 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

the8472 · 2026-06-23T17:06:49Z

cc @Manishearth as our Unicode Knower, if this seems reasonable to you we're okay shipping this

And also, is there something we could get from the unicode consortium to make this less adhoc?

Jules-Bertholet · 2026-06-23T18:05:03Z

And also, is there something we could get from the unicode consortium to make this less adhoc?

I doubt it. It's debug output, there's not much use in a detailed standard for that.

rust-rfcbot · 2026-07-11T15:55:08Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

Jules-Bertholet · 2026-07-30T00:43:36Z

@rustbot ready

Darksonn

@bors r+

View changes since this review

rust-bors · 2026-07-30T05:55:56Z

📋 This PR cannot be approved because it currently has the following label: S-waiting-on-fcp.

Darksonn · 2026-07-30T05:56:21Z

@bors r+

rust-bors · 2026-07-30T05:56:24Z

📌 Commit 4be2c44 has been approved by Darksonn

It is now in the queue for this repository.

@rustbot

…noescape, r=Darksonn Don't escape U+FF9E and U+FF9F in `escape_debug_ext` The two characters `ﾞ` U+FF9E HALFWIDTH KATAKANA VOICED SOUND MARK and `ﾟ` U+FF9F HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK are the odd ones out when it comes to grapheme extenders, in that *graphically* they are not combining at all. See microsoft/terminal#18087 for more background on these characters. I think this needs FCP? It affects `char::escape_debug`, and various `Debug` impls for characters and strings. @rustbot labels T-libs-api needs-fcp A-Unicode

@rustbot

…noescape, r=Darksonn Don't escape U+FF9E and U+FF9F in `escape_debug_ext` The two characters `ﾞ` U+FF9E HALFWIDTH KATAKANA VOICED SOUND MARK and `ﾟ` U+FF9F HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK are the odd ones out when it comes to grapheme extenders, in that *graphically* they are not combining at all. See microsoft/terminal#18087 for more background on these characters. I think this needs FCP? It affects `char::escape_debug`, and various `Debug` impls for characters and strings. @rustbot labels T-libs-api needs-fcp A-Unicode

…uwer Rollup of 8 pull requests Successful merges: - #159817 (Rename splat to avoid stable name collisions) - #158057 (Don't escape U+FF9E and U+FF9F in `escape_debug_ext`) - #159818 (Resolve vars before calling `unnormalized_obligations`) - #160040 (Split function parsing out of `item.rs` to a new module.) - #160044 (Add regression tests for fixed dead-code issues) - #160147 (tests: Remove `-Zthreads` options from tests in `ui/parallel-rustc`) - #160175 (Try to recover less from incorrectly parsed const arg) - #160181 (Mark `Tuple` and `FnPtr` traits `#[fundamental]`)

@rustbot

…noescape, r=Darksonn Don't escape U+FF9E and U+FF9F in `escape_debug_ext` The two characters `ﾞ` U+FF9E HALFWIDTH KATAKANA VOICED SOUND MARK and `ﾟ` U+FF9F HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK are the odd ones out when it comes to grapheme extenders, in that *graphically* they are not combining at all. See microsoft/terminal#18087 for more background on these characters. I think this needs FCP? It affects `char::escape_debug`, and various `Debug` impls for characters and strings. @rustbot labels T-libs-api needs-fcp A-Unicode

Rollup of 14 pull requests Successful merges: - #159817 (Rename splat to avoid stable name collisions) - #160204 (Sync from portable simd 2026 07 30) - #150885 (Revive L4Re target) - #158057 (Don't escape U+FF9E and U+FF9F in `escape_debug_ext`) - #160125 (Fix typing mode handling in transmute checks and rustc_dump_layout) - #160152 (Create on-demand CI job for testing EC2 instances) - #159214 (std: improve the documentation of the random feature) - #159818 (Resolve vars before calling `unnormalized_obligations`) - #160040 (Split function parsing out of `item.rs` to a new module.) - #160044 (Add regression tests for fixed dead-code issues) - #160144 (renovate: group lockfiles PRs) - #160149 (Fix Windows on Arm PAC default) - #160175 (Try to recover less from incorrectly parsed const arg) - #160181 (Mark `Tuple` and `FnPtr` traits `#[fundamental]`)

…uwer Rollup of 25 pull requests Successful merges: - #160204 (Sync from portable simd 2026 07 30) - #138230 (Add `raw_borrows_via_references` lint) - #158057 (Don't escape U+FF9E and U+FF9F in `escape_debug_ext`) - #160015 (refactor(mir-transform): Merge `can_be_overridden`, `is_required` and `is_enabled` into one) - #160031 (std: make positioned I/O unsupported on VxWorks) - #160125 (Fix typing mode handling in transmute checks and rustc_dump_layout) - #160152 (Create on-demand CI job for testing EC2 instances) - #160232 (rustdoc: fix ICE when a grapheme cluster joins a Prepend-class character to `_` or `:`) - #159214 (std: improve the documentation of the random feature) - #159818 (Resolve vars before calling `unnormalized_obligations`) - #159955 (Stop using higher-order macros to declare arenas) - #159958 (Fix avoid cycle for self referential return type notation) - #160040 (Split function parsing out of `item.rs` to a new module.) - #160044 (Add regression tests for fixed dead-code issues) - #160144 (renovate: group lockfiles PRs) - #160149 (Fix Windows on Arm PAC default) - #160164 (Derive `GenericTypeVisitable` for `RegionConstraint`) - #160175 (Try to recover less from incorrectly parsed const arg) - #160177 (A few more "predicate"-to-"clause" renamings) - #160181 (Mark `Tuple` and `FnPtr` traits `#[fundamental]`) - #160192 (Fix ICE for parsing issue with a closing brace) - #160209 (bootstrap: Remove method `Subcommand::kind`) - #160221 (Remove `Copy` supertrait from `VaList`) - #160223 (interpret: rename validate_operand → validate_place) - #160234 (Always use short ty path for call with missing arguments suggestion)

@rustbot

Rollup merge of #158057 - Jules-Bertholet:halfwidth-dakuten-noescape, r=Darksonn Don't escape U+FF9E and U+FF9F in `escape_debug_ext` The two characters `ﾞ` U+FF9E HALFWIDTH KATAKANA VOICED SOUND MARK and `ﾟ` U+FF9F HALFWIDTH KATAKANA SEMI-VOICED SOUND MARK are the odd ones out when it comes to grapheme extenders, in that *graphically* they are not combining at all. See microsoft/terminal#18087 for more background on these characters. I think this needs FCP? It affects `char::escape_debug`, and various `Debug` impls for characters and strings. @rustbot labels T-libs-api needs-fcp A-Unicode

…uwer Rollup of 25 pull requests Successful merges: - rust-lang/rust#160204 (Sync from portable simd 2026 07 30) - rust-lang/rust#138230 (Add `raw_borrows_via_references` lint) - rust-lang/rust#158057 (Don't escape U+FF9E and U+FF9F in `escape_debug_ext`) - rust-lang/rust#160015 (refactor(mir-transform): Merge `can_be_overridden`, `is_required` and `is_enabled` into one) - rust-lang/rust#160031 (std: make positioned I/O unsupported on VxWorks) - rust-lang/rust#160125 (Fix typing mode handling in transmute checks and rustc_dump_layout) - rust-lang/rust#160152 (Create on-demand CI job for testing EC2 instances) - rust-lang/rust#160232 (rustdoc: fix ICE when a grapheme cluster joins a Prepend-class character to `_` or `:`) - rust-lang/rust#159214 (std: improve the documentation of the random feature) - rust-lang/rust#159818 (Resolve vars before calling `unnormalized_obligations`) - rust-lang/rust#159955 (Stop using higher-order macros to declare arenas) - rust-lang/rust#159958 (Fix avoid cycle for self referential return type notation) - rust-lang/rust#160040 (Split function parsing out of `item.rs` to a new module.) - rust-lang/rust#160044 (Add regression tests for fixed dead-code issues) - rust-lang/rust#160144 (renovate: group lockfiles PRs) - rust-lang/rust#160149 (Fix Windows on Arm PAC default) - rust-lang/rust#160164 (Derive `GenericTypeVisitable` for `RegionConstraint`) - rust-lang/rust#160175 (Try to recover less from incorrectly parsed const arg) - rust-lang/rust#160177 (A few more "predicate"-to-"clause" renamings) - rust-lang/rust#160181 (Mark `Tuple` and `FnPtr` traits `#[fundamental]`) - rust-lang/rust#160192 (Fix ICE for parsing issue with a closing brace) - rust-lang/rust#160209 (bootstrap: Remove method `Subcommand::kind`) - rust-lang/rust#160221 (Remove `Copy` supertrait from `VaList`) - rust-lang/rust#160223 (interpret: rename validate_operand → validate_place) - rust-lang/rust#160234 (Always use short ty path for call with missing arguments suggestion)

Don't escape U+FF9E and U+FF9F in escape_debug_ext

e3622c1

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jun 18, 2026

rustbot assigned Darksonn Jun 18, 2026

rustbot added A-Unicode Area: Unicode needs-fcp This change is insta-stable, or significant enough to need a team FCP to proceed. T-libs-api [DEPRECATED; DO NOT USE] labels Jun 18, 2026

Darksonn reviewed Jun 18, 2026

View reviewed changes

Darksonn added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 18, 2026

Jules-Bertholet added 2 commits June 18, 2026 23:28

Add note to doc comment of escape_grapheme_extender field

f3d6ac3

Add tests

903bac5

Jules-Bertholet commented Jun 19, 2026

View reviewed changes

Darksonn reviewed Jun 19, 2026

View reviewed changes

Comment thread library/core/src/char/methods.rs

Add link to additional background info

4be2c44

Amanieu added the I-libs-api-nominated [DEPRECATED; DO NOT USE] label Jun 19, 2026

Jules-Bertholet mentioned this pull request Jun 23, 2026

Escape additional classes of characters in escape_debug_ext #158286

Closed

rust-rfcbot removed the final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. label Jul 11, 2026

rust-rfcbot added the to-announce Announce this issue on triage meeting label Jul 11, 2026

rustbot mentioned this pull request Jul 11, 2026

Tracking issue for release notes of #158057: Don't escape U+FF9E and U+FF9F in escape_debug_ext #159139

Open

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 30, 2026

Darksonn reviewed Jul 30, 2026

View reviewed changes

Darksonn removed the S-waiting-on-fcp Status: PR is in FCP and is awaiting for FCP to complete. label Jul 30, 2026

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 30, 2026

jhpratt mentioned this pull request Jul 30, 2026

Rollup of 7 pull requests #160199

Closed

JonathanBrouwer mentioned this pull request Jul 30, 2026

Rollup of 8 pull requests #160207

Closed

jhpratt mentioned this pull request Jul 30, 2026

Rollup of 14 pull requests #160222

Closed

JonathanBrouwer mentioned this pull request Jul 30, 2026

Rollup of 25 pull requests #160238

Merged

rust-bors Bot merged commit 639d011 into rust-lang:main Jul 31, 2026
13 checks passed

rustbot added this to the 1.99.0 milestone Jul 31, 2026

Jules-Bertholet deleted the halfwidth-dakuten-noescape branch July 31, 2026 12:07

Uh oh!

Uh oh!

Conversation

Jules-Bertholet commented Jun 18, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jun 18, 2026

Uh oh!

Darksonn Jun 18, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jules-Bertholet Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Darksonn Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

Jules-Bertholet Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

Darksonn commented Jun 18, 2026

Uh oh!

Jules-Bertholet commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jules-Bertholet left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Darksonn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Darksonn commented Jun 19, 2026

Uh oh!

Amanieu commented Jun 19, 2026

Uh oh!

nia-e commented Jun 23, 2026

Uh oh!

joshtriplett commented Jun 23, 2026

Uh oh!

rust-rfcbot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

the8472 commented Jun 23, 2026

Uh oh!

Jules-Bertholet commented Jun 23, 2026

Uh oh!

rust-rfcbot commented Jul 11, 2026

Uh oh!

Jules-Bertholet commented Jul 30, 2026

Uh oh!

Darksonn left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rust-bors Bot commented Jul 30, 2026

Uh oh!

Darksonn commented Jul 30, 2026

Uh oh!

rust-bors Bot commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Jules-Bertholet commented Jun 18, 2026 •

edited by rustbot

Loading

Darksonn Jun 18, 2026 •

edited by rustbot

Loading

Jules-Bertholet Jun 18, 2026 •

edited

Loading

Jules-Bertholet commented Jun 18, 2026 •

edited

Loading

Jules-Bertholet left a comment •

edited by rustbot

Loading

Darksonn left a comment •

edited

Loading

rust-rfcbot commented Jun 23, 2026 •

edited

Loading

Darksonn left a comment •

edited by rustbot

Loading