Tracking Issue for ascii::Char (ACP 179)

Feature gate: #![feature(ascii_char)] #![feature(ascii_char_variants)]

This is a tracking issue for the ascii::Char type from rust-lang/libs-team#179

https://doc.rust-lang.org/nightly/std/ascii/enum.Char.html

Public API

// core::ascii

#[repr(u8)]
enum Char {
    Null = 0,
    …
    Tilde = 127,
}

impl Debug for Char { … }
impl Display for Char { … }
impl Default for Char { ... }

impl Step for Char { ... } // so `Range<Char>` is an Iterator

impl Char {
    const fn from_u8(x: u8) -> Option<Self>;
    const unsafe fn from_u8_unchecked(x: u8) -> Self;
    const fn digit(d: u8) -> Option<Self>;
    const unsafe fn digit_unchecked(d: u8) -> Self;
    const fn as_u8(self) -> u8;
    const fn as_char(self) -> char;
    const fn as_str(&self) -> &str;
}

impl [Char] {
    const fn as_str(&self) -> &str;
    const fn as_bytes(&self) -> &[u8];
}

impl From<Char> for u8 { … }
impl From<Char> for char { … }
impl From<&[Char]> for &str { … }

// core::array

impl<const N: usize> [u8; N] {
    const fn as_ascii(&self) -> Option<&[ascii::Char; N]>;
    const unsafe fn as_ascii_unchecked(&self) -> &[ascii::Char; N];
}

// core::char

impl char {
    const fn as_ascii(&self) -> Option<ascii::Char>;
}

// core::num

impl u8 {
    const fn as_ascii(&self) -> Option<ascii::Char>;
}

// core::slice

impl [u8] {
    const fn as_ascii(&self) -> Option<&[ascii::Char]>;
    const unsafe fn as_ascii_unchecked(&self) -> &[ascii::Char];
}

// core::str

impl str {
    const fn as_ascii(&self) -> Option<&[ascii::Char]>;
}

// alloc::String

impl FromIterator<core::ascii::Char> for String { ... }
impl<'a> FromIterator<&'a core::ascii::Char> for String { ... }
impl<'a> FromIterator<core::ascii::Char> for Cow<'a, str> {

Steps / History

Unresolved Questions

What should it be named? Code mixing char and Char might be too confusing.
How should its Debug impl work?
- make Debug impl for ascii::Char match that of char #115434
Is allowing as-casting from it a good or bad feature?
- FWIW, there's no char::to_u32, just as u32 for it.
Some of the as_ascii methods take &self for consistency with is_ascii. Should they take self instead where possible, as the usually-better option, or stick with &self for the consistency?

https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

I guess I'll kick off the discussion about how to actually define the ascii::Char type. Right now, it's an enum like this:

#[repr(u8)]
enum Char {
    Null = 0,
    …
    Tilde = 127,
}

I'd like to first make sure I understand the pros of using this representation. Do I have them right?

It provides a niche optimization such that Option<ascii::Char> is the same size as ascii::Char.
It provides a way to write ch as u8 where ch has type ascii::Char.
For "free," you can get cute and write the names of ASCII characters instead of their visual or numerical representation. (I don't mean to trivialize this by calling it "cute." I think it's a nice benefit, especially for non-printable characters.)

Are there more? Am I mistaken about the above?

And now, the mitigations:

I think the niche optimization can still be obtained in this case because std can use unstable features? And I think there's a way to say, "make this specific value the niche" without any other additional cost.
We can provide a ascii::Char::as_u8() method. (Which is already done in the implementation PR.) Is there anything else that being able to write ch as u8 buys us?
I don't think there is a mitigation here. It feels like a "nice to have" aspect of using an enum that we probably wouldn't go out of our way to re-create via other means.

In terms of negatives, why not do the enum in the first place? Good question. I'm not entirely sure. It feels to me like it stresses my weirdness budget. It's also a generally much bigger enum than most, and I wonder whether that will provoke any surprises in places. Maybe not.

Do we need both u8::as_ascii() and ascii::Char::from_u8()?

Rollup merge of rust-lang#111009 - scottmcm:ascii-char, r=BurntSushi

@kupiakos

Regarding the enum, @kupiakos said the following in

The names or properties of ASCII characters will never change and there aren't many of them, and so we might as well expose them as variants that can participate in pattern matching.

And mentioned icu4x's enum,

https://github.com/unicode-org/icu4x/blob/b6c4018a736e79790898c5b91ff3ab25d33192c2/utils/tinystr/src/asciibyte.rs#L8-L11

Though given those names it's possible it's just doing that for the niche and not for direct use.

(Man I wish we had private variants as a possibility.)

To your questions, @BurntSushi ,

Yeah, there's at least three other possibilites here: a) put the enum in a newtype (like ptr::Alignment) to still get the niches without exposing the variants b) use the rustc-internal magic c) just don't get a niche for now since that's not needed for its core scenario of convertibility to UTF-8 (though it's certainly nice to have)
Given that we're overall not fond of as casts, and it also allows weirder things like as i64, I actually wish the cast could be disabled, and have people use as_u8 as you mention. But that might also just happen by deprecating or warning on these casts, and eventually having ascii::Char: AsRepr<Repr = u8> does seem reasonable.
Now that Refactor core::char::EscapeDefault and co. structures #105076 has landed, I've been thinking of redoing the internals of those using ascii::Char to try out how it feels using the names. That would mean code like

            '\0' => EscapeDebug::backslash(Null),
            '\r' => EscapeDebug::backslash(CarriageReturn),
            '\n' => EscapeDebug::backslash(LineFeed ),

because without the variants it ends up being more like

            '\0' => EscapeDebug::backslash(b'\0'.as_ascii().unwrap()),
            '\r' => EscapeDebug::backslash(b'\r'.as_ascii().unwrap()),
            '\n' => EscapeDebug::backslash(b'\n'.as_ascii().unwrap()),

which is nice in that it keeps the escape sequences lining up, but having to copy-paste .as_ascii().unwrap() all over like that doesn't fill me with joy.

I guess this is yet another situation in which I'd like custom literals. Or something like a'\n', I guess, but that's way too far a jump to propose right now.

EDIT a while later: The PR to update EscapeDebug like that is #111524

Rollup merge of rust-lang#111222 - scottmcm:constify-is_ascii, r=thomcc

Auto merge of rust-lang#111222 - scottmcm:constify-is_ascii, r=thomcc

I wasn't in the discussion when this type was created (which I fully support), but is there a reason to explicitly use an enum instead of a newtype + internal niche attribute like NonZeroU8?

If we decide to go the pattern types route (#107606) then NonZeroU8 could become u8 is 1.. and this could become u8 as ..128. Alternatively, if we go the generic integers route (#2581) then this could become uint<7> or just u7.

IMHO, explicitly going with an enum kind of prevents these kinds of optimisations later down the road, since it forces an enum as the representation.

@clarfonthey

@clarfonthey I think the discussion above answers your question about why we went with the enum for now. In summary, I think its principle upside is that it gives nice names to each character. Not particularly compelling, but nice.

My general feeling is that nobody is stuck on using an enum here. My best guess is that unless something compelling comes up, I wouldn't be surprised if we changed to an opaque representation such that we could change to an enum later in a compatible manner. (I think that's possible?) Purely because it's the conservative route.

I don't grok the second two paragraphs of your comment. Probably because I don't know what "pattern types" are. I also don't know what uint<7> means. (Don't have time to follow your links at the moment.)

I mean, having nice names is still achievable with constants, and this still would make match expressions work as you'd expect. Personally, the benefits I'm seeing here are the ability to have more-performant code without the need for extra unsafe code, since any slice of ASCII characters is automatically valid UTF-8. Otherwise, a type invariant would mostly be unnecessary, and you could just have constants for the names.

Explaining the two proposals:

Pattern types just allow you to write T is pat which is a new type that means T, but only values which match pat. So, Option<T> is Some(_) means "options which are some" and u8 is 1.. is effectively NonZeroU8.
Generic integers is a separate proposal which "converts" all the existing uN and iN types into aliases for a more generic uint<N> and int<N> type, which has the benefit of allowing non-power-of-two sizes and sizes larger than 128. The idea behind uint<7> is that it would be equivalent to u8 internally with the first bit always zero, although still allow all the normal operations on integers like pattern-matching and be castable to u8.

The pattern type version is very enticing because it very naturally allows exhaustive matching, while still just being a compile-time constraint. And like I said, we could still have constants for the different names for characters without really affecting things.

having nice names is still achievable

Yes, as I said:

I don't think there is a mitigation here. It feels like a "nice to have" aspect of using an enum that we probably wouldn't go out of our way to re-create via other means.

I don't know what the stabilization story is for pattern types or generic integers, but I would prefer we focus on finding a path to stabilization that doesn't require being blocked on hypothetical language features. Or at least, focusing discussion on whether we can utilize them later in a backwards compatible fashion.

I still don't really know what pattern types are (I just don't have time right now to grok it), but generic integer types seem like an implementation detail that's consistent with defining ascii::Char as an opaque type?

Right, the point here is that any opaque type would be equivalent to the primary hypothetical future implementations. So, explicitly not blocking on any particular path forward, but making sure that they're compatible.

I mean, having nice names is still achievable with constants, and this still would make match expressions work as you'd expect.

Though you can use a variant, but can't use an associated constant. Thus use ascii::Char::*; works with an enum, but not with constants for the names.

I agree that this could absolutely be done with the magic attributes. But is there anything in particular that you think is better with that approach?

The enum version still gets the niche, notably:

[src/main.rs:3] std::alloc::Layout::new::<Option<Option<Option<Option<Option<std::ascii::Char>>>>>>() = Layout {
    size: 1,
    align: 1 (1 << 0),
}

I will admit that the lack of an ability to use associated constants is an immediate downgrade, although I do think that'll eventually be allowed to eventually deprecate the std::fN::consts modules.

As far as benefits -- the main ones I see are the ability to unify this type with any one of the potential future features I mentioned. If we stabilise an enum now, we are explicitly locked out of this.

That said, there is precedent for separating out the char type from the other integer types, and I guess that ascii::Char fits right in with that. I'm just not convinced that locking in this as an enum is worth losing the benefits we could get later, especially considering how it could be converted from an opaque type into an enum publicly later if we eventually decide that is better than the alternatives.

Oh, I thought the standard library can use unstable features? I agree that waiting for stabilization of pattern types is not a good idea!

EDIT: see for example how NonNull is defined in the standard library:

rust/library/core/src/ptr/non_null.rs

Lines 79 to 81 in c36f145

    
           pub struct NonNull<T: PointeeSized> { 
        
               pointer: crate::pattern_type!(*const T is !null), 
        
           }

I would expect the main concern to be exposing pattern types to users for ergonomics, not for internal usage.

If we don't use an enum, I'd expect we'll just have #[repr(transparent)] pub struct Char(pattern_type!(u8 is 0..0x80)); which should be stabilizable like that regardless of pattern types being stable or not.

which should be stabilizable like that regardless of pattern types being stable or not.

I don't think so, because you'd be able to match on the inner type and not have to provide 0x80..=0xFF. That would be an implementation detail if pattern types aren't stable.

which should be stabilizable like that regardless of pattern types being stable or not.

I don't think so, because you'd be able to match on the inner type and not have to provide 0x80..=0xFF. That would be an implementation detail if pattern types aren't stable.

the inner type is private so you can't directly access it.

you can't exhaustively match on NonZeroU8 which also uses pattern types internally: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=d82372d2d6ba9174944aefcd212d4919

I don't see how that could be a problem for ascii::Char and not also a problem for NonZeroU8

Ah, I misread and thought it was a pub field. Yes, if it's internal, no concerns.

Would switching from a new-type wrapper around an enum to any of the other proposed representations be a breaking change? If not, such a newtype could be stabilized while keeping the choice of representation open.

+1 for making this a newtype with a private field. Unless I missed any others, I've seen only two enum reasons mentioned in this thread that don't also apply to newtypes:

support for as casts; .as_u8() and various From impls are probably fine, however, and would be wanted anyway
you can't use an associated const (still true, afaik); but it's simple enough to declare your own constant if you really need to reference a specific character by name this way:

const COMMA: core::ascii::Char = core::ascii::Char::COMMA;

Using a newtype leaves the internal representation open to change or optimization or experimentation without being a breaking change later, while still allowing for enum's other niceties now, including:

named constants for each character (via associated const)
niche optimization (via an internal enum, pattern type, or whatever else we want to try)

There's perhaps the relative ease of writing Space = 0x20, vs const SPACE: Self = Self(...);, but that's sortof what macros are for.

Did I miss any other points that would make stabilizing on a public exhaustive enum a better option than a newtype wrapper w/ private innards?

I've seen only two enum reasons mentioned in this thread that don't also apply to newtypes

Exhaustive matching has been brought up multiple times as a desirable feature. This is something currently possible (ignoring range patterns) with the enum variants but presumably would not be possible with a private field.

I don't see how that could be a problem for ascii::Char and not also a problem for NonZeroU8

Note that NonZeroU8 is a relatively "homogeneous" type, in the sense that there are few operations that need to handle different values in different ways, especially few with many branches. On the other hand ascii::Char has many natural "subclasses" that can show up in matches (as evidenced by the various methods on it returning bool: is_uppercase(), is_digit(), etc).

Exhaustive matching has been brought up multiple times as a desirable feature.

Perhaps for initial stabilization it's fine to drop this. It could be brought back later in a backwards compatible way. You can also just use unreachable!() (or a [T; 128] table as the godbolt examples above showed) and the compiler will know that this cannot panic.

On the other hand ascii::Char has many natural "subclasses" that can show up in matches

This is an argument in support of range matching, right?

@ronnodas

Exactly what @ronnodas said.

This is an argument in support of range matching, right?

Maybe, but pattern types don't have a monopoly on range matching. I think it should be allowed on C-like enums regardless.

Exhaustive matching has been brought up multiple times as a desirable feature.

Perhaps for initial stabilization it's fine to drop this. It could be brought back later in a backwards compatible way.

I agree that exhaustive matching does not need to be part of the initial stabilization, but it should be backwards compatible with it. I believe changing a struct (even one with all private fields) to an enum is breaking.

On the other hand ascii::Char has many natural "subclasses" that can show up in matches

This is an argument in support of range matching, right?

Indirectly yes, but range matching can currently be emulated (unpleasantly) by listing the variants explicitly, so this tracking issue seems to not be the right place for discussing that feature.

changing a struct (even one with all private fields) to an enum is breaking

Currently, yes. There was a proposal to allow Foo { .. } to match enums which would make it non-breaking.

We have impl From<AsciiChar> for u8 and so forth for the unsigned integer types. I think that we should have that for the signed integer types as well.

This fits perfectly with the semantics of From where the conversion needs to always succeed, as 0 through 127 fit in the nonnegative i8 values. Sometimes ASCII characters are represented as signed numbers in C, so this would make it easy to convert AsciiChar to the type aliases core::ffi::c_char and core::ffi::c_schar without as or try_from.

Per the naming guidelines, as_ascii() should be named to_ascii() since it has an O(N) check.

Navigation Menu

Repository navigation

Tracking Issue for `ascii::Char` (ACP 179) #110998

Public API

Steps / History

Unresolved Questions

192 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	pub struct NonNull<T: PointeeSized> {
	pointer: crate::pattern_type!(*const T is !null),
	}

Tracking Issue for ascii::Char (ACP 179) #110998

Description

Public API

Steps / History

Unresolved Questions

Footnotes

Activity

BurntSushi commented on May 2, 2023

ChayimFriedman2 commented on May 4, 2023

scottmcm commented on May 4, 2023

clarfonthey commented on May 13, 2023

BurntSushi commented on May 13, 2023

clarfonthey commented on May 13, 2023

BurntSushi commented on May 13, 2023

clarfonthey commented on May 13, 2023

scottmcm commented on May 13, 2023

clarfonthey commented on May 14, 2023

192 remaining items

tmke8 commented on Oct 1, 2026

clarfonthey commented on Oct 1, 2026

programmerjake commented on Oct 1, 2026

jhpratt commented on Oct 1, 2026

programmerjake commented on Oct 1, 2026

jhpratt commented on Oct 1, 2026

CodesInChaos commented on Oct 1, 2026

AverageHelper commented on Oct 1, 2026

ronnodas commented on Oct 2, 2026

tmke8 commented on Oct 2, 2026

Kixunil commented on Oct 2, 2026

ronnodas commented on Oct 3, 2026

Kixunil commented on Oct 3, 2026

ChaiTRex commented on Oct 4, 2026

ChayimFriedman2 commented on Oct 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Tracking Issue for `ascii::Char` (ACP 179) #110998