Tracking Issue for `c"…"` string literals

[tmandry]

This is a tracking issue for the RFC "c"…" string literals" (rust-lang/rfcs#3348).
The feature gate for the issue is #![feature(c_str_literals)].

Steps / History

• Implement the RFC: Implement RFC 3348, c"foo" literals #108801
• Interaction with proc macros: c"…" literals are handled incorrectly by proc_macro::Literal #112820
• regression: c"..." are experimental #113235
• Revert the lexing of c"…" string literals #113334
• Reimplement the lexing of c"…" string literals with backward compatibility in mind #113333
• proc_macro does not have a constructor for c string literals #118560
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR Stabilize C string literals #117472

Unresolved Questions

• Also add c'…' C character literals? (u8, i8, c_char, or something more flexible?)
• Should we make &CStr a thin pointer before stabilizing this? (If so, how?)
• Should the (unstable) concat_bytes macro accept C string literals? (If so, should it evaluate to a C string or byte string?)
• Should there be a valid UTF-8 C string type that c"..." string literals support? (comment 1, comment 2)

[CAD97]

w.r.t.

c'…' character literals

IIRC, C23 accepted N2653, which defines char8_t as a typedef for unsigned char. (C++20's char8_t is a distinct type.)

Assuming I'm correct that N2653 was accepted, it probably makes sense to define Rust's c"…" as being compatible with C's u"…" (char8_t[]) and Rust's c'…' as being compatible with C's u8'…' (char8_t).

To be explicit, this is not intended to argue that c"…"_Rust should be usable as *const c_uchar, just that c"…".as_ptr()_Rust should have the same memory content as u8"…"_C. It may argue for c"…"_Rust requiring UTF-8 well formedness; I do not know whether C's u8"…" allows constructing invalid UTF-8 via escapes.

Note that &CStr's pointer representation uses c_char_Rust, not c_uchar_Rust (which would match char8_t_C). This serves as a counterargument that c strings in Rust should match bare string/character literals in C, which use unqualified char_C, rather than the C17 (u8'…') and C23¹ (u8"…") prefixed literals.

The presence of UTF-8 literals being unsigned char in C may additionally add weak motivation for a UTF-8 CStr type for Rust separate from CStr, which corresponds to the unqualified c_char type.

Footnotes

1. To be fully accurate to my understanding, C17 added u8"…" literals of type array of char, and C23 changed their type to array of char8_t. Similarly, C17 added u8'…' literals of type unsigned char, and C23 changed their type to char8_t (for no semantic change, as char8_t is a typedef of unsigned char). ↩

[ojeda]

Assuming I'm correct that N2653 was accepted

Yeah, char8_t was voted into C23.

It may argue for c"…"Rust requiring UTF-8 well formedness; I do not know whether C's u8"…" allows constructing invalid UTF-8 via escapes.

I think it was possible before, and will still be in C23:

"Any hexadecimal escape sequence or octal escape sequence specified in a u8, u, or U string specifies a single char8_t, char16_t, or char32_t value and may result in the full character sequence not being valid UTF-8, UTF-16, or UTF-32."

[dead-claudia]

I do want to note a couple things in the alternatives:

1. While I like the idea of a FromStringLiteral, error messages would need at least a character offset to be actually useful beyond a mere compile-time panic!. Also, &CStrs are really more like byte string literals, and that offset would have to be a byte offset, not a char offset, too.

2. If some platforms use a u8 for c_char while others use a i8, you need the c'...' literal to be assignable to c_char regardless of which it is in order to be portable. Allowing it to coerce to either i8 or u8 is probably better than having it exactly what c_char is, though. (Ideally, c_char should've been #[repr(transparent)] struct c_char(pub i8/u8); to force people to differentiate the two, but the time to change to that is long past.)

[tmandry]

@CAD97 @dead-claudia Would you be willing to open issues for these points and add the F-c_str_literal label (or tag me and I can add it)? Otherwise they are likely to get lost in the discussion. Tracking issues are meant to serve as a hub to point to other threads, not to be a place for technical discussion themselves.

[dead-claudia]

Created #106479 for my first point. The second point's not likely to be relevant before a PR, and there's really not much to discuss there.

[tgross35]

Should we make &CStr a thin pointer before stabilizing this? (If so, how?)

I don't think there would be anything about that change that should block this, it would just have to be updated whenever CStr becomes thin. The change itself is blocked, I think #81513 would solve it, but it is still quite young. Seems like there hasn't been much discussion on the topic recently anyway - #59905 is the only issue I could find, but it's pretty unofficial and stale