String::retain: convert only the untouched UTF-8 tail

[Talha-Dmr]

During compaction, String::retain reads the next character with g.s.get_unchecked(read..len). This invokes String’s Deref, constructing a &str over the entire 0..len buffer before selecting the unread tail.

For example, removing 'é' from "éab" and moving 'a' left changes the buffer from C3 A9 61 62 to 61 A9 61 62. The next iteration reads the valid 'b' tail, but first converts the entire buffer, including the orphaned A9 continuation byte, to &str.

Constructing a non-UTF-8 &str is not immediate language-level UB under the Reference’s validity requirements. However, this conversion violates from_utf8_unchecked’s documented safety precondition. The libs discussion in #134598 is defining which operations accept invalid UTF-8; str::get_unchecked is absent from the tentative list.

The fix slices read..len directly from the byte vector and converts only that untouched tail to &str. The original string was valid UTF-8, and read remains on a character boundary. #154583 uses the same approach for extract_if.

Behavior is unchanged. A separate codegen comparison produced identical optimized x86-64 assembly for the tested predicates.

Validation

I ran:

• ./x test library/alloc: all 1,491 tests passed, including test_retain.
• ./x test tidy: passed.
• ./x miri library/alloc: test_retain and the retain documentation tests passed under both Stacked Borrows and Tree Borrows, without reported UB.

I also ran differential comparisons covering:

• 2,054,353 exhaustive cases, including 1,754,760 cases where the predicate panicked.
• 39,745 randomized cases.
• One dedicated "éab" case.

These total 2,094,099 comparison cases; the panic cases are included in the exhaustive count.

AI involvement

An AI assistant initially noticed the potential issue during a verification attempt. I independently verified the finding and confirmed the issue myself. I wrote the patch, SAFETY comment, and this description.

I used AI assistance to research the implementation and related PRs, and to assist with differential comparisons and Miri checks.

Related: #150067, #134598, #154583.

[rustbot]

Thanks for the pull request, and welcome! The Rust Project has assigned @JohnTitor (or someone else) to review your changes, you should hear from them (or someone else) within the next two weeks.

Please see the contribution instructions and our LLM policy for more information.

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: libs
• libs expanded to 12 candidates
• Random selection from Darksonn, JohnTitor, Mark-Simulacrum, clarfonthey