Repository navigation
Tracking Issue for Hasher::{write_str, write_length_prefix} (feature(hasher_prefixfree_extras)) #96762
Description
Feature gate: #![feature(hasher_prefixfree_extras)]
This is a tracking issue for the new provided methods on Hasher added to fix #94026
write_strlets the hasher customize how it works withstr, so it can use theb'\xFF'trick if it's byte-wise, or a different approach if it does chunked rounds.write_length_prefixgives an obvious choice when implementingHashfor collections (likeVecDeque) which can't just use the slice hash, and allows the hasher to optimize how best to represent the length.
Public API
// core::hash
pub trait Hasher {
// ... existing stuff ...
fn write_length_prefix(&mut self, len: usize);
fn write_str(&mut self, s: &str);
}Steps / History
- Implementation: Add a dedicated length-prefixing method to
Hasher#94598 - Final comment period (FCP)
- Stabilization PR
Unresolved Questions
- What should the
write_strprovided implementation be? It was added matching the previousimpl Hash for strbehaviour to get in without breaking hash checks (like the one incargo), but that's not always prefix-free (it depends on the round strategy), so there's an argument that a different implementation would be better.
Reacted by Liam Gray, Peter Lyons Kehl and teor
Activity
tgross35 commentedon Apr 10, 2024
Possible bikeshed: it was indirectly mentioned in passing here that this is the only std API that comes up when you search
length. The rest all seem to uselenhoxxep commentedon May 17, 2025
Author of the rapidhash crate here, I'm wondering how to go about pushing this towards stabilisation?
Being able to omit the separate
0xFFwrite call when hashing strings would be a helpful performance boost in block-based hashing algorithms that already reseed using the byte length on every.write(&[u8])call. Thanks!ChrisDenton commentedon May 22, 2025
See the stabilization process docs: https://std-dev-guide.rust-lang.org/development/stabilization.html#stabilization-report. Essentially write a stabilization report detailing what is being stabalized and why.
I see one "unresolved question" in the OP. That will probably need to be resolved one way or another.
hoxxep commentedon Jul 27, 2025
Linking a relevant PR and discussion about
write_str's default behaviour and alternate suggestions: #134134 (comment)Key points:
write_bytesmethod, similar towrite_str. This would move thewrite_length_prefixcall out of theimpl<T> Hash for [T]and into theHash::hash_slicemethod. This would allow the unpadded numeric slices to overridehash_sliceto callwrite_bytesand give hashers like foldhash and rapidhash the chance to omit the unnecessary write_length_prefix call for byte slices for their algos.write_length_prefixcan only ever really meanwrite_usize. However, it's still useful to include, as non-DoS resistant hashers can omit the length prefix, but wouldn't want to omit hashing usize fields.write_str(or the inclusion ofwrite_bytes) would break the hash output stability ofHash::hashbetween compiler versions. This isn't necessarily a problem, the std library doesn't promise stability. This would however be the first major breaking change toHash::hashoutput stability, and I'm aware there are many users who currently abuse the standard library types for portable hashing that are blissfully unaware.I am currently trialling
write_bytesin an experimental portable-hash crate as an example, with the intention of offeringPortableHashandPortableHashertraits to make it easier for users to maintain hash output stability between platforms and compiler versions.peter-lyons-kehl commentedon Sep 7, 2025
Thank you so much for
hasher_prefixfree_extras. I have a different use case for this (extra signalling protocol between select types being hashed and a customHasheradapter; I'll share more once it's usable).cargostores/depends on hash values? If that's whatcargodoes, then they could write their ownHasherimplementation/adapter using the existing/legacy code, or find some other way to workaround. It's clearly against https://doc.rust-lang.org/nightly/core/hash/trait.Hash.html#portability:cargousesHashMapwithRandomState(its defaultBuildHasher), that usesDefaultHasher, whose "internal algorithm is not specified, and so it and its hashes should not be relied upon over releases."clarfonthey commentedon Nov 14, 2025
Poking around here since I'd like these:
Right now, I think that
write_strshould be kept, butwrite_string_suffixshould be added to replace calls towrite_str(""). Essentially, for now,write_u8(0xFF)is relying a bit too much on the internals of strings for taste, and it would be better to be able to easily match libstd string hashing behaviour without just using a magic number.I think it would be reasonable, if libstd wants to commit to slices and strings hashing differently, to have a dedicated
write_string_suffixmethod to replace a call towrite_u8(0xFF)as needed, or to eliminate it altogether and requirewrite_length_prefixinstead.