MaybeUninit requires T: Sized but it should not

[mahkoh]

I tried this code:

pub fn as_maybe_uninit<T: ?Sized>(t: &T) -> &MaybeUninit<T> {
    unsafe {
        mem::transmute(t)
    }
}

I expected to see this happen: It compiles

Instead, this happened: It does not compile

Meta

rustc --version --verbose:

rustc 1.48.0 (7eac88abb 2020-11-16)
binary: rustc
commit-hash: 7eac88abb2e57e752f3302f02be5f3ce3d7adfb4
commit-date: 2020-11-16
host: x86_64-unknown-linux-gnu
release: 1.48.0
LLVM version: 11.0

[camelid]

@mahkoh Can you explain why you think MaybeUninit should not require T: Sized?

[mahkoh]

Being MaybeUninit is not a property that has anything to do with being Sized but with the contents of the occupied memory.

The functions

for<T: Sized> &mut T -> &mut MaybeUninit<T>						(unsafe) (should not require Sized)
for<T: Sized> &mut MaybeUninit<T> -> &mut [MaybeUninit<u8>]		(safe)   (should not require Sized)

occur naturally in APIs that are generic over both uninitialized and initialized arguments.

For example, you want to be able to write

read_into(&mut [0u8]);
read_into(&mut [MaybeUninit::uninit()]);
read_into(&mut some_data_structure_that_can_be_read_into);

with the same read function. In this case T = [?] is unsized.

Because of the T: Sized restriction, the above primitives cannot be composed to create

for<T: ?Sized> &mut T -> &mut [MaybeUninit<u8>]  (unsafe)

This is not a T-libs. I'm almost certain that this restriction is caused by the implementation detail of MaybeUninit being implemented using unions which require all variants to be sized. Whether that restriction is necessary is another question.

[camelid]

Nominating for T-lang/T-libs discussion.

[197g]

The unsizing would be valuable for allocators. In some cases, one wants to use UnsafeCell<MaybeUninit<T>> as a generic pool of memory where the alignment and size requirements are controlled by the type parameter T, and thus by the caller. The ability to use a DST here would expand this pattern such that the length can be a runtime value. The currently best choice of using a [MaybeUninit<u8>] is not exactly satisfactor because it can not be parameterized such that it has a sized equivalent.

[RalfJung]

Allowing unsized MaybeUninit is in conflict with ever having proper support for things like "thin trait objects" or user-defined custom DSTs. There, it would be impossible to determine the size of &MaybeUninit<ThinDst>.

So, while this could in principle be implemented right now (though it would need a special case since I don't see a way to support unsized union fields in general), this would also close the door for some other potential future language extensions.

[mahkoh]

Allowing unsized MaybeUninit is in conflict with ever having proper support for things like "thin trait objects" or user-defined custom DSTs. There, it would be impossible to determine the size of &MaybeUninit<ThinDst>.

How is MaybeUninit<T: ?Sized> different from any other C<T: ?Sized> in this regard?

[RalfJung]

The difference is that it may not be initialized. Determining the size of an &ThinDst requires actually dereferencing the thin pointer, finding the vtable ptr at the target, and getting the size from that. With &MaybeUninit<ThinDst>, the pointee (and, in particular, its vtable ptr) might not be initialized. So suddenly, size_of_val becomes unsafe on a reference.

[mahkoh]

I had read somewhere that size_of_val would not be allowed to access the data pointer even in custom DSTs. Maybe that was outdated information.

That aside, you also said that there would be a problem with unsized union fields. I did not see any such problems so I opened rust-lang/rfcs#3041 yesterday. Maybe leave an explanation there.

[197g]

Would it be possible to make MaybeUninit<T: ?Sized> a valid type but have it be unsafe to produce any values of that type? Relaxing the parameter of the type does not necessarily mean relaxing the parameter of all its current methods, including construction. Then it would fall onto the caller to proof that &MaybeUninit<ThinDst> does not inspect the memory for its size and alignment information. But crucially the cases of MaybeUninit<[u8]> and MaybeUninit<str> might get special treatment that allows their construction/unsizing.

[RalfJung]

I had read somewhere that size_of_val would not be allowed to access the data pointer even in custom DSTs. Maybe that was outdated information.

I mean, this is all hypothetical since custom DST aren't actually a thing.^^ But determining the size will be crucial at least to support things like Box<ThinDst>. So, Box<MaybeUninit<ThinsDst>> becomes an impossibility because Box needs to know the size (on drop) but a maybe-uninitialized thin DST does not have a well-defined size.

Would it be possible to make MaybeUninit<T: ?Sized> a valid type but have it be unsafe to produce any values of that type?

I don't see how that would help with the size issue. It would have to be unsafe to ask for the value's size.