Fix repr(align) enum handling

[RalfJung]

enum, for better or worse, supports repr(align). That has already caused a bug in #92464, which was "fixed" in #92932, but it turns out that that fix is wrong and caused #96185.

So this reverts #92932 (which fixes #96185), and attempts another strategy for fixing #92464: special-case enums when doing a cast, re-using the code to load the discriminant rather than assuming that the enum has scalar layout. This works fine for the interpreter.

However, #92464 contained another testcase that was previously not in the test suite -- and after adding it, it ICEs again. This is not surprising; codegen needs the same patch that I did in the interpreter. Probably this has to happen around here. Unfortunately I don't know how to do that -- the interpreter can load a discriminant from an operand, but codegen can only do that from a place. @oli-obk @eddyb @bjorn3 any idea?

[rust-highfive]

Some changes occured to the CTFE / Miri engine

cc @rust-lang/miri

[rust-highfive]

r? @jackh726

(rust-highfive has picked a reviewer for you, use r? to override)

[eddyb]

r=me with comment fix (just the typo would be enough, but I got side-tracked using the change suggestion UI and added some more details)

[eddyb]

However, #92464 contained another testcase that was previously not in the test suite -- and after adding it, it ICEs again. This is not surprising; codegen needs the same patch that I did in the interpreter. Probably this has to happen around here. Unfortunately I don't know how to do that -- the interpreter can load a discriminant from an operand, but codegen can only do that from a place. @oli-obk @eddyb @bjorn3 any idea?

Ah I missed this, right, so this PR not ready for merging.
There's also this very interesting piece of code I noticed while looking around codegen:

rust/compiler/rustc_codegen_ssa/src/mir/rvalue.rs

Lines 301 to 340 in d32ce37

	if let Abi::Scalar(scalar) = operand.layout.abi {
	if let Int(_, s) = scalar.primitive() {
	// We use `i1` for bytes that are always `0` or `1`,
	// e.g., `#[repr(i8)] enum E { A, B }`, but we can't
	// let LLVM interpret the `i1` as signed, because
	// then `i1 1` (i.e., E::B) is effectively `i8 -1`.
	signed = !scalar.is_bool() && s;
	if !scalar.is_always_valid(bx.cx())
	&& scalar.valid_range(bx.cx()).end
	>= scalar.valid_range(bx.cx()).start
	{
	// We want `table[e as usize ± k]` to not
	// have bound checks, and this is the most
	// convenient place to put the `assume`s.
	if scalar.valid_range(bx.cx()).start > 0 {
	let enum_value_lower_bound = bx.cx().const_uint_big(
	ll_t_in,
	scalar.valid_range(bx.cx()).start,
	);
	let cmp_start = bx.icmp(
	IntPredicate::IntUGE,
	llval,
	enum_value_lower_bound,
	);
	bx.assume(cmp_start);
	}
	let enum_value_upper_bound = bx
	.cx()
	.const_uint_big(ll_t_in, scalar.valid_range(bx.cx()).end);
	let cmp_end = bx.icmp(
	IntPredicate::IntULE,
	llval,
	enum_value_upper_bound,
	);
	bx.assume(cmp_end);
	}
	}
	}

So the discriminant reading should happen before that, with let llval = operand.immediate(); presumably special-cased when operand.layout.ty.is_enum().
The llval value just before the actual int2int cast would be identical to this I'm guessing:

rust/compiler/rustc_codegen_ssa/src/mir/rvalue.rs

Lines 475 to 488 in d32ce37

	mir::Rvalue::Discriminant(ref place) => {
	let discr_ty = rvalue.ty(self.mir, bx.tcx());
	let discr_ty = self.monomorphize(discr_ty);
	let discr = self
	.codegen_place(&mut bx, place.as_ref())
	.codegen_get_discr(&mut bx, discr_ty);
	(
	bx,
	OperandRef {
	val: OperandValue::Immediate(discr),
	layout: self.cx.layout_of(discr_ty),
	},
	)
	}

---

This suggests another avenue for these casts: enum->integer casts should be lowered in MIR to Rvalue::Discriminant( (which would produce an integer of the discriminant type) followed by the original cast to integer. It also has the advantage that it would work on enums with data just as well, if we ever want to support those in as casts.

[eddyb]

cc @nagisa wrt the above comment, and because of the codegen logic that #75529 touched - if we go the route of not having direct enum->integer casts in MIR (but instead relying on Rvalue::Discriminant, we would have no obvious place to put that assume.

However, it strikes me that today enum->integer casts can be by-value in codegen, whereas Rvalue::Discriminant is memory-only (i.e. requires a PlaceRef) for now. So instead of the assume we'd get a load with !range metadata attached, that might be just as good?

(And/or if/when we get the ability to read discriminants from OperandRefs directly, that could be where the assume would go, replacing the !range metadata?)

[RalfJung]

The llval value just before the actual int2int cast would be identical to this I'm guessing:

I tried that, but couldn't complete the type golf since I didn't have a place to work with.

This suggests another avenue for these casts: enum->integer casts should be lowered in MIR to Rvalue::Discriminant( (which would produce an integer of the discriminant type) followed by the original cast to integer. It also has the advantage that it would work on enums with data just as well, if we ever want to support those in as casts.

Yeah that would be nice, make both Miri's and codegen's job much easier. :D
But MIR building is even scarier than codegen, so I doubt I can do that...

[nagisa]

The enum as scalar cast was always intended to behave as if it was reading out a discriminant, wasn’t it? I'm quite surprised this isn't already happening.

---

I'm perfectly content with the idea of us losing assumes that have been added before if it helps some agenda that's grander than micro-optimizing a handcrafted test case. I said as such in the linked PR:

I don’t mind adding this either, but I wouldn’t hold my breath that any specific behaviour will last, especially across backend versions.

!range definitely sounds much more palatable and resilient approach too.

[RalfJung]

The enum as scalar cast was always intended to behave as if it was reading out a discriminant, wasn’t it? I'm quite surprised this isn't already happening.

Good, so looks like we have broad agreement here. Then we seem to have 2 options:

• Either this happens already during MIR building. Then we need someone familiar with that code to pick up this PR; that's a bit too much foreign code for me to pick up right now.
• Or the codegen backends all do something like that themselves. I did that for Miri; with a little help hopefully I can get it done for the SSA backend. The main problem is that I have an OperandRef but to call the get disicriminant code I need a PlaceRef, and I don't know how to convert from one to the other. In Miri we have force_allocation. In codegen, does something like that exist -- is that even possible? Or do we need to adjust the SSA logic to not treat "enums that are inputs to cast" as by-value so that we are sure we have a by-ref here and somehow can get that by-ref information out of the OperandRef?

Doing it in the MIR seems more elegant, so if we can get some MIR people to help here that would be great. :)

[oli-obk]

• Then we need someone familiar with that code to pick up this PR; that's a bit too much foreign code for me to pick up right now.

I'll create a PR

[oli-obk]

I'll create a PR

done: #96862

I'm perfectly content with the idea of us losing assumes that have been added before if it helps some agenda that's grander than micro-optimizing a handcrafted test case.

We lost some of them. I tried to get them back by telling LLVM more things about the result of the discriminant (the input automatically gets that via Operand::load). That ended up giving lots of miscompilations. I think the right level of changing things would be in LLVM opts, as it works for enum Foo { A, B }, but just about nothing else (like not even enum Foo { A = 1, B = 2 } or various other repesentation changes).

[bors]

☔ The latest upstream changes (presumably #96891) made this pull request unmergeable. Please resolve the merge conflicts.

[jackh726]

r? @eddyb

[rustbot]

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

[RalfJung]

Now that #96862 landed, this becomes easy. :) (Other than convincing github to not show outdated diffs.)
@oli-obk ready for review again.

[oli-obk]

should this be an assertion instead of a print? Otherwise we're only testing that this code runs, not that it does the right thing

[RalfJung]

I have added assertions but kept the prints, just in case they were relevant to trigger the ICE.

[oli-obk]

r=me with an explanation for the prints or changed to an assert

[RalfJung]

@bors r=oli-obk

[bors]

📌 Commit d5721ce has been approved by oli-obk