Tell LLVM about NonZero integer types

[tromey]

Change the get method on NonZero integer types to insert an
assume, so that LLVM can optimize better.

Closes #54868

[rust-highfive]

r? @dtolnay

(rust_highfive has picked a reviewer for you, use r? to override)

[dtolnay]

r? @eddyb

[scottmcm]

Would it be worth a CHECK-NOT of some sort that there's no branch/select/panic/whatever?

Also, cc @rkruppe who might have thoughts here.

[nagisa]

Yes, the assume is just a hint and might get removed at some stage of the pipeline in the future. Checking for lack of the branch is the more correct way to test this here.

[hanna-kruppe]

I am hesitant to add assumes in random places because they add more IR (particularly if they're in a very common, small and usually-inlined function like this) and they can, unfortunately, also block some optimizations.

[tromey]

they can, unfortunately, also block some optimizations.

I'd like to learn what optimization this could block.

[nagisa]

@tromey anything that would be blocked by a regular non-inlined call to an empty function is going to be blocked by the assume intrinsic the same way. This is what the LLVM reference has to say about it:

Note that the optimizer might limit the transformations performed on values used by the llvm.assume intrinsic in order to preserve the instructions only used to form the intrinsic’s input argument. This might prove undesirable if the extra information provided by the llvm.assume intrinsic does not cause sufficient overall improvement in code quality. For this reason, llvm.assume should not be used to document basic mathematical invariants that the optimizer can otherwise deduce or facts that are of little use to the optimizer.

In addition to that, the call to @llvm.assume might act as a side effect for the purposes of optimisations which do not explicitly take this intrinsic into account, inhibiting optimisation within the function as a whole.

[hanna-kruppe]

Besides passes treating llvm.assume as an unknown function (which I figure can usually be fixed easily), there's also many optimizations that are gated on a value having only one use (this is quite common, grep for hasOneUse) and these are often hard to generalize to multiple uses.

[nagisa]

The most appropriate way to solve this would be to assign the !range metadata to loads from and stores to NonZero, however that would require changes to the compiler instead of the library. Since NonZero<T> is a language item, it should be fairly feasible to do, however.

[hanna-kruppe]

We already have range metadata on loads (there's no range metadata on stores). Without making NonZeroU32 a lang item, too -- it's a consequence of it being laid out as scalar and inheriting the niche from NonZero<u32>. Thus we get the desired optimization when there is an actual load. The problem is when the NonZeroU32 is passed as immediate or turned into one by SROA, since then there are no loads to carry the range metadata and there is no range metadata for arguments (see #50156).

[hanna-kruppe]

Sorry, see #50156, not #50157.

[tromey]

Thanks.