Track test suite failures on s390x

[uweigand]

Running the ./x.py test suite on s390x currently results in a number of failing tests due to various issues. I've gone through and fixed all of those, resulting in a clean run with all my local fixes applied. Opening this issue to track getting all requires fixes upstream.

Specifically, I'm seeing the following failures caused by actual code problems:

• Variadic FFI not implemented
  This is already described in run-make-fulldeps/c-link-to-rust-va-list-fn fails on s390x #84628, and causes the following failures:
  src/test/ui/abi/variadic-ffi.rs
  src/test/run-make-fulldeps/c-link-to-rust-va-list-fn
• Lint suggestions depend on host endianness
  I've opened Lint suggestions depend on host endianness #105379 to track this specifically. This causes failure of:
  src/test/rustdoc-ui/unknown-renamed-lints.rs
• Stack overflow
  The test case described in Stack overflow in rustc, regression from 1.44 #74564 still fails on s390x even though the original fixes are applied. This is probably due to the larger stack requirements on the platforms. Can be fixed by adding another call to ensure_sufficient_stack. This causes failure of:
  src/test/ui/issues/issue-74564-if-expr-stack-overflow.rs

In addition, a number of test cases show (false positive) failures due to assumptions that aren't correct for the platform:

• LLD does not support the s390x target:
  src/test/run-make/issue-71519
  (Test needs to be ignored.)
• The abi_efiapi feature is not supported on s390x:
  src/doc/unstable-book/src/language-features/abi-efiapi.md
  (Test needs to be ignored.)
• A number of LLVM code-gen tests make invalid assumptions on how the IR looks on s390x:
  src/test/codegen/catch-unwind.rs
  src/test/codegen/remap_path_prefix/main.rs
  src/test/codegen/repr-transparent-aggregates-1.rs
  src/test/codegen/repr-transparent.rs
  src/test/codegen/uninit-consts.rs
  (I have fixes for these.)
• Several test cases make little-endian assumptions in output files that are tested against - UI tests
  src/test/ui/const-ptr/forbidden_slices.rs
  src/test/ui/consts/const-eval/ub-enum.rs
  src/test/ui/consts/const-eval/ub-nonnull.rs
  src/test/ui/consts/const-eval/ub-ref-ptr.rs
  src/test/ui/consts/const-eval/ub-uninhabit.rs
  src/test/ui/consts/const-eval/ub-wide-ptr.rs
  src/test/ui/consts/issue-83182.rs
  src/test/ui/consts/std/alloc.rs
  src/test/ui/consts/validate_never_arrays.rs
  Some (but not all) of these are listed in 1.53.0 broke ui/consts/const-eval/ub-*.rs tests on s390x (big-endian, 64-bit) (regression) #89577, with various solutions being proposed. Options would be to either ignore those on big-endian hosts, provide multiple versions of the output to match against, and/or try to handle big- and little-endian outputs within the same file (e.g. via some transformation).
• Several test cases make little-endian assumptions in output files that are tested against - MIR tests
  src/test/mir-opt/const_prop/mutable_variable_no_prop.rs
  src/test/mir-opt/issues/issue_75439.rs
  src/test/mir-opt/building/custom/consts.rs

Longer term, I'm wondering what the best way would be to ensure that the test suite remains clean on the platform - can this be included in CI somehow (either via a native machine somewhere or via qemu-based testing)?

CC @cuviper

[cuviper]

Thanks for your attention on this!

• Several test cases make little-endian assumptions in output files that are tested against

I think I caught most of these in #102379, but only by ignoring them. There's a suggestion that we might apply more fine-grained filtering on that, but I haven't gotten around to implementing that yet.

Longer term, I'm wondering what the best way would be to ensure that the test suite remains clean on the platform - can this be included in CI somehow (either via a native machine somewhere or via qemu-based testing)?

Generally speaking, that's defined by Platform Support and the related Target Tier Policy, where tier-2 (like s390x) is only cross-built in CI, and tier-1 adds test gating. Aarch64-linux is currently the only non-x86 target at tier-1. Part of the challenge is that most developers simply don't have access to s390x hardware to diagnose problems, so it would be burdensome to make that a blocking target.

Perhaps you have resources that could be applied more ad-hoc? e.g. Google has their own CI building Rust with the latest LLVM main branch, from which they report issues (or usually direct fixes). If you could set up similar external CI for s390x testing, we could get a shorter feedback loop without making it a blocking tier-1 target.

[uweigand]

Perhaps you have resources that could be applied more ad-hoc? e.g. Google has their own CI building Rust with the latest LLVM main branch, from which they report issues (or usually direct fixes). If you could set up similar external CI for s390x testing, we could get a shorter feedback loop without making it a blocking tier-1 target.

This is certainly something we could set up. I guess I'm mostly concerned about endian issues, which seem to crop up regularly. Having at least one big-endian target in the CI might be useful ... But we can start with our own external CI, and see how that goes over time.