Rust 1.33.0 failing UI tests when run with single core

[ipetkov]

I'm trying to package/build Rust 1.33.0 on Guix and I'm consistently getting some test failures regarding panic messages.

Several run-fail tests check that a #[test] with a name like test_foo prints a message like thread 'test_foo' panicked..., but instead the output will come out as thread 'main' panicked....

I'm not really sure what can be the cause of the issue, but FWIW we build Rust with the previous stable version already packaged in Guix, and we'll link to our packaged versions of llvm and jemalloc.

Happy to provide more details as needed; log from the failing test cases is below

Details

  < Compiletest { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu", mode: "compile-fail", suite: "compile-fail", path: Some("src/test/compile-fail"), compare_mode: None }
< CompileFail { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
c Assemble { target_compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" } }
> RunFail { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
  > Compiletest { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu", mode: "run-fail", suite: "run-fail", path: Some("src/test/run-fail"), compare_mode: None }
    c Test { target: "x86_64-unknown-linux-gnu", compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" } }
    c Test { target: "x86_64-unknown-linux-gnu", compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" } }
    c TestHelpers { target: "x86_64-unknown-linux-gnu" }
    c TestHelpers { target: "x86_64-unknown-linux-gnu" }
    c RemoteCopyLibs { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
    c Assemble { target_compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
    c Compiletest { compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
    c Assemble { target_compiler: Compiler { stage: 0, host: "x86_64-unknown-linux-gnu" } }
    c Llvm { target: "x86_64-unknown-linux-gnu", emscripten: false }
    c Sysroot { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" } }
    c Libdir { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "x86_64-unknown-linux-gnu" }
    c Sysroot { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" } }
    c Llvm { target: "x86_64-unknown-linux-gnu", emscripten: false }
Check compiletest suite=run-fail mode=run-fail (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
running: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail" "--build-base" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-fail" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/gnu/store/bvnj04sqiwc1gcis29swk6likyc5dw8f-llvm-6.0.1/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/gnu/store/g87hamjyipk1j6dfq5pjfzfnfb64spbv-python2-2.7.15/bin/python2" "--lldb-python" "/gnu/store/g87hamjyipk1j6dfq5pjfzfnfb64spbv-python2-2.7.15/bin/python2" "--gdb" "/gnu/store/pp6nvrw47ip2r54k4kyk5a8fhyxvhsb8-gdb-8.2/bin/gdb" "--verbose" "--quiet" "--llvm-version" "6.0.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" ""
configuration:
compile_lib_path: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib"
run_lib_path: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib"
rustc_path: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/bin/rustc"
rustdoc_path: None
src_base: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail"
build_base: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail"
stage_id: stage2-x86_64-unknown-linux-gnu
mode: run-fail
run_ignored: false
filter: (none)
filter_exact: false
runtool: (none)
host-rustcflags: -Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers
target-rustcflags: -Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers
target: x86_64-unknown-linux-gnu
host: x86_64-unknown-linux-gnu
android-cross-path: ""
adb_path: "adb"
adb_test_dir: "/data/tmp/work"
adb_device_status: false
ar: ar
linker: None
verbose: true
quiet: true
running 143 tests

.................................................................................................... 100/143

........................FF.................

failures:
---- [run-fail] run-fail/test-panic.rs stdout ----
executing "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-panic.rs" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-panic/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--test" "-L" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-panic/auxiliary"

------stdout------------------------------
------stderr------------------------------

executing "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-panic/a"

------stdout------------------------------
running 1 test

test test_foo ... FAILED
failures:
---- test_foo stdout ----

thread 'main' panicked at 'explicit panic', /tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-panic.rs:8:5

note: Run with RUST_BACKTRACE=1 environment variable to display a backtrace.
failures:

test_foo
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
------stderr------------------------------

error: error pattern 'thread 'test_foo' panicked at' not found!

status: exit code: 101

command: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-panic/a"

stdout:
running 1 test

test test_foo ... FAILED
failures:
---- test_foo stdout ----

thread 'main' panicked at 'explicit panic', /tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-panic.rs:8:5

note: Run with RUST_BACKTRACE=1 environment variable to display a backtrace.
failures:

test_foo
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

stderr:

thread '[run-fail] run-fail/test-panic.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3245:9
---- [run-fail] run-fail/test-should-fail-bad-message.rs stdout ----
executing "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-should-fail-bad-message.rs" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-should-fail-bad-message/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--test" "-L" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-should-fail-bad-message/auxiliary"

------stdout------------------------------
------stderr------------------------------

executing "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-should-fail-bad-message/a"

------stdout------------------------------
running 1 test

test test_foo ... FAILED
failures:
---- test_foo stdout ----

thread 'main' panicked at 'blah', /tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-should-fail-bad-message.rs:9:5

note: Run with RUST_BACKTRACE=1 environment variable to display a backtrace.

note: Panic did not include expected string 'foobar'
failures:

test_foo
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
------stderr------------------------------

error: error pattern 'thread 'test_foo' panicked at' not found!

status: exit code: 101

command: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail/test-should-fail-bad-message/a"

stdout:
running 1 test

test test_foo ... FAILED
failures:
---- test_foo stdout ----

thread 'main' panicked at 'blah', /tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail/test-should-fail-bad-message.rs:9:5

note: Run with RUST_BACKTRACE=1 environment variable to display a backtrace.

note: Panic did not include expected string 'foobar'
failures:

test_foo
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

stderr:

thread '[run-fail] run-fail/test-should-fail-bad-message.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3245:9
failures:

[run-fail] run-fail/test-panic.rs

[run-fail] run-fail/test-should-fail-bad-message.rs
test result: FAILED. 141 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out
thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:495:22
command did not execute successfully: "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/src/test/run-fail" "--build-base" "/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/test/run-fail" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-fail" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/gnu/store/bvnj04sqiwc1gcis29swk6likyc5dw8f-llvm-6.0.1/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/tmp/guix-build-rust-1.33.0.drv-0/rustc-1.33.0-src/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/gnu/store/g87hamjyipk1j6dfq5pjfzfnfb64spbv-python2-2.7.15/bin/python2" "--lldb-python" "/gnu/store/g87hamjyipk1j6dfq5pjfzfnfb64spbv-python2-2.7.15/bin/python2" "--gdb" "/gnu/store/pp6nvrw47ip2r54k4kyk5a8fhyxvhsb8-gdb-8.2/bin/gdb" "--verbose" "--quiet" "--llvm-version" "6.0.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" ""

expected success, got: exit code: 101

[ipetkov]

I've managed to confirm that the issue occurs when the compiler is built with vendor = true set in the config.toml, the test suite will pass when built with the 1.33.0 tarball but vendor = false.

What is the process for creating the vendor directory when creating a source tarball of the compiler? Are there any tests that verify the right dependencies were vendored in?

[ipetkov]

Aha! I turns out it's not a vendoring issue, but rather a number of test threads issue!

I was able to reproduce without vendoring, but by invoking the test suite with a single core:

./x.py test -vv -j1 src/test/run-fail # fails the two test cases above
./x.py test -vv src/test/run-fail # succeeds