rust build on ARMv8 fails in test macro-stepping.rs, other debuginfo-gdb tests.

[vielmetti]

I built rust on a 96-core ARMv8 machine, and "make -j check" failed in one test:

test [debuginfo-gdb] debuginfo-gdb/macro-stepping.rs ... FAILED

I expected to see everything pass.

A second time through, I got this, with four failures:

thread 'main' panicked at 'Some tests failed', /home/emv/src/rust-lang/rust/src/tools/compiletest/src/main.rs:296
make: *** [tmp/check-stage2-T-aarch64-unknown-linux-gnu-H-aarch64-unknown-linux-gnu-debuginfo-gdb.ok] Error 101

test [debuginfo-gdb] debuginfo-gdb/issue13213.rs ... FAILED
test [debuginfo-gdb] debuginfo-gdb/cross-crate-type-uniquing.rs ... FAILED
test [debuginfo-gdb] debuginfo-gdb/cross-crate-spans.rs ... FAILED
test [debuginfo-gdb] debuginfo-gdb/macro-stepping.rs ... FAILED
test result: FAILED. 95 passed; 4 failed; 6 ignored; 0 measured

Meta

$ ./aarch64-unknown-linux-gnu/stage2/bin/rustc --version --verbose
rustc 1.14.0-dev (6dc035ed9 2016-10-15)
binary: rustc
commit-hash: 6dc035ed911672c6a1f7afc9eed15fb08e574e5b
commit-date: 2016-10-15
host: aarch64-unknown-linux-gnu
release: 1.14.0-dev

% uname -a
Linux armv8hello.local.lan 4.4.0-38-generic #57-Ubuntu SMP Wed Sep 7 10:19:14 UTC 2016 aarch64 aarch64 aarch64 GNU/Linux

[    0.000000] Linux version 4.4.0-38-generic (buildd@bos01-arm64-006) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.2) ) #57-Ubuntu SMP Wed Sep 7 10:19:14 UTC 2016 (Ubuntu 4.4.0-38.57-generic 4.4.19)

[vielmetti]

This might be related to #36323

[vielmetti]

I'm not an expert here (on gdb or on rust), but if someone wants to get access to this hardware I'm happy to make an introduction, it's currently in beta test.

[alexcrichton]

Thanks for the report! Could you also gist the full build logs of the failed tests as well? I don't think anyone's ever run the full test suite on armv8 yet so this may just be a standard "need to do some porting" failure.

[vielmetti]

Sure! Four build logs here.

https://gist.github.com/94ae1bb0318aba060c7b9a39c46c8f7b
https://gist.github.com/893e2f1cb7b7b361beaa087a709f83f2
https://gist.github.com/ec3451e365c77dfb9a32ad12e0a031c8
https://gist.github.com/57d08cf8d298d0859c08e9ac94fa3c0b

[jacobsmith928]

@alexcrichton happy to give you access to a machine for testing, just let me know! Thanks @vielmetti

[alexcrichton]

Looks like the primarily failing test is macro-stepping.rs (all of those logs have this failing test):

---- [debuginfo-gdb] debuginfo-gdb/macro-stepping.rs stdout ----
    NOTE: compiletest thinks it is using GDB version 7.11

error: line not found in debugger output: [...]#loc6[...]
status: exit code: 0
command: gdb -quiet -batch -nx -command=aarch64-unknown-linux-gnu/test/debuginfo-gdb/macro-stepping.debugger.script
stdout:
------------------------------------------
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Breakpoint 1 at 0x19dc: file /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs, line 87.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".

Breakpoint 1, macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:87
87      zzz(); // #break
89      foo!(); // #loc1
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:89
89      foo!(); // #loc1
91      foo2!(); // #loc2
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:91
91      foo2!(); // #loc2
93      let x = vec![42]; // #loc3
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:93
93      let x = vec![42]; // #loc3
93      let x = vec![42]; // #loc3
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:93
93      let x = vec![42]; // #loc3
95      new_scope!(); // #loc4
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:95
95      new_scope!(); // #loc4
97      println!("Hello {}", // #loc5
#0  macro_stepping::main () at /home/emv/src/rust-lang/rust/src/test/debuginfo/macro-stepping.rs:97
97      println!("Hello {}", // #loc5
A debugging session is active.

    Inferior 1 [process 7811] will be killed.

Quit anyway? (y or n) [answered Y; input not from terminal]

------------------------------------------
stderr:
------------------------------------------

------------------------------------------

thread '[debuginfo-gdb] debuginfo-gdb/macro-stepping.rs' panicked at 'explicit panic', /home/emv/src/rust-lang/rust/src/tools/compiletest/src/runtest.rs:2372
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
[debuginfo-gdb] debuginfo-gdb/macro-stepping.rs

Other failures look like:

error: could not write output to [..]: No such file or directory

The former may just be a legit bug with AArch64 on our end (or LLVM's) and the latter may just be a bug in the build system or something like that maybe? Does the machine here have anything odd with its filesystem that may cause that to arise?

[vielmetti]

The file system it is using is an iscsi mount. I will run tests again on a local file system.

[vielmetti]

I'm able to replicate a failing test on a different file system. I captured a gist with RUST_BACKTRACE=1 to show more of what is going on.

https://gist.github.com/a0304ca94a51fd55f4853f609db65d67

[alexcrichton]

Ah ok looks like those filesystem errors aren't related to any weird mount issues or anything like that.

@michaelwoerister do you thoughts about the errors here? Either the what appears to be legit error or the filesystem issues?

[vielmetti]

The next thing I'm going to try is "make check" instead of "make -j check" just in case there are parallelism issues in the makefile.

[michaelwoerister]

Off the top of my head I can't say what the problem could be. The file system errors occur during builds of auxiliary libraries which is something that many other kinds of tests do too.

As for the stepping, it might be a problem with the aarch64 code generator? I'll try if I can reproduce the problem.

[vielmetti]

The only thing that comes to mind (which we ran into in the Go language port) was PAGE_SIZE variances among aarch64 systems. A little bit of detail is here

https://marcin.juszkiewicz.com.pl/2015/09/14/from-a-diary-of-aarch64-porter-page_size/

I'm stabbing in the dark here, but that said, this seems to be a common portability issue with aarch64.

[jacobsmith928]

@michaelwoerister nice to see a fellow NYC infrastructure-r on this thread. If you need any access to ARMv8 test boxes, we can give you access to our on demand beta, just email me jacob at packet dot net.

[michaelwoerister]

@jacobsmith928 Thanks! I recently got an ODROID C2 which should have the right architecture, right?