rustbuild: can't cross compile compiler-rt for ARM

[japaric]

rustbuild is using cmake to build compiler-rt instead of Makefiles like the old build system does,
and there are some issues that make the build fail when the target is ARM.

Background info

For ARM, there 3 floats ABI (see -mfloat-abi in 1):

• soft: processor has no Floating Point Unit (FPU), floating point operations are software
  routines/library calls.
• hard: processor has FPU, floating point operations are hardware instructions.
• softfp: processor has FPU, can use the hardware instructions but still uses soft calling
  convention.

As far as Rust is concerned:

• arm(v7)-unknown-linux-gnueabihf uses the hard ABI.
• arm-unknown-linux-gnueabi uses the soft ABI.

Observations when using cmake

In general:

• compiler-rt makes a distinction between an arm target and an armhf target in its sanity
  checks, but then goes off and generates the same set of symbols for both targets. Not a
  problem, but why then bother making a distinction in the first place?
• (a) compiler-rt demands a C++ compiler in its checks even though it's not used at all to compile
  the builtins that form part of libcompiler-rt.a. This forces the user to install a g++ compiler
  were gcc was sufficient before (in the Makefile case). This particularly annoying for musl
  targets where only a musl-gcc wrapper is available in the buildbots. Note: It seems the C++
  compiler is required to compile the sanitizers, but we don't build those right now.
• When libcompiler-rt.a is built using cmake for ARM it has more symbols than when is built
  using the Makefile system. Note the *vfp symbols which are in the cmake version but not in the
  Makefile version; they are relevant in the next section.

When targeting arm-unknown-linux-gnueabi:

• (b) The extra *vfp symbols can't be compiled for this target; you get a compiler error: "Error:
  selected processor does not support ARM mode". At least that's the case with Ubuntu's
  arm-linux-gnueabi-gcc toolchain and a crosstool-ng generated toolchain.

When targeting arm(v7)-unknown-linux-gnueabihf:

• (c) compiler-rt expects the arm compiler to be able to generate programs for both the soft
  ABI and the hard ABI. The arm toolchains shipped in Ubuntu have no problems with this check
  (both tests pass), but when a crosstool-ng compiler is used this check fails in the soft
  case and the cmake build is aborted.

Other than that, I have had no problems (cross) compiling compiler-rt to i686, x86_64, mips or
mipsel apart from the annoying extra g++ requirement.

What to do?

The simplest way to get rid of all these issues is to build compiler-rt using Makefiles at least
when the target is ARM. The other alternative is to patch compiler-rt:

• To fix (a), we can disable the C++ compiler check because we are not compiling the sanitizers
  (yet).
• For (b), we can filter out the problematic symbols (about 34 assembly files) when compiling for
  soft float ARM. It's OK, I think, to exclude these symbols because our current releases don't include
  them for any ARM target (AFAICT).
• For (c), we could disable the soft float check when the target is hard float ARM.

Thoughts @alexcrichton?

[alexcrichton]

compiler-rt demands a C++ compiler in its checks even though it's not used at all to compile the builtins that form part of libcompiler-rt.a.

Yeah this kinda sucks :. I managed to hack around it for now, but I don't think that works for all platforms. It would be best to upstream support for this most likely.

compiler-rt expects the arm compiler to be able to generate programs for both the soft ABI and the hard ABI [..] but when a crosstool-ng compiler is used this check fails in the soft case and the cmake build is aborted

Oh my god building compiler-rt is a nightmare

Thoughts @alexcrichton?

I was under the impression that the makefiles are basically dead. I know they are for the LLVM project itself, and I've heard that compiler-rt folks have been surprised in the past we're still using makefiles. Along those lines we probably want to continue using CMake where possible.

I guess the best thing to do would be to figure out which patches need to make their way upstream, and try to do that. We've diverged from compiler-rt "quite a bit" I think which is pretty unfortunate, so upgrading may be hard for us.

All around this isn't a great situation unfortunately :(

[japaric]

Yeah this kinda sucks :. I managed to hack around it for now, but I don't think that works for all platforms. It would be best to upstream support for this most likely.

FWIW, this didn't work when I tried to use mips musl-gcc wrapper to cross compile compiler-rt; it failed in these c++ tests. What did work was installing a crosstool-ng generated mips-unknown-linux-musl-{gcc,g++} toolchains and then using them to cross compile compiler-rt, that worked flawlessly.

Oh my god building compiler-rt is a nightmare

The weirdest thing is that the arm-unknown-linux-gnueabi target passes the soft float test and not the hard one (the opposite of the arm-unknown-linux-gnueabihf target) but it doesn't abort the cmake build, instead it successfully passes all the sanity/configuration tests. And then it goes and fails at point (b) :P.

I guess the best thing to do would be to figure out which patches need to make their way upstream, and try to do that.

OK, I'll see what I can do, but I don't actually speak cmake.