Rust source make install fails due to missing vendor folder

[itarato]

I was building Rust (branch master) from source. ./configure and make completed without any problems, however sudo make install immediately fails with the following problem:

sudo make install
info: looks like you are running this command under `sudo`
      and so in order to preserve your $HOME this will now
      use vendored sources by default. Note that if this
      does not work you should run a normal build first
      before running a command like `sudo make install`
error: failed to load source for a dependency on `toml`

Caused by:
  Unable to update registry https://github.com/rust-lang/crates.io-index

Caused by:
  failed to update replaced source `registry https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to read root of directory source: /mnt/shared/Checkout/rust/src/vendor

To learn more, run the command again with --verbose.
Makefile:71: recipe for target 'install' failed
make: *** [install] Error 101

The error: failed to load source for a dependency on 'toml' line is complaining about seemingly rangom things, such as cmake, toml or getopts. However the last line (failed to read root of directory source: /mnt/shared/Checkout/rust/src/vendor) is odd as it doesn't have any src/vendor folder and this kind of error is more likely on Rust projects using cargo.toml.

For reference, my build env:

• os: Ubuntu 16.04
• clang 3.8
• python 2.7
• make 4.1
• cmake 3.5.1
• curl 7.47
• git 2.7

I also have already installed via the official rustup script.

So far no luck getting closer to what would cause this. Do you have any idea where to look for?

[retep998]

As that info message quite helpfully mentioned, because you're doing that command under sudo, it will not fetch sources from the internet but instead rely on vendored sources. In order to not clog up this repository with all those vendored sources however, those sources are only vendored into the folder when requested. So you'll just have to call the appropriate command to tell the build system to vendor the sources and then you can do your sudo make install. Alternatively, you could realize that globally installing a Rust toolchain when you're using Rustup is probably a bad idea.

[itarato]

Thanks @retep998 . I've uninstalled the default rust installation (from rustup) are restarted the whole process and unfortunately ended up with the same error. If I run make install without sudo it fails due to permission issues in /usr/local. With sudo make install I go back to my original problem.
I roughly see what you're saying but I don't get what you mean by

you'll just have to call the appropriate command to tell the build system to vendor the sources

What is this command you're referring to?

Appreciate the help.

[itarato]

Without sudo I could compile if I set the destination dir, however I'm still curious of that command in order to be able to do it with sudo.

[retep998]

I believe it might be x.py dist which vendors the stuff so that it can create a tarball.

[itarato]

Appreciate @retep998 found it however I need to to do some digging how to call it in order to generate the missing vendor files. Thanks.

[retep998]

cc @alexcrichton

[est31]

Note that if you get the source code from the official tarballs, the vendor directory is populated: https://static.rust-lang.org/dist/2017-02-26/rust-src-nightly.tar.gz

src/vendor is just not checked into git, to not pollute the history as @retep998 said.

Of course, one could argue whether this is a bug or not, whether configure is supposed to access the internet, or whether having tools rely on the internet is a good thing, but we could certainly have a better error message, and possibly have a option to rely on the internet via the configure/make interface as well.

[steveklabnik]

/cc @eddyb who implemented this behavior

[eddyb]

@steveklabnik The sudo behavior predates my changes, seems that make install needs more things built than make, even if it really shouldn't. cc @rust-lang/tools

@itarato Does ./x.py dist --install work (without sudo)?

[itarato]

@eddyb I did a few tries with make install and ./x.py dist --install, both without sudo, and it works if I'm explicitly selecting a path with configure --prefix= or make DESTDIR=.. install. I assume they'd fail if I keep them on the default install dir (/usr/local).
This is fine for me as I both have the source in git (for contribution) and can compile. As well as in the meantime I've found https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md that explaines a bit more about the build process.
However my ocd is still troubled by the sudo make install not working (due to the missing vendor folder - which wasn't generated in any cases I mentioned at the beginning). It would be nice to have some explanation on how that scenario would work out. I'm happy to run some tests if you have ideas. Thanks again.

[eddyb]

It's not supposed to build anything make didn't, AFAIK, in order to not trigger this exact issue.
The vendor folder is not supposed to be there, nor should it be needed.

[alexcrichton]

@eddyb the guards here are very much intentional to prevent Cargo from spraying root-owned files in CARGO_HOME.

Without these guards there's a very high risk that Cargo decides to download crates from crates.io and cache them in CARGO_HOME, messing with the permissions there. The original thinking was that the vendored source code was always available so we'd just switch to using that.

@itarato thanks for the report! This is definitely a bug that we should fix.

[eddyb]

Oh, so no precautions were taken for sudo make install to not need to run cargo if make succeeded?