Downloading source

[ehuss]

This issue is for working through the implementation issues with downloading the standard library source.

Rustup is currently capable of downloading the rust-src component which contains the source for the standard library crates. It does not include dependencies (like libc) but it does include Cargo.lock from which dependencies can be inferred.

There are multiple questions on how Cargo will handle acquiring the standard library source:

• I would prefer if it didn't solely rely on rustup being installed, though what the alternative is I do not know.
• Acquiring dependencies may be tricky. Likely the Cargo.lock file will need to be parsed to find the versions, and then download from crates.io. Some possible alternatives:
  • Include all dependencies in the original source download.
  • Record dependency versions in a different format than Cargo.lock.

[alexcrichton]

For this to be a truly first class feature I think that this download needs to be handled transparently. Cargo already has tons of support for downloading things but the major trickiness will be finding where the source code is hosted. Rustup knows the toolchain and commit (especially for things like rustup toolchain install master), but cargo generally doesn't know this.

It I think would be a reasonable first shot though to say cargo has a default probing location (for Linux distroa) and it then attempts to query rustup. All of that could have a manual override as well, and I think that would roughly cover most setups at the start

[SimonSapin]

Two random ideas for specific mechanims:

• If the argv[0] for Cargo (not rustup’s proxy) is $PREFIX/bin/cargo, look for source code at $PREFIX/lib/rustlib/src/rust. This is where rustup installs its rust-src components. Maybe it’s reasonable to declare that non-rustup installers / packaging should have a similar directory layout? (Possibly with tweaks for Windows or other platforms where bin and lib directory are not usual.)

• Rustup has a ~/.cargo/bin/cargo proxy executable that decides which toolchain to use, then calls the "real" cargo executable. It could at that time set a RUST_SRC_PATH environment variable that Cargo could read without making a query back to rustup.

[Ericson2314]

In https://github.com/rust-lang/cargo/pull/2768/files I just put in a config setting for a path source/registry for the stdlib sources. Simple and stupid, but fine for MVP I'd hope.

[crlf0710]

A (maybe stupid) approach after MVP is to reserve core, std, etc... on crates.io, and publish the source of every version of libcore and libstd on it as 0.0 for rustc 1.0.0, 1.0 for rustc 1.1.0, ... 37.0 for rustc 1.37.0, etc.

[Ericson2314]

I wouldn't mind that at all. Nightly and alternate implementations can use [patch].

[ehuss]

If using the rustup-downloaded component is used long-term, then we need to use the cached value of rustc --print=sysroot. The problem is that TargetInfo is created too late, so BuildConfig will need to be restructured so that it can be created earlier.

[SimonSapin]

Some names are already reserved on crates.io. However I don’t think this is a good strategy: we should aim to support not only official releases and Nightlies from rust-lang.org, but also toolchains compiled elsewhere for various reasons.

[madsmtm]

we should aim to support not only official releases and Nightlies from rust-lang.org, but also toolchains compiled elsewhere for various reasons.

As a user, I'd expect that to happen using [crates-io.patch], though of course that's probably difficult to actually implement.