Need a reliable way to get the target dir from the build script

[kennykerr]

Challenges: #9661 (comment)

Use cases (with alternatives): #9661 (comment)

Original Issue

The problem: I need to locate the target dir from the build script.

The solution: I'd love a CARGO_TARGET_DIR environment variable. There's clearly precedent for this as cargo doc depends on such a thing today.

@jyn514 suggested I reach out to the Cargo team for advice.

Currently the Windows crate has been using various hacky solutions that are problematic.

rust-lang/docs.rs#1444 (comment)

[kennykerr]

And grabbing the target_directory reported by "cargo metadata" doesn't work. If the crate is coming from crates.io then Cargo builds it in C:\Users\kenny\.cargo\registry\src\<random>\<some-crate> and the "target_directory" that cargo metadata reports is C:\Users\kenny\.cargo\registry\src\<random>\<some-crate>\target - what I need is the target directory for the crate the developer is building directly, not an intermediate target directly only known to the dependency.

[zeerooth]

I've been recently working on a project where there's need to link the generated binary in user project to another directory and I couldn't find a reliable way to get the target directory either.
Currently, the code just guesses that the target directory is set as default <project dir>/target/ but if the project is located in a workspace or the build.target-dir is set to something else then this linking simply fails.
Such feature to get the target dir reliably would be greatly appreciated ❤️

[jyn514]

The solution: I'd love a CARGO_TARGET_DIR environment variable. There's clearly precedent for this as cargo doc depends on such a thing today.

This has cause and effect backwards. cargo doc works fine without CARGO_TARGET_DIR set in the environment. Docs.rs uses that as an input to cargo because the source directory is sandboxed, and cargo happens to pass it through to any build scripts it invokes.

[kennykerr]

Yes, I since noticed that this is an optional input to Cargo:

https://doc.rust-lang.org/cargo/reference/environment-variables.html

Indeed, that's not what I want. I really need the target dir for the most dependent crate being directly built. I've tried numerous unsatisfying hacks. The one that works the most reliably is to read the first path from the PATH environment variable. This always appears to be:

<target_dir>\<profile>\deps

Although this only works on Windows. And I don't know how "supported" this is but I haven't found a better solution.

[ehuss]

Can you provide more information on exactly why you need the target directory? Cargo is designed so that build scripts are intentionally constrained on what they should do, and their only interaction should be through the OUT_DIR.

[riverar]

@ehuss Kenny is out this week but some scenarios are listed here microsoft/windows-rs#979 (comment).

I'll take a stab here at paraphrasing:

The windows crate has a build script that generates bindings for the developer, sourcing from its built-in metadata or local workspace metadata. Things get tricky when this metadata is delivered via multiple crates in the dependency graph.

For example, envision a developer's crate depending on windows, windows-fizz, and windows-buzz each with their own unique metadata. The build script (macro) running in the context of the developer's crate needs to have access to all the dependent crates' metadata to generate the developer's desired bindings.

One proposed change is to align the normal build script behavior with the cargo doc build script behavior. CARGO_TARGET_DIR would then exist and point to the developer's crate target directory and all dependent crates could write their metadata to that path for consumption.

One proposed change is to provide a pointer of sorts to the developer's crate target directory and all dependent crates could write their metadata to that path for consumption. Open to other ideas, of course.

[jyn514]

One proposed change is to align the normal build script behavior with the cargo doc build script behavior.

@riverar you're making the same mistake as #9661 (comment), cargo doc is not related to the target dir. Docs.rs just happens to set CARGO_TARGET_DIR and you've never tried setting it locally.

[riverar]

@jyn514 Ah right, forgot. Thanks!

This doesn't affect the proposed change much. The bottom line is we'd like a pointer to the target directory for the aforementioned reasons.

[matklad]

One use-case I have is to store build-script's own caches. I can use a temporary directory as a scratch space (though it might still be nice to place this somewhere inside ./target rather than /tmp), I can use OUT_DIR for a real output, but I am not sure what's the right place for intermediate artifacts generated by the build script, which are not necessary output, but which I nonetheless want to preserve between invocations.

For example, if I am building some C code, I want to put intermediate .o files somewhere. Or, my specific use-case, I recursively invoke Cargo to compile some helpers to wasm. I mistakenly tried to use target dir for that, but that broke in all kinds of interesting ways down the line. Looking at how cc crate just puts .o into OUT_DIR, I think it's correct to use that for intermediate artifacts as well, but it isn't obvious from the docs. Will send a PR shortly (#10326).