Repository navigation
Tracking Issue for proc_macro::{tracked_env, tracked_path} #99515
Description
Feature gate: #![feature(proc_macro_tracked_env, proc_macro_tracked_path)]
This is a tracking issue for proc_macro::tracked*, to allow adding files and environment variables to the build system's dependency tracking.
Public API
// proc_macro
mod tracked {
pub fn env_var<K: AsRef<OsStr> + AsRef<str>>(key: K) -> > Result<String, VarError>;
pub fn path<P: AsRef<Path>>(path: P);
}Steps / History
- Implementation of
tracked_env::var: proc_macro: Add API for tracked access to environment variables #74653 - Implementation of
tracked_path::path: addtrack_path::pathfn for usage inproc_macros #84029 - Add tracking issue: Use new tracking issue for proc_macro::tracked_*. #99516
- Change
pathto acceptAsRef<Path>instead ofAsRef<str> - Design the public API
- Documentation
- Final comment period (FCP)
- Stabilization PR
Unresolved Questions
- The full API design.
- The implementaiton PRs were mostly focussed on the implementation details behind the scenes of
proc_macro, but there's barely been any discussion on the public interface.
- The implementaiton PRs were mostly focussed on the implementation details behind the scenes of
Reacted by Guillaume Gomez, Finn Bear, Dave Patrick, Colton Donnelly, Connor Skees, Ho 229, cyqsimon, Koichi Akabe, Wojciech Daniło, ousado and 34 more
Reacted by Finn Bear, cyqsimon and Jonas
Reacted by Guillaume Gomez, Rahul Butani, Jonas and Antonin Delpeuch
Reacted by acheron and Niklas Wimmer
Reacted by cyqsimon and Jonas
Activity
m-ou-se commentedon Jul 20, 2022
This feature is not in a great state. The original PRs were mostly about the implementation details, with almost no discussion on the public API. There's still many unanswered design questions, such as naming, whether
pathshould return something (aFile? aString? aTokenStream? aPathBuf? nothing?), how it should be documented, how these functions should relate to each other, whether they should report errors through panics or throughResult, whether it should takeAsRef<OsStr>orAsRef<str>, and so on.Instead of discussing every single detail separately, it'd probably be best to start with a more complete proposal for how these APIs should look, and then discuss that.
(Usually most of this happens before an unstable feature is merged and tracked, but this has already been merged, hence this tracking issue.)
mitsuhiko commentedon Aug 28, 2022
Considering these APIs effectively mirror the
cargo:rerun-if-changedandcargo:rerun-if-env-changedfunctionality from build scripts I think it's reasonable to assume they don't return anything but just register interest.Right now the main reason you run into this issue is because someone already has functionality in a build script and wants to convert it into a
proc_macro. I would also strongly recommend for it to not report an error if the file does not exist as absence and presence of a file is both legitimate for proc macros the same way as it works for build scripts again.bjorn3 commentedon Aug 28, 2022
I don't think that is a good argument. It is relatively common to forget to register interest with build scripts I believe. In addition it is pretty common to write helper functions that register interest and at the same time read the file/env var to prevent forgetting this. The reason that no such function exists by default for build scripts is because there is no way for cargo to provide a crate to build scripts with said functions. Furthermore returning the value is essential for making proc macros work with virtual filesystems and with compiler session local env vars.
mitsuhiko commentedon Aug 28, 2022
Maybe? But that is probably a very different argument to begin with. Today where I'm standing there is functionality available with
build.rsthat you cannot have withproc_macroand the same arguments for the use of it withbuild.rsare equally appying to proc macros.By coupling these things together this has a high change of turning into a never ending argument because there will always be a disagreement about how this should work. In fact, even today you will find no agreement on how to best use the build script equivalents and if we had to come to an agreement about his this should work there, we probably wouldn't have solution there today with build scripts.
If there is a need for such an API it can always be added at a later point.
Of course there is. It would always have been possible for cargo to ship a utility library that is linked against build scripts the same way as
proc_macroexposes itself.That same argument can be brought up for build scripts.
bjorn3 commentedon Aug 28, 2022
The interest registering version can be trivially implemented in terms of the version that returns the value by discarding the return value, but not the other way around. In addition the interest registering versions may become entirely useless if we start sandboxing proc macros depending on the way we do this.
For build scripts it isn't all that important that changing a file in an editor without saving it reruns them. It is essential for proc macros. In addition rust-analyzer is already having a lot of trouble with proc macros going outside of their implied (unenforced) sandbox, while it doesn't have any issues with build scripts as those run in the same environment as with a regular cargo build due to rust-analyzer using cargo for invoking them.
mitsuhiko commentedon Aug 28, 2022
I want to leave another piece of input here that I believe are relevant:
rerun-if-changedtoday can track a directory and will in fact track changes to the contents. My expectation as a user would be that I can replicate the same functionality forproc_macros. This is in fact right now what I care about was I'm trying to build something that bundles folders.If returning contents turns out to be useful it can always be added as an additional API. I think coupling these things will make this feature stay in not stable land for a lot longer than if those discussions are separated.
est31 commentedon Dec 12, 2022
I think
tracked_path::pathshould be redesigned to also open read access to the specified path, as suggested by RFC 3200, because the current API leaves that question unresolved, which basically begs for the proc macro to use outside methods to access files, which would be a break of current policy that this is unsupported.wdanilo commentedon Feb 18, 2023
Just a note here that the current implementation does not work with relative paths. It works if I use absolute ones though.
ehuss commentedon Feb 18, 2023
@wdanilo Can you put together a reproduction? It seems to work fine for me. The path will be relative to cwd where rustc is run, which for Cargo is the workspace root (which matches the way a proc-macro would access relative paths).
wdanilo commentedon Feb 18, 2023
@ehuss Not this kind of relative path, sorry for not being clear enough here. In my case it was something along the lines of
/some/absolute/path/.../.../../../some/other/fileand it did not work until I runcanonicalizeon myPathBufhere. Could you tell me if that was enough for you to repro it, or should I try creating a minimal scene, please? :)ehuss commentedon Feb 19, 2023
Yes, please create a minimal reproduction if you can. Also, please post it as a new issue (and cc me). Tracking issues are not a good way to discuss individual issues.
usamoi commentedon Mar 29, 2024
Can I imitate the behaviors of these two APIs by adding
const _: Option<&str> = option_env!("tracked_env");andconst _: &str = include_bytes!("tracked_path");to returned token streams? Is it guaranteed that the env and the path are tracked even if they do not really make a difference to the program?27 remaining items
hanna-kruppe commentedon Oct 29, 2025
Make implementations (plural) are not the only consumers of dep-info files, though. For example, Cargo reads them, and Cargo generally doesn't support non-UTF-8 paths (e.g., rust-lang/cargo#6537). I've also written code parsing those files and guess what, that was using
Stringtoo.There is no general, portable solution to what the so-called Makefile problem. It's nice that Rust has the option of supporting
Pathwhen onlyPathmanipulation is needed, but as soon as a path is embedded into a larger text file, you're on thin ice at best.clarfonthey commentedon Nov 20, 2025
I've been interested in this because built-in macros can automatically do this if you call, for example,
include_str!orenv!, so, I think it's reasonable to not require people to add a cargo build script just to mirror that functionality.The current crate I've been working on that I'd like to use this is
include-pnm, although it currently doesn't use this functionality since it's not stable.(It also has its own problems because
rust-analyzer'sproc_macroserver doesn't implementfileorlocal_file, but that's separate from this issue entirely.)Skgland commentedon Nov 27, 2025
It was brought up in #87173 (comment) that how relative paths work might not be intuitive. Passing the same path to
proc_macro::tracked::pathandstd::fs::File::open/std::fs::read_dirmight result in a different file/directory being opened than is being tracked.So would then make sense for
proc_macro::tracked::pathto return the absolute path that is being tracked by rustc so that it can then be used to correctly access the path?I think this could also help somewhat to ensure that a path is not used before its tracked i.e. prevent check if it exists and only track it if it does.
clarfonthey commentedon Nov 27, 2025
this is actually wrong, but leaving for posterity
I would personally expect
tracked::pathto work based upon the current directory likeFile::opendoes, since the current directory for proc macros is the current directory of the source file in which the macro is called. So, for example, if you were to implementinclude_str!with a proc macro, you would wantFile::openon the string passed as an argument to load the file relative to the same source file, andtracked::pathto also track that same file.Skgland commentedon Nov 27, 2025
Makes sense but also makes implementing
include!like macros harder as I don't think it currently easy/possible for a proc_macro to determine the path that it should be relative to.clarfonthey commentedon Nov 27, 2025
Actually, I was completely wrong so I'm going to just state what I actually did when implementing this:
Span::call_site()gives you the span of the call site.span.local_file()gives you the path to the relevant source file.path.pop()followed bypath.push(file).Probably the simplest option semantically is to say that
local_filealways returns an absolute path, and thus, you need absolute paths fortracked::path. But it's not 100% clear whether we want that, because for example, what shouldtracked::path("/dev/stdin")do?As an aside, as I mentioned, rust-analyzer does not return anything via
local_file(it returnsOption, so, R-A just returnsNone), and I have to work around this to make things work. I think that we can currently ignore R-A here since I think in general folks will probably just need to do work around it like I have until they properly implement this API: just detect whenNoneis returned and return something type-checking-but-wrong from the macro so R-A doesn't detect an error that the compiler doesn't.Rollup merge of rust-lang#149400 - Skgland:tracked_mod, r=Amanieu
Amanieu commentedon Dec 9, 2025
The issue of relative paths was discussed in the @rust-lang/libs-api meeting last week. There was a strong consensus that this path should be interpreted the same way as
File::open, which is relative to the current working directory in which the proc macro process is executing.Unrolled build for #149400
Rollup merge of #149400 - Skgland:tracked_mod, r=Amanieu
Fix build on recent nightly
Skgland commentedon Apr 7, 2026
Amanieu commentedon Apr 7, 2026
Thanks! I've updated the issue description.