Surface multiple "can't find crate" errors at a time

[cfredric]

It would be nice if rustc could surface multiple "can't find crate xxxx" errors at once, instead of one at a time.

Given the following code: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6a10dfbf5f9cc7dc6488fc9c3efaec70

extern crate asdf;
extern crate foobar;

fn main() {}

The current output is:

error[E0463]: can't find crate for `asdf`
 --> src/main.rs:2:1
  |
2 | extern crate asdf;
  | ^^^^^^^^^^^^^^^^^^ can't find crate

For more information about this error, try `rustc --explain E0463`.
error: could not compile `playground` due to previous error

Ideally the output should look like:

error[E0463]: can't find crate for `asdf`
 --> src/main.rs:2:1
  |
2 | extern crate asdf;
  | ^^^^^^^^^^^^^^^^^^ can't find crate

For more information about this error, try `rustc --explain E0463`.
error[E0463]: can't find crate for `foobar`
 --> src/main.rs:3:1
  |
3 | extern crate foobar;
  | ^^^^^^^^^^^^^^^^^^^^ can't find crate

For more information about this error, try `rustc --explain E0463`.
error: could not compile `playground` due to previous error

[jyn514]

I think this should be as simple as removing the abort_if_errors() call in the resolver:

rust/compiler/rustc_metadata/src/locator.rs

Line 1214 in 495322d

sess.abort_if_errors();

[jyn514]

You'll also have to find some reasonable behavior to do for the places currently calling report() and expecting it to never return:

../rust/compiler/rustc_metadata/src/creader.rs
519:            err.report(&self.sess, span, missing_core)

../rust/compiler/rustc_metadata/src/locator.rs
780:        Err(err) => err.report(sess, span, false),

@petrochenkov may have suggestions.

[Manishearth]

I think this should be as simple as removing the abort_if_errors() call in the resolver:

Not so sure, most such calls are introduced because it's tricky to have the compiler keep marching on after it already knows it's operating on bad code. For example, the error for an extern crate serdy typo would get drowned out by a million "cannot find serde::Serialize" errors.

It might be worth getting the resolver to continue looking for crates but nothing else (or something similar? idk).

cc @estebank who probably has thoughts on how to correctly get the compiler to keep marching on after seeing errors that are the upstream cause of a lot of other errors that would be emitted if it kept naïvely marching on. I know we do this well in a bunch of places.

[mcy]

At a minimum it would be ok to:

• Halt macro expansion and discard all macro nodes.
• Discard all type-defining items.
• Recurse only into items inside of function, static, const, and impl bodies.

Our only hope here is probably to search for further extern crates, and use statements that are rooted in crates we've managed to resolve... not sure about use crate:: in this scenario. At a minimum, being able to diagnose

extern crate serdy;
mod ohno {
  extern crate hypur;
}
const _: () = { extern crate aho_korasic; }

would be nice...

(I say this having ~little knowledge of the HIR.)

[mjptree]

@jyn514 I'm looking for a beginner friendly mentored issue to start. Is it okay to claim this one?

[jyn514]

@mjptree feel free to claim this :) but I'm not sure how easy it will be.

[mjptree]

@rustbot claim

[estebank]

In general, when we encounter an error in a type or during parsing we try and introduce a "poison pill" for its corresponding id (DefId, generally) or to the node being computed. This way we can check for these cases and silence errors. Otherwise rustc would complain about trait Foo is not implemented for [type error]. In the case of name resolution I'm pretty sure we already account for failed use, but we are likely not accounting for extern crate, simply because it's less prevalent since 2018 and the efforts to silence this errors (without aborting) happened after that edition landed.

So the file that you likely want to start looking at is https://github.com/rust-lang/rust/blob/9f6ca7482ca885bca14ba6870ed39a783ab939b5/compiler/rustc_resolve/src/lib.rs

On a quick glance, fn record_use shows some special casing that we perform for some prelude entries.

Then I would look at

rust/compiler/rustc_resolve/src/build_reduced_graph.rs

Line 685 in 9f6ca74

ItemKind::ExternCrate(orig_name) => {

particularly

rust/compiler/rustc_resolve/src/build_reduced_graph.rs

Lines 701 to 707 in 9f6ca74

	let crate_id = self.r.crate_loader.process_extern_crate(
	item,
	&self.r.definitions,
	local_def_id,
	);
	self.r.extern_crate_map.insert(local_def_id, crate_id);
	self.r.expect_module(crate_id.as_def_id())

digging into it brought me to

rust/compiler/rustc_metadata/src/creader.rs

Line 1014 in 2834f57

let cnum = self.resolve_crate(name, item.span, dep_kind);

which led me to

rust/compiler/rustc_metadata/src/creader.rs

Lines 510 to 522 in 2834f57

	fn resolve_crate<'b>(
	&'b mut self,
	name: Symbol,
	span: Span,
	dep_kind: CrateDepKind,
	) -> CrateNum {
	self.used_extern_options.insert(name);
	self.maybe_resolve_crate(name, dep_kind, None).unwrap_or_else(|err| {
	let missing_core =
	self.maybe_resolve_crate(sym::core, CrateDepKind::Explicit, None).is_err();
	err.report(&self.sess, span, missing_core)
	})
	}

which is where we are causing the error to be emitted today. We must instead report and continue. To do so, I would try and change the call to resolve_crate with a call to maybe_resolve_crate (provably move the self.used_extern_options.insert(name); to it as well), and create a new ModuleData for unresolved crates (look at what fn get_module is doing).

Ping me if you need more context!

[mjptree]

@estebank Thank you so much for the information! This is really helpful. I ran into some rather unexpected problems yesterday trying to set up my working environment and spent my evening puzzling over some spurious build errors on a freshly cloned repo. I blame it on me having already updated to Windows 11 for now and tackle it one issue at a time, beginning with this one that I committed to :).

I tried to start at

rust/compiler/rustc_metadata/src/locator.rs

Line 934 in d212d90

crate fn report(self, sess: &Session, span: Span, missing_core: bool) -> ! {

like jyn514 already mentioned, and walk backwards from there to all invocations, that could cause a CrateError::report and a subsequent abort.

Like you, I arrived also in

rust/compiler/rustc_metadata/src/creader.rs

Lines 510 to 522 in 2834f57

	fn resolve_crate<'b>(
	&'b mut self,
	name: Symbol,
	span: Span,
	dep_kind: CrateDepKind,
	) -> CrateNum {
	self.used_extern_options.insert(name);
	self.maybe_resolve_crate(name, dep_kind, None).unwrap_or_else(|err| {
	let missing_core =
	self.maybe_resolve_crate(sym::core, CrateDepKind::Explicit, None).is_err();
	err.report(&self.sess, span, missing_core)
	})
	}

which is where we are causing the error to be emitted today.

and also interestingly (or less so - can't judge at this point)

rust/compiler/rustc_metadata/src/locator.rs

Lines 811 to 822 in d212d90

	pub fn find_plugin_registrar(
	sess: &Session,
	metadata_loader: &dyn MetadataLoader,
	span: Span,
	name: Symbol,
	) -> PathBuf {
	match find_plugin_registrar_impl(sess, metadata_loader, name) {
	Ok(res) => res,
	// `core` is always available if we got as far as loading plugins.
	Err(err) => err.report(sess, span, false),
	}
	}

which is probably less relevant, because as far as I can tell

rust/compiler/rustc_plugin_impl/src/load.rs

Lines 27 to 61 in d212d90

	pub fn load_plugins(
	sess: &Session,
	metadata_loader: &dyn MetadataLoader,
	krate: &Crate,
	) -> Vec<PluginRegistrarFn> {
	let mut plugins = Vec::new();
	for attr in &krate.attrs {
	if !attr.has_name(sym::plugin) {
	continue;
	}
	for plugin in attr.meta_item_list().unwrap_or_default() {
	match plugin.ident() {
	Some(ident) if plugin.is_word() => {
	load_plugin(&mut plugins, sess, metadata_loader, ident)
	}
	_ => call_malformed_plugin_attribute(sess, plugin.span()),
	}
	}
	}
	plugins
	}
	fn load_plugin(
	plugins: &mut Vec<PluginRegistrarFn>,
	sess: &Session,
	metadata_loader: &dyn MetadataLoader,
	ident: Ident,
	) {
	let lib = locator::find_plugin_registrar(sess, metadata_loader, ident.span, ident.name);
	let fun = dylink_registrar(sess, ident.span, lib);
	plugins.push(fun);
	}

these functions are not called anywhere at the moment. Does "plugins" in this context refer to proc-macros?

I'll have a closer look at

rust/compiler/rustc_metadata/src/creader.rs

Line 1014 in 2834f57

let cnum = self.resolve_crate(name, item.span, dep_kind);

as you mentioned, as it looks the most promising.

It would be good to see an applied example of the "poison pill" mechanism that you mentioned, in a similar context. Also, I'm not familiar with this crate fn syntax. Is it equivalent to pub(crate) fn?

[estebank]

Plugins were the way you could modify how the compiler behaved before proc-macros were designed. They are still in the codebase, but they were never in a path to stabilization. They worked similarly in rustc, but had full access to the compiler internals, in contrast with proc-macros which only have access to the TokenStream.

By "poison pill" I mean something like Err in the following enum:

enum TyKind {
    Adt(Def, Substs),
    Str,
    Infer(InferTy),
    /// Placeholder used to silence errors
    Err,
}

crate fn is equivalent to pub(crate) fn, but the later works on all editions, while the former works on 2018 onwards. They mean the same.