Index out of bounds when running `cargo doc` in `rustc_metadata/src/creader.rs`

[XAMPPRocky]

EDIT(camelid): This issue is now being used to track a different, but related, bug. The discussion starting here is current.

Steps

• Clone rust-gpu.
• cargo doc -p spirv-std --no-deps

Meta

rustc --version --verbose:

rustc 1.53.0-nightly (42816d61e 2021-04-24)
binary: rustc
commit-hash: 42816d61ead7e46d462df997958ccfd514f8c21c
commit-date: 2021-04-24
host: x86_64-apple-darwin
release: 1.53.0-nightly
LLVM version: 12.0.0

Error output

thread 'rustc' panicked at 'index out of bounds: the len is 8 but the index is 8', compiler/rustc_metadata/src/creader.rs:137:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

error: Unrecognized option: 'crate-version'

error: could not document `spirv-std`

Caused by:
  process didn't exit successfully: `rustdoc --edition=2018 --crate-type lib --crate-name spirv_std crates/spirv-std/src/lib.rs -o /Users/erin.power/src/rust-gpu2/target/doc --cfg 'feature="default"' --error-format=json --json=diagnostic-rendered-ansi -L dependency=/Users/erin.power/src/rust-gpu2/target/debug/deps --extern bitflags=/Users/erin.power/src/rust-gpu2/target/debug/deps/libbitflags-7c13a8e06c86cae3.rmeta --extern num_traits=/Users/erin.power/src/rust-gpu2/target/debug/deps/libnum_traits-226168b7b64579d8.rmeta --extern spirv_std_macros=/Users/erin.power/src/rust-gpu2/target/debug/deps/libspirv_std_macros-63f1f4de237cb2bc.dylib --extern spirv_types=/Users/erin.power/src/rust-gpu2/target/debug/deps/libspirv_types-7559b6c8fd9bbf59.rmeta --crate-version 0.4.0-alpha.7` (exit status: 1)

Backtrace

   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic_bounds_check
   3: rustc_metadata::rmeta::decoder::cstore_impl::provide_extern::crate_name
   4: rustc_query_system::dep_graph::graph::DepGraph<K>::with_task_impl
   5: rustc_data_structures::stack::ensure_sufficient_stack
   6: rustc_query_system::query::plumbing::force_query_with_job
   7: rustc_query_system::query::plumbing::get_query_impl
   8: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::crate_name
   9: rustdoc::clean::inline::record_extern_fqn
  10: rustdoc::clean::utils::register_res
  11: rustdoc::passes::collect_intra_doc_links::LinkCollector::resolve_link
  12: <rustdoc::passes::collect_intra_doc_links::LinkCollector as rustdoc::fold::DocFolder>::fold_item
  13: alloc::vec::source_iter_marker::<impl alloc::vec::spec_from_iter::SpecFromIter<T,I> for alloc::vec::Vec<T>>::from_iter
  14: rustdoc::fold::DocFolder::fold_inner_recur
  15: rustdoc::fold::DocFolder::fold_item_recur
  16: <rustdoc::passes::collect_intra_doc_links::LinkCollector as rustdoc::fold::DocFolder>::fold_item
  17: rustdoc::passes::collect_intra_doc_links::collect_intra_doc_links
  18: rustdoc::core::run_global_ctxt
  19: rustc_interface::passes::QueryContext::enter
  20: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  21: rustc_span::with_source_map
  22: rustc_interface::interface::create_compiler_and_run
  23: rustdoc::main_options
  24: scoped_tls::ScopedKey<T>::set

[jyn514]

@XAMPPRocky this is probably related to #83761 somehow. I don't plan to investigate any of the proximate issues until the underlying problem is fixed. Feel free to take a look yourself if this is blocking your work, but it won't be easy.

[jyn514]

Oh, I guess this could also be a missed case from #83738. That should be easier to fix.

[jyn514]

If you can come up with an MCVE it would be easier to tell which of the two issues it is.

[mautamu]

Not exactly an MCVE, but I think I'm able to reproduce this with only a handful of files @jyn514. The line
thread 'rustc' panicked at 'index out of bounds: the len is 8 but the index is 8', compiler/rustc_metadata/src/creader.rs:137:21
does change slightly though:
thread 'rustc' panicked at 'index out of bounds: the len is 20 but the index is 20', compiler/rustc_metadata/src/creader.rs:137:21

https://github.com/mautamu/spirv-std-3

(Same rustc version output as above).

Hope this helps :).

Best,
Mautamu

[jyn514]

DEBUG rustdoc::passes::collect_intra_doc_links combined_docs=[ImageFormat]: spirv_types::image_params::ImageFormat
DEBUG rustdoc::passes::collect_intra_doc_links resolving spirv_types::image_params::ImageFormat as a macro in the module DefId(19:0 ~ spirv_std_macros[a0e2])

I think the issue is that the crate is first used in a dependency, so it isn't loaded in

rust/src/librustdoc/passes/collect_intra_doc_links/early.rs

Lines 30 to 31 in e327a82

	if let Some(doc) = attr.doc_str() {
	for link in markdown_links(&doc.as_str()) {

Maybe a fix is to recursively traverse the docs of re-exports? That sounds kind of expensive, but the only other option (besides fixing #83761) is to revert and re-open #68427.