Add an after_expansion callback in the driver

[Xanewok]

To format a given file RLS needs to know the Rust edition associated with it. It's not enough to look at the edition key in Cargo.toml - each crate target can have a different edition associated with it so the sure way to fetch a correct edition is to scan the input files used to compile a given crate target.

Right now this was done in the after_analysis callback of our shim, however this leads to other problems - if a crate cannot be successfully compiled (e.g. it has a type error) then a callback would not be invoked meaning we can't populate the files -> edition mapping.

However, doing this only after parsing is not enough, since expansion can pull in additional source files (e.g. by invoking macro_rules! include_my_mod { () => { mod some_mod; }; }).

Without copy-pasting the entire driver setup it's also not possible to expand the crate ourselves in the after_parsing callback - to expand crate we'd need to register plugins and initialize session ourselves. However, this is done normally after executing the callback itself, thus triggering the Once::set assertions in Session::init_features.

r? @Zoxc

cc @RalfJung @oli-obk this affects public driver interface used by Miri and Clippy

[RalfJung]

this affects public driver interface used by Miri and Clippy

How hard is it to adjust existing drivers to the changed interface? Do we just have replace the bool by this Compilation? That's a welcome change actually. :) Thanks for the heads-up.

[Xanewok]

Addressed the feedback. Is this good now?

[Xanewok]

How hard is it to adjust existing drivers to the changed interface? Do we just have replace the bool by this Compilation?

Yup, that's it! Figured it'd be more readable than bool, I found myself commenting

// Continue execution
true

so a Compilation::Continue should hopefully be more readable here.

[Zoxc]

@bors r+

[bors]

📌 Commit ff63336 has been approved by Zoxc

[bors]

⌛ Testing commit ff63336 with merge e8e0c666bad1a6e0af6a11fcff0db39a881e04a0...

[bors]

💔 Test failed - checks-azure

[rust-highfive]

The job x86_64-msvc-cargo of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-07-18T12:18:01.0202751Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-18T12:18:01.0203247Z 
2019-07-18T12:18:01.0203541Z   git checkout -b <new-branch-name>
2019-07-18T12:18:01.0203697Z 
2019-07-18T12:18:01.0203882Z HEAD is now at e8e0c666b Auto merge of #62679 - Xanewok:after-expansion, r=Zoxc
2019-07-18T12:18:01.0569364Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-18T12:18:01.0691851Z ==============================================================================
2019-07-18T12:18:01.0691925Z Task         : Bash
2019-07-18T12:18:01.0691998Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-18T13:07:24.6996374Z [551/2435] Linking CXX executable bin\llvm-tblgen.exe
2019-07-18T13:07:24.7449352Z [552/2435] Building Options.inc...
2019-07-18T13:07:24.7618049Z [553/2435] Building Options.inc...
2019-07-18T13:07:25.0875071Z [554/2435] Building CXX object lib\ToolDrivers\llvm-lib\CMakeFiles\LLVMLibDriver.dir\LibDriver.cpp.obj
2019-07-18T13:07:25.0875847Z FAILED: lib/ToolDrivers/llvm-lib/CMakeFiles/LLVMLibDriver.dir/LibDriver.cpp.obj 
2019-07-18T13:07:25.0876541Z D:\a\1\s\build\bootstrap\debug\sccache-plus-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_DEBUG_POINTER_IMPL="" -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib\ToolDrivers\llvm-lib -ID:\a\1\s\src\llvm-project\llvm\lib\ToolDrivers\llvm-lib -Iinclude -ID:\a\1\s\src\llvm-project\llvm\include /nologo /MT /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /Brepro /W4  -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wno-noexcept-type -Wno-comment /MT /O2 /Ob2   -UNDEBUG  /EHs-c- /GR- /showIncludes /Folib\ToolDrivers\llvm-lib\CMakeFiles\LLVMLibDriver.dir\LibDriver.cpp.obj /Fdlib\ToolDrivers\llvm-lib\CMakeFiles\LLVMLibDriver.dir\LLVMLibDriver.pdb -c D:\a\1\s\src\llvm-project\llvm\lib\ToolDrivers\llvm-lib\LibDriver.cpp
2019-07-18T13:07:25.0877227Z clang-cl.exe: warning: argument unused during compilation: '-mno-incremental-linker-compatible' [-Wunused-command-line-argument]
2019-07-18T13:07:25.0877455Z In file included from D:\a\1\s\src\llvm-project\llvm\lib\ToolDrivers\llvm-lib\LibDriver.cpp:18:
2019-07-18T13:07:25.0877686Z In file included from D:\a\1\s\src\llvm-project\llvm\include\llvm/Bitcode/BitcodeReader.h:19:
2019-07-18T13:07:25.0878009Z In file included from D:\a\1\s\src\llvm-project\llvm\include\llvm/IR/ModuleSummaryIndex.h:27:
2019-07-18T13:07:25.0878155Z In file included from D:\a\1\s\src\llvm-project\llvm\include\llvm/IR/Module.h:23:
2019-07-18T13:07:25.0878301Z D:\a\1\s\src\llvm-project\llvm\include\llvm/IR/Attributes.h(73,14):  fatal error: 'llvm/IR/Attributes.inc' file not found
2019-07-18T13:07:25.0878387Z     #include "llvm/IR/Attributes.inc"
2019-07-18T13:07:25.0878720Z 1 error generated.
2019-07-18T13:07:25.1652973Z [555/2435] Building CXX object lib\ToolDrivers\llvm-dlltool\CMakeFiles\LLVMDlltoolDriver.dir\DlltoolDriver.cpp.obj
2019-07-18T13:07:25.1652973Z [555/2435] Building CXX object lib\ToolDrivers\llvm-dlltool\CMakeFiles\LLVMDlltoolDriver.dir\DlltoolDriver.cpp.obj
2019-07-18T13:07:25.1655529Z ninja: build stopped: subcommand failed.
2019-07-18T13:07:25.1760604Z command did not execute successfully, got: exit code: 1
2019-07-18T13:07:25.1760645Z 
2019-07-18T13:07:25.1760645Z 
2019-07-18T13:07:25.1760718Z build script failed, must exit now', C:\Program Files\Rust\.cargo\registry\src\github.com-1ecc6299db9ec823\cmake-0.1.38\src\lib.rs:813:5
2019-07-18T13:07:25.1765696Z  finished in 175.807
2019-07-18T13:07:25.1804949Z failed to run: D:\a\1\s\build\bootstrap\debug\bootstrap test src/tools/cargotest src/tools/cargo
2019-07-18T13:07:25.1805066Z Build completed unsuccessfully in 0:37:38
2019-07-18T13:07:25.1805066Z Build completed unsuccessfully in 0:37:38
2019-07-18T13:07:25.2576867Z ##[error]Bash exited with code '1'.
2019-07-18T13:07:25.3017895Z ##[section]Starting: Upload CPU usage statistics
2019-07-18T13:07:25.3119827Z ==============================================================================
2019-07-18T13:07:25.3119902Z Task         : Bash
2019-07-18T13:07:25.3119975Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Xanewok]

2019-07-18T13:07:25.0878301Z D:\a\1\s\src\llvm-project\llvm\include\llvm/IR/Attributes.h(73,14):  fatal error: 'llvm/IR/Attributes.inc' file not found

Seems spurious?

@bors retry

[bors]

⌛ Testing commit ff63336 with merge e1693a9700e87126a97497ccf3ccc582ed2231f4...

[bors]

⌛ Testing commit ff63336 with merge 527dce7...

[bors]

☀️ Test successful - checks-azure
Approved by: Zoxc
Pushing 527dce7 to master...

[rust-highfive]

📣 Toolstate changed by #62679!

Tested on commit 527dce7.
Direct link to PR: #62679

💔 clippy-driver on windows: test-fail → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 clippy-driver on linux: test-fail → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 miri on windows: test-fail → build-fail (cc @oli-obk @RalfJung @eddyb, @rust-lang/infra).
💔 miri on linux: test-fail → build-fail (cc @oli-obk @RalfJung @eddyb, @rust-lang/infra).
💔 rls on windows: test-pass → build-fail (cc @Xanewok, @rust-lang/infra).
💔 rls on linux: test-pass → build-fail (cc @Xanewok, @rust-lang/infra).