rustc_errors: Add (heuristic) Syntax Highlighting for `rustc --explain` #150895

JayanAXHF · 2026-01-09T21:09:53Z

This PR adds a feature that enables rustc --explain <error> to have syntax highlighted code blocks. Due to performance, size and complexity constraints, the highlighter is very heuristc, relying on conventions for capitalizations and such to infer what an identifier represents. The details for the implementation are specified below.

Changes

Change term::entrypoint to term::entrypoint_with_formatter, which takes an optional third argument, which is a function pointer to a formatter. (compiler/rustc_errors/src/markdown/mod.rs)
Change MdStream::write_anstream_buf to be a wrapper around a new function, MdStream::write_anstream_buf_with_formatter, which takes a function pointer to a formatter. (compiler/rustc_errors/src/markdown/mod.rs)
Change compiler/rustc_driver_impl/src/lib.rs to call MdStream::write_anstream_buf_with_formatter instead of MdStream::write_anstream_buf.
Add a compiler/rustc_driver_impl/src/highlighter.rs file, which contains the actual syntax highlighter.

Implementation Details

The highlighter starts from the highlight function defined in compiler/rustc_driver_impl/src/highlighter.rs. It creates a new instance of the Highlighter struct, and calls its highlight_rustc_lexer function to start highlighting.
The highlight_rustc_lexer function uses rustc_lexer to lex the code into Tokens. rustc_lexer was chosen since it preserves the newlines after scanning.
Based on the kind of token (TokenKind), we color the corresponding lexeme.

Highlighter Implementation

Identifiers

All identifiers that match a (non-exhaustive and minimal) list of keywords are coloured magenta.
An identifier that begins with a capital letter is assumed as a type. There is no distinction between a Trait and a type, since that would involve name resolution, and the parts of rustc that perform name resolution on code do not preserve the original formatting. (An attempt to use rustc_parse's lexer and TokenStream was made, which was then printed with the pretty printer, but failed to preserve the formatting and was generally more complex to work with)
An identifier that is immediately followed by a parenthesis is recognized as a function identifier, and coloured blue.

Literals

A String literal (or its corresponding Raw, C and Byte versions) is colored green.
All other literals are colored bright red (orange-esque)

Everything Else

Everything else is colored bright white and dimmed, to create a grayish colour.

Demo

Command: rustc --explain E0520

This description was not generated by an LLM (:p)

cc: @bjorn3

Changed `highlight_rustc_lexer` to have only one place where the original lexeme is extracted. This reduces dereferences and slices. Very minor perf change.

rustbot · 2026-01-09T21:09:56Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

rustbot · 2026-01-09T21:09:59Z

r? @fee1-dead

rustbot has assigned @fee1-dead.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

JayanAXHF · 2026-01-09T21:13:53Z

PS (Link to Zulip Topic) : #t-compiler/diagnostics > Colored output for `rustc --explain`

Kivooeo · 2026-01-09T21:17:04Z

Is this change requires MCP?

JayanAXHF · 2026-01-09T21:19:04Z

Is this change requires MCP?

Could you clarify what you mean by MCP? I'm very new to rustc (this is my first contribution)

Kivooeo · 2026-01-09T21:20:07Z

A process where you propose a change and other members brainstorm it whether it worth adding or not

Kivooeo · 2026-01-09T21:20:46Z

https://forge.rust-lang.org/compiler/proposals-and-stabilization.html

JayanAXHF · 2026-01-09T21:23:28Z

A process where you propose a change and other members brainstorm it whether it worth adding or not

Oh thanks. I don't think that this qualifies as a major change. It just adds some very simple code that basically colours tokens by matching on its kind and lexeme. IMO it falls in the light-weight proposal category, as it doesn't break anything (if it errors, the plain block is printed)

tgross35 · 2026-01-09T21:33:22Z

I don't think this needs an MCP, it's a reasonably local change that could be backed out at any time.

Kivooeo · 2026-01-09T21:34:35Z

r? me

tgross35 · 2026-01-09T21:44:27Z

Fyi this will need a test somehow, a UI test is probably fine

JayanAXHF · 2026-01-09T21:46:32Z

Fyi this will need a test somehow, a UI test is probably fine

The test could probably me giving it a string of code and comparing its colored output to what was expected.

JayanAXHF · 2026-01-09T21:50:47Z

The job aarch64-gnu-llvm-20-2 failed! Check out the build log: (web) (plain enhanced) (plain)

The existing tests need to be updated

tgross35 · 2026-01-09T21:58:06Z

IIRC that test can be updated with RUSTC_BLESS=1

JayanAXHF · 2026-01-09T21:59:20Z

IIRC that test can be updated with RUSTC_BLESS=1

Could you elaborate please (I'm sorry I'm new to the codebase)

tgross35 · 2026-01-09T22:07:45Z

Sorry, you don't need to set that env - just run --bless when you run the tests locally (./x t <relevant path>).

rust/compiler/rustc_errors/src/markdown/tests/term.rs

Line 65 in a3f2d5a

let bless = std::env::var_os("RUSTC_BLESS").is_some_and(|v| v != "0");

. Just means to auto update test that check output and can be annoying to update manually, so you can look at the diff (of course, doesn't mean it is correct!).

Not actually sure why that test failed, the code formatter isn't hooked there.

compiler/rustc_errors/src/markdown/term.rs

The problem was something related to the fast that we use a `Style::new` default style when no style is matched

JayanAXHF · 2026-01-10T07:00:24Z

I have fixed the failing test.

JayanAXHF · 2026-01-10T07:08:13Z

Fyi this will need a test somehow, a UI test is probably fine

Where would I create the test?

Kivooeo · 2026-01-10T17:21:16Z

Can we consider colouring all primitives types as well? The same way it was made for keywords

https://doc.rust-lang.org/rust-by-example/primitives.html

Kivooeo · 2026-01-10T17:28:03Z

Can write_anstream_buf now be deleted? I don't see any reasons to keep it for now, since it just calls write_anstream_buf_with_formatter, why not call it directly

Same for write_stream

JayanAXHF · 2026-01-10T17:35:52Z

Can we consider colouring all primitives types as well? The same way it was made for keywords

https://doc.rust-lang.org/rust-by-example/primitives.html

Sure!

JayanAXHF · 2026-01-10T17:38:07Z

Same for write_stream

This would require having to change stuff in a few places (in the same file i believe)

Can write_anstream_buf now be deleted? I don't see any reasons to keep it for now, since it just calls write_anstream_buf_with_formatter, why not call it directly

This can done, since iirc write_anstream_buf is not called anywhere except the current test (which can be updated)

Kivooeo · 2026-01-10T18:56:32Z

This would require having to change stuff in a few places

Feel free to do this changes

Removed unnecessary wrapper around `write_anstream_buf_with_formatter` and renamed it to `write_anstream_buf`.

JayanAXHF · 2026-01-10T21:07:59Z

This would require having to change stuff in a few places

Feel free to do this changes

Done in #b84220 and #34f6890!

cc: @Kivooeo

Kivooeo

All changes looks good to me!

There's just a small typo. Also, I need some more time to think about whether we need to create a test, and if so, how

View changes since this review

compiler/rustc_driver_impl/src/highlighter.rs

Co-authored-by: Kivooeo <[email protected]>

JayanAXHF · 2026-01-10T21:24:42Z

All changes looks good to me!

Wonderful! I'll push the highlighting for the primitive types tomorrow.

There's just a small typo.

Fixed

Also, I need some more time to think about whether we need to create a test, and if so, how

Please, take your time :D

tgross35 · 2026-01-10T23:57:08Z

There's just a small typo. Also, I need some more time to think about whether we need to create a test, and if so, how

Yes any kind of behavior change generally needs a test. Why the green check if you're not yet sure :)

@JayanAXHF the test can be the same as tests/ui/explain/basic.rs but adding --color=always. Also please squash the commits before merge.

Kivooeo · 2026-01-11T00:16:03Z

Why the green check if you're not yet sure :)

This approval is for implementation

Kivooeo · 2026-01-11T00:20:03Z

but adding --color=always

What will this look like in the .stdout file?

tgross35 · 2026-01-11T01:42:21Z

but adding --color=always

What will this look like in the .stdout file?

I believe it should just show the escape codes, similar to the output.stdout file that's in this PR now (though code highlighting isn't hooked there of course)

These are two commits squashed together: feat: add support for primitive types refactor: refactor out the list of keywords into a global const

Kivooeo · 2026-01-11T14:41:58Z

I believe it should just show the escape codes, similar to the output.stdout file that's in this PR now

I've just tried this locally. Instead of showing ANSI codes in the .stdout file, it created a basic.svg file containing just a solid black rectangle, which doesn't seem correct

Here is what I did so you can try to reproduce it or point out what I might have done wrong

//@ compile-flags: --explain E0591 --error-format=human --color=always
//@ check-pass

(Without specifying --error-format, it gave an error. It seems the --color flag might be for diagnostics)

JayanAXHF · 2026-01-11T14:42:43Z

Can we consider colouring all primitives types as well? The same way it was made for keywords

https://doc.rust-lang.org/rust-by-example/primitives.html

Done in 7d96470

JayanAXHF · 2026-01-11T16:35:29Z

but adding --color=always

What will this look like in the .stdout file?

I believe it should just show the escape codes, similar to the output.stdout file that's in this PR now (though code highlighting isn't hooked there of course)

Would it be fine if the test just focused on the highlighter?

JayanAXHF · 2026-01-13T14:18:45Z

@Kivooeo (cc: @tgross35) I have figured out (i think) why --color always doesn't always print out in color. Apparently, within rustc_driver_impl,

        if io::stdout().is_terminal() {
            show_md_content_with_pager(&text, color);
        } else {
            safe_print!("{text}");
        }

prevents printing the output with color if stdout isn't a terminal. This is why --bless didn't output a .stdout file with ANSI escapes

JayanAXHF · 2026-01-13T14:48:54Z

Following my previous message, I have added the tests for the highlighter. However, I did have to change rustc_driver_impl's handle_explain function to be indifferent to stdout when --color always is passed. I hope that is alright

Kivooeo · 2026-01-13T18:21:49Z

compiler/rustc_driver_impl/src/lib.rs

+            if color == ColorConfig::Always {
+                show_md_content_with_pager(&text, color);
+            }


My original thought was to create not a UI test, but something that just tests the highlight function, maybe? I don't really like how this ANSI test looks; this change feels unrelated to this PR, and creating this SVG file is also something that I'd like to try to avoid

I agree with the test looking weird. Should I convert this to a highlighter test? IMO the change, however unrelated, makes the function behave as it should.

Maybe we could do this for different PR if you really thinks so

But I don't really think that this change will address any real problem or even be any kind of improvement (maybe I don't see something about this change, so you could elaborate on this a little more)

The fact that --color always didn't always output in color made its name and description a misnomer. Usually, when you set a program to output color always, it does so, regardless of the output being a terminal or not. This is just my opinion, I'd love to hear what you think about this.

I think it's like #[inline(always)] which is also not always doing inline

Yeah, I think it's better to separate this problems, in this PR we are working on making rustc --explain colourful, and I'd maybe suggest to open a zulip discussion on that matter

Opened #t-compiler > Rustc `--color always` doesn't always color output

I think it's like #[inline(always)] which is also not always doing inline

Yeah, I think it's better to separate this problems, in this PR we are working on making rustc --explain colourful, and I'd maybe suggest to open a zulip discussion on that matter

So should I refactor the test to be of the highlighter instead?

So should I refactor the test to be of the highlighter instead?

Yeah would be nice, maybe even revert last commit

I see there is some tests in compiler/rustc_errors/src/markdown/tests/, maybe it would be possible to add such test case somewhere here? Could to take a look on possibility to add a test here? The simplest test would also be fine

JayanAXHF added 3 commits January 10, 2026 01:37

feat: added colored code blocks to rustc --explain

6b79008

perf: cached string slices

2a9f7c2

Changed `highlight_rustc_lexer` to have only one place where the original lexeme is extracted. This reduces dereferences and slices. Very minor perf change.

style: made tidy happy

cbd1cfa

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 9, 2026

rustbot assigned fee1-dead Jan 9, 2026

rustbot assigned Kivooeo and unassigned fee1-dead Jan 9, 2026

This comment has been minimized.

Sign in to view

tgross35 reviewed Jan 9, 2026

View reviewed changes

compiler/rustc_errors/src/markdown/term.rs Outdated Show resolved Hide resolved

fix: fixed tests

9cf9316

The problem was something related to the fast that we use a `Style::new` default style when no style is matched

fix: rename entrypoint_with_formatter to entrypoint

1e95f71

JayanAXHF added 2 commits January 11, 2026 02:32

fix: removed unnecessary wrapper

b824220

Removed unnecessary wrapper around `write_anstream_buf_with_formatter` and renamed it to `write_anstream_buf`.

fix: removed write_stream_with_formatter in favour of write_stream

34f6890

Kivooeo approved these changes Jan 10, 2026

View reviewed changes

compiler/rustc_driver_impl/src/highlighter.rs Outdated Show resolved Hide resolved

typo: Update highlighter.rs

862ad78

Co-authored-by: Kivooeo <[email protected]>

This comment has been minimized.

Sign in to view

fix: Added missing function arg in test

ea82c01

feat: Support primitive types and refactor keywork detection

7d96470

These are two commits squashed together: feat: add support for primitive types refactor: refactor out the list of keywords into a global const

This comment has been minimized.

Sign in to view

test: added tests for colored rustc --explain

f53159e

JayanAXHF force-pushed the rustc_colored_explain branch from dddc3ea to f53159e Compare January 13, 2026 16:20

Kivooeo reviewed Jan 13, 2026

View reviewed changes

docs: fix doc comments

1755d4d

Uh oh!

rustc_errors: Add (heuristic) Syntax Highlighting for rustc --explain #150895

Are you sure you want to change the base?

rustc_errors: Add (heuristic) Syntax Highlighting for rustc --explain #150895

Conversation

JayanAXHF commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Implementation Details

Highlighter Implementation

Identifiers

Literals

Everything Else

Demo

Uh oh!

rustbot commented Jan 9, 2026

Uh oh!

rustbot commented Jan 9, 2026

Uh oh!

JayanAXHF commented Jan 9, 2026

Uh oh!

Kivooeo commented Jan 9, 2026

Uh oh!

JayanAXHF commented Jan 9, 2026

Uh oh!

Kivooeo commented Jan 9, 2026

Uh oh!

Kivooeo commented Jan 9, 2026

Uh oh!

JayanAXHF commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgross35 commented Jan 9, 2026

Uh oh!

Kivooeo commented Jan 9, 2026

Uh oh!

tgross35 commented Jan 9, 2026

Uh oh!

JayanAXHF commented Jan 9, 2026

Uh oh!

This comment has been minimized.

JayanAXHF commented Jan 9, 2026

Uh oh!

tgross35 commented Jan 9, 2026

Uh oh!

JayanAXHF commented Jan 9, 2026

Uh oh!

tgross35 commented Jan 9, 2026

Uh oh!

Uh oh!

JayanAXHF commented Jan 10, 2026

Uh oh!

JayanAXHF commented Jan 10, 2026

Uh oh!

Kivooeo commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kivooeo commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JayanAXHF commented Jan 10, 2026

Uh oh!

JayanAXHF commented Jan 10, 2026

Uh oh!

Kivooeo commented Jan 10, 2026

Uh oh!

JayanAXHF commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kivooeo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JayanAXHF commented Jan 10, 2026

Uh oh!

This comment has been minimized.

tgross35 commented Jan 10, 2026

Uh oh!

Kivooeo commented Jan 11, 2026

rustc_errors: Add (heuristic) Syntax Highlighting for `rustc --explain` #150895

rustc_errors: Add (heuristic) Syntax Highlighting for `rustc --explain` #150895

JayanAXHF commented Jan 9, 2026 •

edited

Loading

JayanAXHF commented Jan 9, 2026 •

edited

Loading

Kivooeo commented Jan 10, 2026 •

edited

Loading

Kivooeo commented Jan 10, 2026 •

edited

Loading

JayanAXHF commented Jan 10, 2026 •

edited

Loading

Kivooeo left a comment •

edited

Loading

JayanAXHF commented Jan 11, 2026 •

edited

Loading

JayanAXHF commented Jan 13, 2026 •

edited

Loading

Kivooeo Jan 13, 2026 •

edited

Loading

Kivooeo Jan 13, 2026 •

edited

Loading