Convert tests/debuginfo/pretty-std.rs to lldb-repr

[Walnut356]

Two things of note:

• I had to fix another small issue with msvc template args
• Enabling this test on windows-gnu caused the GDB test to fail because GDB decodes the emoji to raw bytes when working with wtf-8 strings. There's not an easy way to handle wtf-8 in python i think? So i just replaced the emoji with a wildcard. The target-specific differences should also fix itself once gdb-repr is implemented

r? @jieyouxu, @Kobzol

---

try-job: aarch64-apple
try-job: aarch64-apple-macos-26

[rustbot]

jieyouxu is currently at their maximum review capacity.
They may take a while to respond.

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[Kobzol]

Damn. This PR makes me quite concerned about the approach we took with the JSON files. This is a single ported test, which results in a ~3k diff. I knew it would be bad, but not this bad. And this is still only for LLDB, not even with GDB included! That means that every ported test, but also every new added debuginfo test, would likely add a massive JSON file. That's not great.

Another thing that I don't like here is how to review the JSON files. The thing that I want to check the most is the user-facing output for the individual debugged variables, primarily their pretty print and the rendering of their children. This data is currently buried within ~1k lines of JSON that contains a lot of other information, and it is difficult for me to find the interesting data within all that noise.

I yet again wonder if we are optimizing for the right thing here. I know that you want to see all these details in case something breaks and you need to debug it, and that you also want to see historical changes of these details, in case something in them changes without the pretty print/children changing at the same time. But it still feels to me like we are essentially committing compiler debug logs and Debug prints of internal compiler data to git for each test, so that we can check historically how they changed. We don't do this for pretty much any other test suite, and I don't think that we should do it here.

The common case is the user observable parts (pretty print/children) breaking, and I think that is the thing that we should focus on, same as with other UI tests.

I think that there is a compromise that we could make to support your use-case of going back in time and checking when a given internal representation changed, while making the committed files be much smaller and making it easier to review them.

• By default, we would commit only a much simpler JSON file with basic information about the debugged variables, and the debugger metadata used to create the file, and nothing else.
• We would implement a simple cargo-rustc-bisect wrapper, which would run a given debuginfo test starting from some past version, generate the complex JSON (that is committed now) for each version, and display the diffs. This would allow going back in time and checking when a given internal representation might have changed. Note that only a single LLDB version would be required for this, so the bisection should be rather simple. Because LLDB bumps would anyway invalidate the JSON's internal representation. But if needed, support for multiple LLDB version could be added.

What do you think? :)

[Walnut356]

I'm not particularly happy about the file size either. There are ways we can reduce it (e.g. collapsing "array-like" children into a flat array instead of full child listings, consolidating generics into a single type listing, etc.). Lots of the existing tests cover redundant things, so they can (and probably should) be consolidated into a smaller number of longer tests, which amortizes the type data. We can also use an alternative data format, we'd just need to swap out the ser/de code. That kind of space optimization seems beside the point at the moment though, as you're suggesting we don't store the type data at all.

It is my understanding that the old test logic never had issues detecting user-facing regressions, so long as the tests were actually enabled. Rather, the problem was that people would disable the tests instead of fixing them because the effort and time to learn enough to fix them, and then actually fix them, was not deemed worthwhile. Storing the type information intends to solve that problem.

To be entirely clear, I do not personally need the type information stored. It saves me a bunch of time, but I don't need it. I am relatively familiar with the debug info we generate, how it appears in the DWARF/PDB data, and how the debuggers read, represent, and expose it. The same cannot be said for almost any other contributor to the rust repo. There's just too much surface area for everyone (or even most people) to be familiar with debug info. Someone could change the heuristic for niche optimization and break visualizers. Someone could (and has, multiple times), changed the field names of Vec and broken the visualizers¹. Debug info is pervasive and unavoidable. It is impossible to keep these breakages away from people who aren't familiar with debug info. The type-data storage and error messages are largely to prevent those cases from turning into "fuck it, just disable the tests", or people just updating it in-place and causing a regression because the contributor and reviewer both don't know enough to spot the regression.

The intent is to not have to inspect the json data except in extreme circumstances (and/or in diffs of ~a few lines for patches and such). The data feeds the error messages that describe the issue in enough detail that people don't have to look at the data. I understand that the initial diff is rough, but that's a 1-time problem when the test is first created.

The thing that I want to check the most is the user-facing output for the individual debugged variables, primarily their pretty print and the rendering of their children. This data is currently buried within ~1k lines of JSON that contains a lot of other information, and it is difficult for me to find the interesting data within all that noise...But it still feels to me like we are essentially committing compiler debug logs and Debug prints of internal compiler data to git for each test, so that we can check historically how they changed. We don't do this for pretty much any other test suite, and I don't think that we should do it here.

I still disagree with the notion that anything not user-facing is "noise" in the context of these tests. The type information in the json file is not rustc's type information. It is the debugger's interpretation of the debug info rustc asked LLVM for, through the lens of LLVM lossily translating it to-and-from 2 file formats we don't control.

The visualizers are ~100% load-bearing assumptions based data we receive from a massive black box that can do literally anything at all before it gives it back to us. For example, LLDB ignores the primitive type names we ask LLVM for and uses C-style names. GDB ignores pointer-type names and makes them all *mut. LLDBs truncate every enum discriminant to 32 bits. LLDB could trivially replace every type name with aaaaa, every value with 12345, add arbitrary fake fields to everything, lie about size and alignment, and only allow emojis when indexing Vec, and no changes to rustc's debug info generation would fix that.

A TypeSystem is supplied a debug info file, but it doesn't have to use it or respect any of the information in it. It's simply asked "get me the thing with this name" or "get me the fields of this type" and can provide literally whatever it wants to. GDB has similar capabilities from what I've seen.

I don't believe it makes sense to ignore the fact that we are operating on completely untrusted data. The tests disabled with the message:

// LLDB 1800+ tests were not tested in CI, broke, and now are disabled

Attest to that. I haven't personally checked all of them, but it's highly likely that the data LLDB provided changed in version 18+ (despite no changes occurring on our end). That broke our assumptions, but those assumptions were completely undocumented and we did not test that they were upheld. The failure happens too late in the pipeline, making the source of the failure unclear. That resulted in the tests being disabled instead of fixed.

I don't think testing untrusted data is unique in rust's test suites. tests/codegen_llvm seems to be exclusively filled with "is <the black box of LLVM> giving us the IR/ASM output we expect with the way we are asking for it?". There are even debuginfo codegen tests there that check that we're getting the correct debug info IR nodes (the reason those checks are semi-irrelevant to the visualizers is because, as stated earlier, LLVM translates these nodes lossily to DWARF and PDB, and the debuggers don't have to respect what's in those files).

Imo, not testing the data we receive from lldb/gdb would be akin to rm rf ./tests/codegen_llvm because when we compile println!("{:?}", vec![1, 2, 3) it prints [1, 2, 3]

Footnotes

1. I do want to point out, though, that the reason debug info became such a big issue (and the reason I started working on it) was due to Rust "outsourcing" the solution to CodeLLDB's TypeSystemRust and custom visualizer scripts, and the maintainer of CodeLLDB getting burnt out and removing their custom visualizers. They specifically referenced this sort of churn as the reason why they burnt out and did not want to maintain Rust visualizers anymore (see: 1, 2, 3). Making this process as easy as possible seems like a worthwhile goal. ↩