Start improving monomorphization items stats

[lqd]

As described in this zulip discussion, some stats about monomorphization collection would be interesting to have, in a different form than -Zprint-mono-items: to have some visibility into the cost of the mono items, we'd like to know how many are instantiated and what is their estimated size.

That can be a proxy to analyze sources of slow compile times, although in the future, we'd also like to add more realistic stats from the actual backend's lowering.

This PR adds a new -Z dump-mono-stats flag which will output some stats in a {crate_name}.mono-items.md file (the flag optionally takes an output directory parameter, for easier use within a workspace than printing to stdout).

For example,

fn compute<T>(collection: Vec<T>) -> usize {
    collection.len() + 19 - 0 * 9 - 18 - 1 * 1 // random code to increase the function's size
}

fn main() {
    dbg!(compute(vec![0u8, 1, 2]));
    dbg!(compute(vec![0u64, 1, 2]));
    dbg!(compute(vec!["0", "1", "2", "3"]));
}

will output a file with this markdown table (abridged for readability), for a debug build:

Item	Instantiation count	Estimated Cost Per Instantiation	Total Estimated Cost
alloc::alloc::box_free	3	122	366
std::alloc::Global::alloc_impl	1	284	284
alloc::raw_vec::RawVec::<T, A>::current_memory	3	82	246
std::ptr::align_offset	1	222	222
std::slice::hack::into_vec	3	67	201
<std::vec::Vec<T, A> as std::ops::Drop>::drop	3	66	198
std::ptr::mut_ptr::<impl *mut T>::is_null	4	47	188
main	1	163	163
std::ptr::NonNull::::new_unchecked	4	37	148
...

Click for full output

Item	Instantiation count	Estimated Cost Per Instantiation	Total Estimated Cost
alloc::alloc::box_free	3	122	366
std::alloc::Global::alloc_impl	1	284	284
alloc::raw_vec::RawVec::<T, A>::current_memory	3	82	246
std::ptr::align_offset	1	222	222
std::slice::hack::into_vec	3	67	201
<std::vec::Vec<T, A> as std::ops::Drop>::drop	3	66	198
std::ptr::mut_ptr::<impl *mut T>::is_null	4	47	188
main	1	163	163
std::ptr::NonNull::::new_unchecked	4	37	148
std::boxed::Box::<T, A>::into_unique	3	48	144
std::boxed::Box::<T, A>::leak	3	39	117
std::alloc::Layout::array::inner	1	107	107
std::ptr::align_offset::mod_inv	1	103	103
std::boxed::Box::<T, A>::into_raw_with_allocator	3	31	93
std::fmt::Arguments::<'a>::new_v1	1	80	80
<alloc::raw_vec::RawVec<T, A> as std::ops::Drop>::drop	3	26	78
alloc::raw_vec::RawVec::<T, A>::from_raw_parts_in	3	26	78
alloc::alloc::exchange_malloc	1	75	75
std::ptr::const_ptr::<impl *const T>::is_null	1	75	75
std::ptr::const_ptr::<impl *const T>::is_aligned_to	1	64	64
compute	3	20	60
std::ptr::const_ptr::<impl *const T>::align_offset	1	55	55
std::ptr::read	1	52	52
<std::alloc::Global as std::alloc::Allocator>::deallocate	1	50	50
std::ptr::mut_ptr::<impl *mut T>::guaranteed_eq	1	48	48
std::fmt::ArgumentV1::<'a>::new_display	2	22	44
std::ptr::Alignment::new_unchecked	1	42	42
core::fmt::num::::fmt	1	40	40
std::result::Result::<T, E>::unwrap_unchecked	1	37	37
std::cmp::Ord::min	1	32	32
std::cmp::impls::::cmp	1	31	31
std::intrinsics::is_aligned_and_not_null	1	27	27
std::rt::lang_start	1	27	27
std::ptr::NonNull::::new	1	24	24
std::fmt::ArgumentV1::<'a>::new_debug	1	22	22
std::fmt::Arguments::<'a>::new_v1_formatted	1	19	19
std::rt::lang_start::{closure#0}	1	17	17
std::sys_common::backtrace::__rust_begin_short_backtrace	1	16	16
std::slice::<impl [T]>::into_vec	3	5	15
<std::ptr::NonNull as std::convert::From<std::ptr::Unique>>::from	1	14	14
<&T as std::fmt::Debug>::fmt	1	12	12
<&T as std::fmt::Display>::fmt	1	12	12
std::vec::Vec::<T, A>::len	3	2	6
<T as std::convert::Into>::into	1	5	5
<T as std::convert::From>::from	1	2	2
<() as std::process::Termination>::report	1	2	2
std::hint::unreachable_unchecked	1	2	2
core::fmt::UnsafeArg::new	1	1	1

Since we discussed it together, r? @wesleywiser.

[rustbot]

rustc_error_messages was changed

cc @davidtwco, @compiler-errors, @JohnTitor, @estebank, @TaKO8Ki

[jyn514]

Outputting as markdown doesn't seem very extensible, it's hard to parse. Could we output JSON instead (and document a simple JQ command to turn it into markdown)?

[klensy]

Outputting as markdown doesn't seem very extensible, it's hard to parse. Could we output JSON instead (and document a simple JQ command to turn it into markdown)?

Let csv win in that case.

[wesleywiser]

Outputting as markdown doesn't seem very extensible, it's hard to parse. Could we output JSON instead (and document a simple JQ command to turn it into markdown)?

@jyn514 Do you have use cases in mind where you'd want to parse the output?

[jyn514]

@wesleywiser if print-mono-items were extensible / used JSON, we could have added to that flag instead of adding a new one. I don't have concrete use cases in mind, but it seems nice to make the output reusable instead of hardcoding markdown.

[lqd]

You could add the same information to -Zprint-mono-items after the existing output, add some information to the per-line output, or change it altogether with a different argument than "lazy/eager", it's still extensible. Having a file with the crate name, and in a human-readable format, seems easier to use though.

@wesleywiser what do you want to do here ?

[jyn514]

Ok, makes sense. I don't have strong objections to markdown.

[wesleywiser]

I used this a bit yesterday to look at some codesize issues and found the current format helpful as it was easy to just set RUSTFLAGS, run the normal compilation and then look at the various files that were created in the working directory. I see the value of having this available as json as well but that seems like an improvement that can come later as people want it.

@bors r+

[bors]

📌 Commit b720847 has been approved by wesleywiser

It is now in the queue for this repository.

[nnethercote]

How does everyone view markdown files?

BTW, I was a little surprised that the markdown files ended up in the root directory. I went looking in the build directory first, where things like .ll files are put...

[jyn514]

@nnethercote you can use pulldown-cmark to convert them to html and view them in a browser, or paste them into a gist and have GitHub render them.

[lqd]

@nnethercote btw, similarly to -Zself-profiler, you can also pass a directory as the flag’s parameter and it’ll dump files there instead of the current directory.

Some editors also support markdown rendering, either natively or via plugins, VSCode for example, and I often use that or hackmd.