trans/LLVM: Don't keep all LLVM modules in memory at the same time

[michaelwoerister]

UPDATE: There are some mentoring instructions below if you're interested in looking into this issue.

---

NB: Part of the roadmap issue on incremental compilation.

Currently the compiler translates all codegen units into LLVM modules and then runs LLVM on all of them in parallel. In the context of incremental compilation, where there can be hundreds of codegen units for a single crate, but also for non-incremental builds with a high number of codegen units, this can mean a lot of memory pressure -- e.g. for Servo's script crate that's more than 10 GB peak memory usage (while otherwise it's around 4.5 GB).

There's no real need to keep more LLVM modules in memory than are currently being worked on. Two possible solutions:

1. If the compiler uses N LLVM threads, start persisting LLVM modules to disk at the N+1st and reload them later for optimization and codegen. That's probably relatively easy to implement but involves more disk I/O than necessary.
2. Translate only N codegen units to LLVM modules, translate them all the way to object files, then go back and translate the next codegen unit. This has the advantage that we would not need to temporarily store anything on disk, but we would need to keep the tcx in memory as long as there are still untranslated codegen units. It would also require a much bigger refactoring than the first approach.

Any other ideas?

cc @rust-lang/compiler @rust-lang/tools @nagisa

[nrc]

Will this actually cause problems? Unless there are too many threads (in which case that should be addressed), it feels like the OS will handle this better by paging to disk than we will by (effectively) doing so manually. Also, for many devs, 10GB is not too much memory usage and anything we do here (unless we're really smart) will slow down compilation for those users.

[alexcrichton]

My first thought would be the latter, consider translation part of a "work unit" and don't actually start doing it until it's ready to go all the way into LLVM. That way we cap the number of active LLVM modules at any one point in time.

I think @nrc's got a good point, though, in that using the disk as a cache may be better left to the OS rather than the compiler itself (until proven otherwise, of course)

[michaelwoerister]

Some data points:

Will this actually cause problems?

It lead to my not being able to compile the script crate on Linux with 8 GB. Maybe if I had more swap space it would have worked. (EDIT: The VM had 4GB swap space)

Unless there are too many threads (in which case that should be addressed)

If everything fits into memory at once, having hundreds of parallel LLVM threads does not seem to have a detrimental effect (running 140 threads was a few percent faster than having 4 threads on a 4 core processor). OTOH, I would suspect that paging would completely destroy performance if many threads had to fight for RAM.

Also, for many devs, 10GB is not too much memory usage and anything we do here (unless we're really smart) will slow down compilation for those users.

That's certainly a valid concern!

[retep998]

If you really want to allow the OS to do a good job of paging out memory you need to ensure that modules don't share pages with each other. If a given module is spread out across your memory with lots of fragmentation, then you'll have to have a lot more memory paged in than necessary, reducing the effectiveness of the OS paging stuff out. Also, there are ways to hint to the OS that you don't need some memory at the moment, so it can be more willing to page it out sooner in favor of keeping other memory in memory.

[michaelwoerister]

It seems to me like letting the OS take care of this via paging is really the most efficient and most easily implemented solution. But I'm also a bit concerned about rustc getting the reputation of being a memory hog, even if unfairly. It also allows for fewer crates to be compiled by a 32-bit compiler (although the script crate, for example, is already above 2GB before trans, I think).

[hanna-kruppe]

Would option 2 actually be any slower or otherwise less efficient? It doesn't seem to do any additional work. tcx would remain in memory longer, but since (in the case of huge crates with many modules) the proposed change saves a lot of memory on LLVM modules that shouldn't hurt much, if at all. Besides, in the easy case (enough threads to trans all CGUs in parallel), the tcx can still be dropped as soon as IR was generated and before the LLVM passes run.

As @retep998 outlines, it is not guaranteed that the OS will actually do a perfect job of paging modules in and out. So ignoring option 1, which does a lot of unnecessary disk I/O, the choice seems to be between doing the same work and either:

• Allocate all the memory up-front and pray the OS handles it gracefully, or
• Allocate and free memory as needed, to hopefully avoid the need for paging in the first place.

[nikomatsakis]

I'm a bit confused. It seems to me that there are three things that need to be done:

• Translation from HIR to LLVM IR.
• Optimization and transformation of LLVM IR.
• Generate .o file.

Currently, we do the first one for everything. Then we start some threads to do the second one -- at what point do we do the third one? (I can't remember.) In particular, do we wait for all the threads to finish?

[michaelwoerister]

at what point do we do the third one?

That's done by the last LLVM pass, IIRC.

[nikomatsakis]

@michaelwoerister so, in that case, I disagree with the assessment of "Let the O/S do it". In particular, it seems likely that "optimization and transformation of the LLVM IR" takes a lot of (temporary) memory. If we start N threads (one per CGU) and those are all going simultaneously, that's going to lead to a larger peak than is necessary. But if we throttle starting at phase 2, this will reduce the peak, right?

In particular, the O/S is good at paging memory that is no longer used, but if we start N threads, all that memory is in active use ("working set" is large). If we throttle, working set is smaller, and we can free as we finish each item (since .o file is generated), no?

It might reduce the peak further to do the full pipeline one CGU at a time, but maybe not, because as it is we can free the HIR and tcx entirely once trans is done (and I believe we do so).