Rust doesn't reliably build on 32-bit operating systems anymore

[thestinger]

Rust often requires more than 2G of virtual memory so it's not possible to build it on a 32-bit OS. This is especially problematic on Windows.

#18678 may be enough to fix this on Linux / FreeBSD, and would allow preventing regressions via the jemalloc mapped statistic (+ accounting for the VM used by libraries, etc.).

[nikomatsakis]

Regardless of what else we do, reducing memory consumption would also, of course, be a good idea. I ran massif on master (16d80de, to be precise) and the output is here. I haven't had time to look yet. I suspect we can do a lot to reduce the type-checking peak; not sure what's the best way to approach the LLVM side -- perhaps the parallel compilation logic can help here.

[eddyb]

I've had this idea of splitting the type interner/context into a global and a local (per-function) one.
The reasoning is that most of the types being interned (almost 90% at the time I did some stats) have function-specific inference variables and never outlive the writeback phase of that function's type-checking.

However, it's not an easy thing to do - #18483 aka safe-ty wasn't possible at all for many months, only a few weeks ago all the ICEs it was triggering were fixed, and it's still just a pre-requisite for the actual change.

Which is pretty much impossible right now. Certain lifetime elision features need to be implemented in order for further experimentation to be possible, and I can't do much, given school and personal defficiencies.

If desired, I can update the code generating those statistics, add indirect memory usage estimates and run it on master, to get us an estimation.

[nikomatsakis]

So I think we have to attack this problem with a variety of approaches. It's blocking lots of important PRs and that has to stop -- at the same time, we should think about longer term fixes as well. From reading the massif output, I see that:

1. The peak memory usage is indeed by LLVM, and it does (indeed) not go through jemalloc. Do we have actual measurements of how much jemalloc helps here?
2. Another way to reduce the LLVM peak might be to use the work on parallel compilation to break the output into chunks and compile them sequentially, freeing memory in between.
3. The second-highest peak occurs after monomorphization I believe. Looking at the massif output, it appears that roughly 20% of the type memory is due to types created during trans, and approximately 10% is created due to types created during memory matching. It may be possible to reduce those. I've done some instrumentation looking into the contents of those types but need to do more.
4. It occurs to me that the easiest way to sidestep this in the short term is probably just to break librustc into crates, so as to reduce the overall size of the compilation unit.

cc @aturon @pcwalton @alexcrichton

[alexcrichton]

I think that using jemalloc in LLVM or splitting up librustc is the best way to go for now.

Getting the benefit of extra compilation units may be somewhat difficult. I think we'd have to translate/destroy each one of them in turn instead of what we do now which is to build all compilation units at once and then compile them. I would suspect that there's a large amount of duplicated data between contexts so the trans peak would be higher. But then again of the LLVM peak is higher than the trans peak, maybe it wouldn't be so bad...

[nikomatsakis]

On Wed, Nov 12, 2014 at 09:05:45AM -0800, Alex Crichton wrote:

I think that using jemalloc in LLVM or splitting up librustc is the best way to go for now.

I've made some progress on splitting up librustc. I've also found that
the method dispatch PR I want to land winds up creating a lot of types
and taking more memory. I'm going to see what can be done about that,
I don't understand yet what's going on precisely.

[thestinger]

#18896 will be the new attempt at using jemalloc in just executables via symbol aliases

[nikomatsakis]

This branch breaks up rustc into two crates. It builds up to rustdoc. The division is not "logical" necessarily, I based it purely on the dependencies in the modules, though it kind of makes sense -- basically it moves the overarching driver plus everything that occurs after trans into another crate. I haven't measured the overall effect on memory usage.

[nikomatsakis]

#19002 implements the proposed crate split.

[nikomatsakis]

I think the crate splits have basically fixed this issue. Closing. Re-open if you disagree.