[RFC] Use pre-compiled headers to speed up LLVM build (~1.5-2x) #173868
Conversation
aengelke
added
cmake
🐧 Linux x64 Test Results
✅ The build succeeded and all tests passed. |
🪟 Windows x64 Test Results
✅ The build succeeded and all tests passed. |
|
FWIW, statistics about optimization passes >5s (no adaptors, wrappers, etc.) on libLLVM compiled with -O1 (NB: pass times can include analyses): |
…3869) This avoids looking at the individual sources for mixed C/C++ libraries. The previous code was written ~2014. Generator expressions were added in CMake 3.3 (2015). We currently require CMake 3.20 and therefore can rely on more modern features. Apart from simplifying the code, this is preliminary work to make more use of pre-compiled headers (#173868).
|
Another data point: I can build LLVM (incl. tools) in 6-7 minutes on my laptop now (M2 Macbook Air, 8 cores); building unit-tests take another 2.5mins. Interestingly, with PCH optimizations are comparably expensive here: Most expensive CUs >15s: |
boomanaiden154
left a comment
boomanaiden154
left a comment
There was a problem hiding this comment.
Not exactly sure how you need to invoke the clang driver to use precompiled headers, but it seems like sccache might not have support currently (mozilla/sccache#615). Might still be useful for local builds, but we would have to turn it off in premerge CI until it works with caching.
Either way, I would probably like to see a proper RFC on discourse for this.
…m#173869) This avoids looking at the individual sources for mixed C/C++ libraries. The previous code was written ~2014. Generator expressions were added in CMake 3.3 (2015). We currently require CMake 3.20 and therefore can rely on more modern features. Apart from simplifying the code, this is preliminary work to make more use of pre-compiled headers (llvm#173868).
Don't duplicate the EnumEntry type in llvm-objdump. Spliced off from #173868, where this is required to avoid the name collision.
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
|
FWIW, some test feedback on this patchset: On Ubuntu 20.04, with the stock system libstdc++ 9, I get this: On Ubuntu 24.04 with GCC, I get lots of these warnings, and no measurable speedup: On Ubuntu 24.04 with Clang, I do see something of a speedup though. |
I think the simplest workaround for that error w/ gcc+libstdc++9 would be to remove the |
|
Thanks for checking GCC (14.2), I rarely use it for building LLVM. Reducing the number of headers seems to be important for GCC, there's certainly some room for improvements. Some observations from quick testing and profiling of GCC:
The warnings seem to only come from executables, most files in libraries are not afffected. |
|
I dumped all template instantiations GCC additionally performs when using PCH. These are round 50k additional instantations performed just when compiling LLVMSupport.a. extra-templ-LLVMSupport.txt Clang doesn't have this problem due to I don't think the GCC compile-time is fixable on our side. We might want to consider not using PCH with GCC. Here is data for a standard single-target LLVM release build:
PCH sizes are also much larger with GCC: Build directory sizes:
|
Due to heavy use of using namespace llvm, Reloc is often ambiguous with llvm::Reloc, the relocation model. Previously, this was sometimes disambiguated with macho::Reloc. This ambiguity is even more problematic when using pre-compiled headers, where it's no longer "obvious" whether it should be Reloc or macho::Reloc. Therefore, rename Reloc to Relocation. This is also consistent with lld/ELF, where the type is also named Relocation.
|
@aengelke one issue I’ve noticed w/ this PR: currently, when building llvm w/ rtti on and also building Polly, Polly fails to compile You should probably somehow just edit your pr to exclude the use of pch stuffs for polly |
|
Thanks for catching this, I added a fix to #176420. |
|
Llvm CPU time: 5497.8 usr, 297.6 sys Hmm. So from the looks of it clang/llvm could be optimized more. Gcc is most likely spending more time in system as it writes out the .s file and then gas reads it back in and assembles it. |
5 participants
Building LLVM is slow and dominated by the C++ front-end. Most time is spent in (repeatedly) parsing headers. C++ modules build didn't really help, is fragile, and kills parallelism. Therefore, I propose to use PCH for frequently used headers (i.e., C++ stdlib, Support, IR, CodeGen). There shouldn't be much difference for incremental compilation, as many headers I put into the PCHs transitively get included into most source files anyway.
Time breakdown in seconds for building libLLVM.so (collected with -ftime-trace, -O1, -DLLVM_TARGETS_TO_BUILD="X86;AArch64"):
(Edit: new data with idle machine + assertions disabled, individual results are still a bit noisy due to hyper-threading; no substantial change in relative factors.) (NB: on this machine, much larger wall-time reductions are rather unlikely: SLPVectorizer.cpp takes >40s alone.)
The back-end is not really relevant here, but enabling PCH (this PR) reduces the front-end time by 2x. On c-t-t, this shows up as a 30% wall-time/instructions reduction in stage2-clang. Further improvements for Clang should also be possible (clang/AST/Decl.h is expensive with 1219s), but is not trivial due to the use of object libraries.
I'm opening this PR for early feedback/direction before spending more time on this:
cc @nikic @rnk @boomanaiden154 (not sure who else is interested in LLVM build times + looking at CMake)