rav1e compilation time regression (4s -> >1hr) on LLVM 17

[FauxFaux]

Code

Applications including https://github.com/xiph/rav1e , including the repo's example, never complete compilation in --release. I waited over an hour on my 2019 desktop (i7-8700k).

I tested f8b95d97517a6e223f1c6a90a467188586dd090f.

$ CARGO_INCREMENTAL=0 cargo +beta build --no-default-features --example simple_encoding --release

Version it worked on

It most recently worked on: rustc 1.73.0-nightly (03a119b 2023-08-07)
Stable is fine: rustc 1.72.1 (d5c2e9c 2023-09-13)

rustc 1.73.0-nightly (03a119b0b 2023-08-07)
binary: rustc
commit-hash: 03a119b0b0e310d22d94399b24ed030056050f13
commit-date: 2023-08-07
host: x86_64-unknown-linux-gnu
release: 1.73.0-nightly
LLVM version: 16.0.5

Version with regression

rustc --version --verbose:

rustc 1.73.0-beta.6 (9b9539791 2023-09-17)
binary: rustc
commit-hash: 9b95397915e672e76c43eb013289bca7b9a23370
commit-date: 2023-09-17
host: x86_64-unknown-linux-gnu
release: 1.73.0-beta.6
LLVM version: 17.0.0

rustc 1.73.0-nightly (f88a8b71c 2023-08-08)
binary: rustc
commit-hash: f88a8b71cebb730cbd5058c45ebcae1d4d9be377
commit-date: 2023-08-08
host: x86_64-unknown-linux-gnu
release: 1.73.0-nightly
LLVM version: 17.0.0

I can only assume this is the new LLVM: #114048

@rustbot modify labels: +regression-from-stable-to-beta -regression-untriaged

Thoughts

If you add random panics in places, the compilation speeds up. The end of the main loop is a good candidate, this can bring the compile time down to only a few minutes (still a massive regression).

For this, --timings doesn't seem to know what's up: https://b.goeswhere.com/cargo-timing-20230921T083504Z.html

RUSTFLAGS='-Z time-llvm-passes' produces a bunch of output, including this one? I don't know how to read this output and I expect anyone who can is able to reproduce the problem, so I'm not including it in full.

00:01:46 ===-------------------------------------------------------------------------===
00:01:46                           Pass execution timing report
00:01:46 ===-------------------------------------------------------------------------===
00:01:46   Total Execution Time: 104.3365 seconds (101.2781 wall clock)
00:01:46-
00:01:46    ---User Time---   --System Time--   --User+System--   ---Wall Time---  --- Name ---
00:01:46   82.7290 ( 86.1%)   7.8338 ( 95.0%)  90.5628 ( 86.8%)  90.5593 ( 89.4%)  GVNPass
00:01:46    5.1964 (  5.4%)   0.2020 (  2.4%)   5.3984 (  5.2%)   3.8898 (  3.8%)  SimpleLoopUnswitchPass
00:01:46    3.2395 (  3.4%)   0.0006 (  0.0%)   3.2401 (  3.1%)   3.2407 (  3.2%)  LoopSimplifyPass
00:01:46    1.3124 (  1.4%)   0.0090 (  0.1%)   1.3214 (  1.3%)   1.3228 (  1.3%)  IndVarSimplifyPass
00:01:46    1.0127 (  1.1%)   0.0333 (  0.4%)   1.0460 (  1.0%)   0.7634 (  0.8%)  InstCombinePass
00:01:46    0.3558 (  0.4%)   0.0043 (  0.1%)   0.3601 (  0.3%)   0.2950 (  0.3%)  SimplifyCFGPass
00:01:46    0.1910 (  0.2%)   0.0047 (  0.1%)   0.1957 (  0.2%)   0.1844 (  0.2%)  SCCPPass
00:01:46    0.1673 (  0.2%)   0.0012 (  0.0%)   0.1685 (  0.2%)   0.1681 (  0.2%)  LoopFullUnrollPass
00:01:46    0.3241 (  0.3%)   0.0189 (  0.2%)   0.3430 (  0.3%)   0.1667 (  0.2%)  LICMPass
00:01:46    0.1501 (  0.2%)   0.0019 (  0.0%)   0.1520 (  0.1%)   0.1510 (  0.1%)  LoopIdiomRecognizePass
00:01:46    0.2368 (  0.2%)   0.0145 (  0.2%)   0.2514 (  0.2%)   0.1318 (  0.1%)  LoopSimplifyCFGPass
00:01:46    0.1849 (  0.2%)   0.0208 (  0.3%)   0.2057 (  0.2%)   0.1049 (  0.1%)  LoopInstSimplifyPass
00:01:46    0.1198 (  0.1%)   0.0219 (  0.3%)   0.1416 (  0.1%)   0.0775 (  0.1%)  InlinerPass
00:01:46    0.0270 (  0.0%)   0.0035 (  0.0%)   0.0304 (  0.0%)   0.0301 (  0.0%)  LoopDeletionPass
00:01:46    0.2481 (  0.3%)   0.0184 (  0.2%)   0.2666 (  0.3%)   0.0288 (  0.0%)  SROAPass
00:01:46    0.0330 (  0.0%)   0.0013 (  0.0%)   0.0344 (  0.0%)   0.0237 (  0.0%)  JumpThreadingPass
00:01:46    0.0896 (  0.1%)   0.0058 (  0.1%)   0.0954 (  0.1%)   0.0173 (  0.0%)  EarlyCSEPass
00:01:46    0.0264 (  0.0%)   0.0020 (  0.0%)   0.0284 (  0.0%)   0.0152 (  0.0%)  CorrelatedValuePropagationPass
00:01:46    0.0159 (  0.0%)   0.0004 (  0.0%)   0.0163 (  0.0%)   0.0135 (  0.0%)  LCSSAPass
00:01:46    0.0257 (  0.0%)   0.0013 (  0.0%)   0.0270 (  0.0%)   0.0130 (  0.0%)  LoopRotatePass
00:01:46    0.1138 (  0.1%)   0.0127 (  0.2%)   0.1265 (  0.1%)   0.0105 (  0.0%)  IPSCCPPass
00:01:46    0.0801 (  0.1%)   0.0131 (  0.2%)   0.0931 (  0.1%)   0.0089 (  0.0%)  GlobalOptPass
00:01:46    0.0267 (  0.0%)   0.0027 (  0.0%)   0.0294 (  0.0%)   0.0070 (  0.0%)  PostOrderFunctionAttrsPass
00:01:46    0.0117 (  0.0%)   0.0011 (  0.0%)   0.0127 (  0.0%)   0.0067 (  0.0%)  DSEPass
00:01:46    0.0121 (  0.0%)   0.0009 (  0.0%)   0.0130 (  0.0%)   0.0063 (  0.0%)  ADCEPass
00:01:46    0.0089 (  0.0%)   0.0003 (  0.0%)   0.0092 (  0.0%)   0.0062 (  0.0%)  VectorCombinePass
00:01:46    0.0087 (  0.0%)   0.0007 (  0.0%)   0.0095 (  0.0%)   0.0055 (  0.0%)  BDCEPass
00:01:46    0.0100 (  0.0%)   0.0025 (  0.0%)   0.0125 (  0.0%)   0.0055 (  0.0%)  MemCpyOptPass
00:01:46    0.0489 (  0.1%)   0.0064 (  0.1%)   0.0553 (  0.1%)   0.0051 (  0.0%)  CalledValuePropagationPass
00:01:46    0.0096 (  0.0%)   0.0012 (  0.0%)   0.0109 (  0.0%)   0.0027 (  0.0%)  ArgumentPromotionPass
00:01:46    0.0061 (  0.0%)   0.0004 (  0.0%)   0.0065 (  0.0%)   0.0024 (  0.0%)  MergedLoadStoreMotionPass
00:01:46    0.0042 (  0.0%)   0.0004 (  0.0%)   0.0046 (  0.0%)   0.0024 (  0.0%)  ReassociatePass
00:01:46    0.0046 (  0.0%)   0.0004 (  0.0%)   0.0051 (  0.0%)   0.0019 (  0.0%)  ConstraintEliminationPass
00:01:46    0.0147 (  0.0%)   0.0005 (  0.0%)   0.0152 (  0.0%)   0.0016 (  0.0%)  LowerExpectIntrinsicPass
00:01:46    0.0115 (  0.0%)   0.0011 (  0.0%)   0.0126 (  0.0%)   0.0015 (  0.0%)  RequireAnalysisPass<llvm::GlobalsAA, llvm::Module, llvm::AnalysisManager<Module>>
00:01:46    0.0027 (  0.0%)   0.0003 (  0.0%)   0.0029 (  0.0%)   0.0013 (  0.0%)  TailCallElimPass
00:01:46    0.0022 (  0.0%)   0.0004 (  0.0%)   0.0025 (  0.0%)   0.0010 (  0.0%)  AggressiveInstCombinePass
00:01:46    0.0065 (  0.0%)   0.0006 (  0.0%)   0.0071 (  0.0%)   0.0008 (  0.0%)  PromotePass
00:01:46    0.0014 (  0.0%)   0.0002 (  0.0%)   0.0017 (  0.0%)   0.0005 (  0.0%)  RequireAnalysisPass<llvm::ShouldNotRunFunctionPassesAnalysis, llvm::Function, llvm::AnalysisManager<Function>>
00:01:46    0.0027 (  0.0%)   0.0001 (  0.0%)   0.0028 (  0.0%)   0.0005 (  0.0%)  CallSiteSplittingPass
00:01:46    0.0010 (  0.0%)   0.0001 (  0.0%)   0.0011 (  0.0%)   0.0004 (  0.0%)  LibCallsShrinkWrapPass
00:01:46    0.0034 (  0.0%)   0.0003 (  0.0%)   0.0038 (  0.0%)   0.0004 (  0.0%)  InvalidateAnalysisPass<llvm::AAManager>
00:01:46    0.0003 (  0.0%)   0.0000 (  0.0%)   0.0003 (  0.0%)   0.0003 (  0.0%)  DeadArgumentEliminationPass
00:01:46    0.0041 (  0.0%)   0.0000 (  0.0%)   0.0041 (  0.0%)   0.0003 (  0.0%)  InferFunctionAttrsPass
00:01:46    0.0002 (  0.0%)   0.0000 (  0.0%)   0.0002 (  0.0%)   0.0002 (  0.0%)  GlobalDCEPass
00:01:46    0.0054 (  0.0%)   0.0002 (  0.0%)   0.0056 (  0.0%)   0.0002 (  0.0%)  OpenMPOptCGSCCPass
00:01:46    0.0004 (  0.0%)   0.0001 (  0.0%)   0.0005 (  0.0%)   0.0002 (  0.0%)  MoveAutoInitPass
00:01:46    0.0006 (  0.0%)   0.0001 (  0.0%)   0.0006 (  0.0%)   0.0002 (  0.0%)  CoroSplitPass
00:01:46    0.0009 (  0.0%)   0.0001 (  0.0%)   0.0010 (  0.0%)   0.0002 (  0.0%)  CoroElidePass
00:01:46    0.0005 (  0.0%)   0.0001 (  0.0%)   0.0006 (  0.0%)   0.0002 (  0.0%)  SpeculativeExecutionPass
00:01:46    0.0001 (  0.0%)   0.0000 (  0.0%)   0.0001 (  0.0%)   0.0001 (  0.0%)  NameAnonGlobalPass
00:01:46    0.0001 (  0.0%)   0.0000 (  0.0%)   0.0001 (  0.0%)   0.0001 (  0.0%)  InvalidateAnalysisPass<llvm::ShouldNotRunFunctionPassesAnalysis>
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  AnnotationRemarksPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  RequireAnalysisPass<llvm::ProfileSummaryAnalysis, llvm::Module, llvm::AnalysisManager<Module>>
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  OpenMPOptPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  CoroEarlyPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  Annotation2MetadataPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  ForceFunctionAttrsPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  CoroCleanupPass
00:01:46    0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  CanonicalizeAliasesPass
00:01:46   96.0900 (100.0%)   8.2465 (100.0%)  104.3365 (100.0%)  101.2781 (100.0%)  Total

[nikic]

Possibly related to #115970. The crate names are similar :)

[FauxFaux]

Yup, those are the same crate / author / code; that's another application that links the same library.

[FauxFaux]

Looks like y'all are way ahead of me, let's close this one?