[LTO] Sort DefinedGlobals in LTO cache key#210025
Merged
Merged
Conversation
Sort the DefindeGlobals by GUID when computing the LTO cache key, to avoid making the cache key dependent on the insertion order. This fixes large compile-time regressions in rust incremental builds. Alternatively one could make collectDefinedGVSummariesPerModule() work on the sortedRange(), but as that is also used in other places, it probably makes sense to sort locally. The issue was introduced in 760bb06.
llvmorg-github-actions
Bot
added
the
LTO
|
@llvm/pr-subscribers-lto Author: Nikita Popov (nikic) ChangesSort the DefindeGlobals by GUID when computing the LTO cache key, to avoid making the cache key dependent on the insertion order. This fixes large compile-time regressions in rust incremental builds. Alternatively one could make collectDefinedGVSummariesPerModule() work on the sortedRange(), but as that is also used in other places, it probably makes sense to sort locally. The issue was introduced in 760bb06. Full diff: https://github.com/llvm/llvm-project/pull/210025.diff 1 Files Affected:
diff --git a/llvm/lib/LTO/LTO.cpp b/llvm/lib/LTO/LTO.cpp
index b0fd1d77a2d66..4863ab2973948 100644
--- a/llvm/lib/LTO/LTO.cpp
+++ b/llvm/lib/LTO/LTO.cpp
@@ -306,7 +306,10 @@ std::string llvm::computeLTOCacheKey(
// Include the hash for the linkage type to reflect internalization and weak
// resolution, and collect any used type identifier resolutions.
- for (auto &GS : DefinedGlobals) {
+ SmallVector<std::pair<GlobalValue::GUID, GlobalValueSummary *>>
+ SortedDefinedGlobals(DefinedGlobals.begin(), DefinedGlobals.end());
+ llvm::sort(SortedDefinedGlobals, llvm::less_first());
+ for (auto &GS : SortedDefinedGlobals) {
GlobalValue::LinkageTypes Linkage = GS.second->linkage();
Hasher.update(
ArrayRef<uint8_t>((const uint8_t *)&Linkage, sizeof(Linkage)));
|
nocchijiang
approved these changes
Jul 16, 2026
nocchijiang
left a comment
nocchijiang
left a comment
Contributor
There was a problem hiding this comment.
Could we add a regression test for this? Otherwise LGTM. The failed Linux check looks unrelated.
Contributor
Author
|
I added a test that fails before the change and passes after. It's not quite representative of what happens in Rust (which does not use linkonce_odr), but demonstrates the general issue at least. |
teresajohnson
approved these changes
Jul 16, 2026
| // Include the hash for the linkage type to reflect internalization and weak | ||
| // resolution, and collect any used type identifier resolutions. | ||
| for (auto &GS : DefinedGlobals) { | ||
| SmallVector<std::pair<GlobalValue::GUID, GlobalValueSummary *>> |
Contributor
There was a problem hiding this comment.
Add a comment above this about why it is needed.
nikic
enabled auto-merge (squash)
July 17, 2026 07:16
Contributor
Author
|
/cherry-pick 90e31c4 |
Member
|
/pull-request #210274 |
dyung
pushed a commit
to llvmbot/llvm-project
that referenced
this pull request
Jul 17, 2026
Sort the DefindeGlobals by GUID when computing the LTO cache key, to avoid making the cache key dependent on the insertion order. This fixes large compile-time regressions in rust incremental builds. Alternatively one could make collectDefinedGVSummariesPerModule() work on the sortedRange(), but as that is also used in other places, it probably makes sense to sort locally. The issue was introduced in 760bb06. (cherry picked from commit 90e31c4)
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Jul 24, 2026
Update to LLVM 23 Changes in this PR (apart from the LLVM update): * Updated cov-map tests due to minor ordering changes introduced in llvm/llvm-project@5931034. The affected tests are limited to LLVM >= 23 now. * Set `LLVM_VERSIONED_DYLIB_NAME_ON_DARWIN=OFF`, which restores the previous unversioned naming of the LLVM dylib on Darwin. I expect we'll want to change this for consistency with Linux, but given the amount of trouble Darwin dylibs have caused in the past, I'm not trying to do it in this PR. * Distribute the llvm-project/libc subproject, which is now a build dependency of LLVM. * Pull in f128 Windows ABI changes from #158778. These need to land in sync with the LLVM update (as ABI of libcalls is controlled by LLVM, not our ABI lowering). * Disable RISCV backend on dist-i686-mingw, because the build OOMs on a large generated file. This is a temporary measure until host tools are removed by rust-lang/compiler-team#1020. * The llvm-project fork includes a new downstream patch for mingw: rust-lang/llvm-project@b89cd8e It works around mingw having broken TLS prior to GCC 16. (We're currently on GCC 14.) Issues: * [x] llvm/llvm-project#209512 * [x] llvm/llvm-project#209714 * [x] llvm/llvm-project#209718 * [x] rust-lang/stdarch#2190 (synced in #159517) * [x] llvm/llvm-project#210025 * [x] #159391 * [x] EnzymeAD/Enzyme#2951 * [x] #159301 * [x] llvm/llvm-project#210300
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Jul 24, 2026
Update to LLVM 23 Changes in this PR (apart from the LLVM update): * Updated cov-map tests due to minor ordering changes introduced in llvm/llvm-project@5931034. The affected tests are limited to LLVM >= 23 now. * Set `LLVM_VERSIONED_DYLIB_NAME_ON_DARWIN=OFF`, which restores the previous unversioned naming of the LLVM dylib on Darwin. I expect we'll want to change this for consistency with Linux, but given the amount of trouble Darwin dylibs have caused in the past, I'm not trying to do it in this PR. * Distribute the llvm-project/libc subproject, which is now a build dependency of LLVM. * Pull in f128 Windows ABI changes from #158778. These need to land in sync with the LLVM update (as ABI of libcalls is controlled by LLVM, not our ABI lowering). * Disable RISCV backend on dist-i686-mingw, because the build OOMs on a large generated file. This is a temporary measure until host tools are removed by rust-lang/compiler-team#1020. * The llvm-project fork includes a new downstream patch for mingw: rust-lang/llvm-project@b89cd8e It works around mingw having broken TLS prior to GCC 16. (We're currently on GCC 14.) Issues: * [x] llvm/llvm-project#209512 * [x] llvm/llvm-project#209714 * [x] llvm/llvm-project#209718 * [x] rust-lang/stdarch#2190 (synced in #159517) * [x] llvm/llvm-project#210025 * [x] #159391 * [x] EnzymeAD/Enzyme#2951 * [x] #159301 * [x] llvm/llvm-project#210300
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Jul 25, 2026
Update to LLVM 23 Changes in this PR (apart from the LLVM update): * Updated cov-map tests due to minor ordering changes introduced in llvm/llvm-project@5931034. The affected tests are limited to LLVM >= 23 now. * Set `LLVM_VERSIONED_DYLIB_NAME_ON_DARWIN=OFF`, which restores the previous unversioned naming of the LLVM dylib on Darwin. I expect we'll want to change this for consistency with Linux, but given the amount of trouble Darwin dylibs have caused in the past, I'm not trying to do it in this PR. * Distribute the llvm-project/libc subproject, which is now a build dependency of LLVM. * Pull in f128 Windows ABI changes from #158778. These need to land in sync with the LLVM update (as ABI of libcalls is controlled by LLVM, not our ABI lowering). * Disable RISCV backend on dist-i686-mingw, because the build OOMs on a large generated file. This is a temporary measure until host tools are removed by rust-lang/compiler-team#1020. * The llvm-project fork includes a new downstream patch for mingw: rust-lang/llvm-project@b89cd8e It works around mingw having broken TLS prior to GCC 16. (We're currently on GCC 14.) Issues: * [x] llvm/llvm-project#209512 * [x] llvm/llvm-project#209714 * [x] llvm/llvm-project#209718 * [x] rust-lang/stdarch#2190 (synced in #159517) * [x] llvm/llvm-project#210025 * [x] #159391 * [x] EnzymeAD/Enzyme#2951 * [x] #159301 * [x] llvm/llvm-project#210300
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Jul 25, 2026
Update to LLVM 23 Changes in this PR (apart from the LLVM update): * Updated cov-map tests due to minor ordering changes introduced in llvm/llvm-project@5931034. The affected tests are limited to LLVM >= 23 now. * Set `LLVM_VERSIONED_DYLIB_NAME_ON_DARWIN=OFF`, which restores the previous unversioned naming of the LLVM dylib on Darwin. I expect we'll want to change this for consistency with Linux, but given the amount of trouble Darwin dylibs have caused in the past, I'm not trying to do it in this PR. * Distribute the llvm-project/libc subproject, which is now a build dependency of LLVM. * Pull in f128 Windows ABI changes from #158778. These need to land in sync with the LLVM update (as ABI of libcalls is controlled by LLVM, not our ABI lowering). * Disable RISCV backend on dist-i686-mingw, because the build OOMs on a large generated file. This is a temporary measure until host tools are removed by rust-lang/compiler-team#1020. * The llvm-project fork includes a new downstream patch for mingw: rust-lang/llvm-project@b89cd8e It works around mingw having broken TLS prior to GCC 16. (We're currently on GCC 14.) Issues: * [x] llvm/llvm-project#209512 * [x] llvm/llvm-project#209714 * [x] llvm/llvm-project#209718 * [x] rust-lang/stdarch#2190 (synced in #159517) * [x] llvm/llvm-project#210025 * [x] #159391 * [x] EnzymeAD/Enzyme#2951 * [x] #159301 * [x] llvm/llvm-project#210300
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Jul 25, 2026
Update to LLVM 23 Changes in this PR (apart from the LLVM update): * Updated cov-map tests due to minor ordering changes introduced in llvm/llvm-project@5931034. The affected tests are limited to LLVM >= 23 now. * Set `LLVM_VERSIONED_DYLIB_NAME_ON_DARWIN=OFF`, which restores the previous unversioned naming of the LLVM dylib on Darwin. I expect we'll want to change this for consistency with Linux, but given the amount of trouble Darwin dylibs have caused in the past, I'm not trying to do it in this PR. * Distribute the llvm-project/libc subproject, which is now a build dependency of LLVM. * Pull in f128 Windows ABI changes from #158778. These need to land in sync with the LLVM update (as ABI of libcalls is controlled by LLVM, not our ABI lowering). * Disable RISCV backend on dist-i686-mingw, because the build OOMs on a large generated file. This is a temporary measure until host tools are removed by rust-lang/compiler-team#1020. * The llvm-project fork includes a new downstream patch for mingw: rust-lang/llvm-project@b89cd8e It works around mingw having broken TLS prior to GCC 16. (We're currently on GCC 14.) Issues: * [x] llvm/llvm-project#209512 * [x] llvm/llvm-project#209714 * [x] llvm/llvm-project#209718 * [x] rust-lang/stdarch#2190 (synced in #159517) * [x] llvm/llvm-project#210025 * [x] #159391 * [x] EnzymeAD/Enzyme#2951 * [x] #159301 * [x] llvm/llvm-project#210300
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sort the DefindeGlobals by GUID when computing the LTO cache key, to avoid making the cache key dependent on the insertion order. This fixes large compile-time regressions in rust incremental builds.
Alternatively one could make collectDefinedGVSummariesPerModule() work on the sortedRange(), but as that is also used in other places, it probably makes sense to sort locally.
The issue was introduced in 760bb06.