Repository navigation
[Perf] unified-memory: Reduce allocation and SWA retirement overhead - #44970
Merged
Merged
Conversation
ch-wan
requested review from
alphabetc1,
hnyls2002,
huangtingwei9988,
hzh0425,
ispobock and
xiezhq-hermann
as code owners
October 18, 2026 07:22
3 of 4 tasks
This was referenced Oct 18, 2026
ch-wan
force-pushed
the
cheng/dev/unified-allocator-overhead
branch
from
October 18, 2026 08:13
88e2d59 to
c59c07f
Compare
ch-wan
requested review from
BBuf,
DarkSharpness,
HaiShaw,
HydraQYH,
celve and
yuan-luo
as code owners
October 18, 2026 08:13
ch-wan
force-pushed
the
cheng/refactor/decode-metadata-mappings
branch
from
October 18, 2026 08:13
95903cb to
d9b03fe
Compare
ch-wan
force-pushed
the
cheng/refactor/decode-metadata-mappings
branch
from
October 18, 2026 09:02
d9b03fe to
9273bf9
Compare
ch-wan
force-pushed
the
cheng/dev/unified-allocator-overhead
branch
from
October 18, 2026 09:02
c59c07f to
68553ac
Compare
ch-wan
force-pushed
the
cheng/refactor/decode-metadata-mappings
branch
from
October 18, 2026 12:39
9273bf9 to
d68283d
Compare
ch-wan
requested review from
Fridge003,
Jialin,
Qiaolin-Yu,
hanming-lu,
hebiao064 and
yizhang2077
as code owners
October 18, 2026 12:39
Base automatically changed from
cheng/refactor/decode-metadata-mappings
to
main
October 18, 2026 12:39
ch-wan
force-pushed
the
cheng/dev/unified-allocator-overhead
branch
from
October 18, 2026 12:39
68553ac to
d541e80
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of a stack (oldest at bottom):
Current head
d541e806cd, based on maina5b21145e1. Stack predecessor: #44969. The Files changed tab isolates this PR from its predecessor.Motivation
Unified allocation and sliding-window retirement repeatedly queried stable geometry and launched avoidable copies, mapping lookups, and tombstone writes in the scheduler path.
Modifications
The bind kernels reuse compiled code when consuming an odd number of int64 free-list entries changes pointer alignment. A GPU test verifies both mappings and JIT-cache reuse; the prefix-view comment states its lifetime contract.
CI registration updates
Updated
est_timein 1 affected test files from observed file-level CI timings, including process/import cost and margin. Existing estimates with sufficient margin are retained. The expanded weekly configurator test also passed all 11 cases in a 30.1-second CPU measurement.The CI registry check validated 2,960 registrations before landing. An AST comparison confirmed that this follow-up changes registration metadata only. Runtime test measurements elsewhere in this description refer to their stated earlier revisions.
Accuracy Tests
Earlier combined stack
b550cacde6adds a test-fixture correction in #43326: the Gemma4 read-stream fixture declares its FULL address space, and the HiSparse registry test uses realCacheInitParamsfor the retained-window calculation. Both complete test files pass locally with GPUs hidden: 4 Gemma4 tests and 26 registry tests. Pre-commit passed. The five descendant patches are unchanged (git range-diff); this follow-up changes no production code. The earlier 455-test run did not cover these two files; GitHub CI has not yet been revalidated for this head.Validation before the test-fixture follow-up, on combined stack
4cf27a551c: 455 tests and 921 subtests passed on B300, plus the two widened-history regressions against Rust TreeCore. The new FULL-fallback metadata cases cover eager execution and CUDA graph replay. New allocation-reclaim, explicit-prefill and JIT-reuse regressions fail on the previous code and pass with the fixes. These are combined-stack checks, not independent model reruns of each PR.157 tests and 732 subtests passed on B300 at the previous head
88e2d59d29, covering multi-ended allocation, capacity memoization, no-sync frees, and allocation eviction.Speed Tests and Profiling
Performance validation is reported for the final combined stack below; this run does not attribute an isolated throughput gain to this PR.
Prior combined-stack E2E output/acceptance and throughput results: #44973. These measurements include all five changes before the review updates.
Checklist
CI States
Latest PR Test (Base): 🚫 Run #39128235423
Latest PR Test (Extra): ❌ Run #39128235132
Latest PR Test (AMD ROCm 10): ❌ Run #39128235129