Repository navigation
[mem_cache] test: ratchet the allocator and pool_host layout - #35638
Draft
alphabetc1 wants to merge 1 commit into
Draft
alphabetc1 wants to merge 1 commit into
alphabetc1 wants to merge 1 commit into
Conversation
This was referenced Aug 20, 2026
Merged
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
alphabetc1
force-pushed
the
refactor/mem-cache-layout-ratchet
branch
from
September 2, 2026 10:44
164e0cc to
b855c8c
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Part of #25371. A directory is not a rule.
allocator/landed in May andpool_host/has been landing since June, but nothingstops new code from ignoring them. #29678 added
multi_ended_allocator.py— 2559lines, 3 allocator classes — at the
mem_cache/top level two months afterallocator/existed. And while the split was being agreed,memory_pool.pygrew from2257 to 5042 lines and from 11 to 17 classes.
The restructure only converges if the layout cannot regress while it is in progress.
This adds that guard, plus the skill that tells coding agents the same rules.
Modifications
test/registered/unit/mem_cache/test_mem_cache_layout_ratchet.py— four shrink-onlypins, each covering one way the layout regressed:
_LEGACY_HOMES_TOP_LEVEL_MODULESmem_cache/root_SHRINKING_MODULES_FORBIDDEN_DEPSRole is decided by transitive base class, never by class name —
HostTensorAllocatoris named
Allocatorbut hands out pinned host memory, not KV slots. Class names docollide (three modules define
TreeNode), so ancestry is unioned across same-nameddefinitions: over-approximating turns a collision into a review conversation, where
keeping only one definition would turn it into a silent miss.
R1 covers
allocator/andpool_host/only.mem_cache/pool/does not exist yet,and an error telling someone to file a class into a package nobody has created is a
trap, not a rule. R3 stands in for the pool layer meanwhile — it needs no target
package to exist, and it is the rule that catches the case R1 and R2 both miss: a class
appended to a file that already exists, which is exactly how
memory_pool.pygainedPageMajorMHATokenToKVPool,MHATokenToKVPoolMXFP8,MHATokenToKOnlyPool,MiniMaxSparseKVPoolandDSATokenToKVPool. When the firstpool/PR lands,pooljoins the enforced roles and its classes move from R3's pin into
_LEGACY_HOMES.Every pin also fails when it rots — a class that has moved, a module that is gone.
So each migration PR under #25371 shrinks a pin, and the pins double as the roadmap's
progress ledger.
R4 passes on
maintoday, so it locks in a property the code already has rather thandemanding new work. Scope is
python/sglang/srt/mem_cache/only;hardware_backend/is explicitly excluded as a separate device axis.
Also adds
.claude/skills/mem-cache-layout/SKILL.mdand its trigger line in.claude/rules/modify-component-must-read.md. The skill teaches; only the ratchetblocks — human PRs never read
.claude/skills/.Accuracy Test
Not applicable — no runtime code changes. The test is AST-only and reads no tensors.
Benchmark and Profiling Results
Not applicable.
Checklist
Stacked on #35306 (
[mem_cache][9/N] DSAIndexerPoolHost). The pins are generatedagainst a tree with that move applied. Draft until #35306 merges; landing this first
instead would only mean adding
DSAIndexerPoolHostback to_LEGACY_HOMESand_SHRINKING_MODULES["memory_pool_host.py"].Verification
Run on a GB300 devbox against the real package (
PYTHONPATHshadowing confirmed viasglang.srt.__path__, so the test reads the working tree, not the image's copy):A guard nobody has seen go red is not a guard, so all six failure paths were exercised
by injecting the violation into a scratch copy of the tree — the four rules above, plus
the two rot cases (a pinned class that moved, a pinned module that was deleted). Each
fired with an actionable message naming the target file. The
HostKVCache-subclassinjection was additionally reproduced on the devbox and the tree restored (md5 checked).
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #33621060344
Latest PR Test (Extra): ❌ Run #33621060175
Latest PR Test (AMD ROCm 7.2): ❌ Run #33621060311