Skip to content

[Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check - #39980

Merged
alphabetc1 merged 1 commit into
sgl-project:mainfrom
alphabetc1:fix/sanity-host-locked-aux-lru
Sep 18, 2026
Merged

alphabetc1 merged 1 commit into
sgl-project:mainfrom
alphabetc1:fix/sanity-host-locked-aux-lru

Conversation

@alphabetc1

@alphabetc1 alphabetc1 commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

sanity_check raises while a storage-backup or load-back host lock is in flight.

The aux (SWA / Mamba) host-lock acquire delists its node from the host LRU on the
0 -> 1 transition and relists it on -> 0, so the eviction walker cannot pick a
node whose host slice is being read. sanity_check did not know about that: it
required every host-only aux node to be in the host LRU, so any outstanding host
lock was reported as <component> host LRU mismatch.

SGLANG_ENABLE_TREE_CACHE_SANITY_CHECK defaults to on in CI, and
InvariantChecker._check_tree_cache runs it for hybrid-SWA / hybrid-SSM models, so
a HiCache backup in flight when the checker fires turns into an intermittent CI
failure that is not a product defect.

Modifications

Exempt nodes with host_lock_ref > 0 from the aux host-LRU membership comparison in
both tree cores. The exemption is two-sided on purpose: acquire delists a node, but a
node locked while already listed stays listed (eviction relists it at the device
tombstone), so both states are legal for a locked node.

The unlocked invariant is unchanged and still enforced -- an unlocked host-only value
missing from the LRU is still a violation
(sanity_check_detects_a_host_only_value_missing_from_the_lru).

sanity_check_detects_a_host_locked_value_missing_from_the_lru asserted the behaviour
being removed, so it becomes the positive case and now shares the fixture helper with
its two siblings. No production behaviour changes; this is a diagnostic-only fix.

Accuracy Tests

  • cargo test --locked --lib: 867 passed, 0 failed, 1 ignored.
  • test_unified_radix_cache_unittest.py: 1256 passed, 1435 skipped, 70 subtests.
  • Targeted repro on one H200, driving a real UnifiedRadixCache at the tree API:
    sanity_check passes before, during and after an outstanding storage-backup host
    lock (it raised during the lock before this change).
  • Randomized lock-ledger stress (insert / match / split / lock / early SWA release /
    release / evict), 6 seeds x 150 steps for full, full,swa and full,swa,mamba:
    0 invariant violations in all three.

Checklist

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ✅ Run #35295275230
Latest PR Test (Extra): ✅ Run #35295274985
Latest PR Test (AMD ROCm 10): ❌ Run #35295275311

@alphabetc1 alphabetc1 added run-ci CI: run the baseline test suite on this PR bypass-fastfail run-ci-extra CI: also run the extra suite (requires run-ci) labels Sep 18, 2026
@alphabetc1
alphabetc1 force-pushed the fix/sanity-host-locked-aux-lru branch from b0b7f8c to 64b3d6d Compare September 18, 2026 01:20
@alphabetc1
alphabetc1 force-pushed the fix/sanity-host-locked-aux-lru branch from 64b3d6d to 6ee86da Compare September 18, 2026 01:25
@alphabetc1

Copy link
Copy Markdown
Collaborator Author

/rerun-group radix_cache/unified_radix_tree

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-group radix_cache/unified_radix_tree:

🚀 4-gpu-h100 (5 tests): ❌ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/linker/test_unified_cache_linker_kl_dsv4.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_hicache_pp_kl.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_cp.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dsv4.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_mamba.py

🚀 8-gpu-h200 (4 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/linker/test_unified_cache_linker_kl_glm52.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dsv4_pp.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_glm52.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_mimo.py

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dcp.py

🚀 2-gpu-h100 (2 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_swa.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_hybrid_bitexact.py

@alphabetc1
alphabetc1 enabled auto-merge (squash) September 18, 2026 13:55
@alphabetc1
alphabetc1 merged commit 191172f into sgl-project:main Sep 18, 2026
292 of 340 checks passed
@alphabetc1
alphabetc1 deleted the fix/sanity-host-locked-aux-lru branch September 19, 2026 02:00
ishandhanani added a commit that referenced this pull request Sep 21, 2026
Picks up the host-locked aux-node exemption in sanity_check (#39980), the
DSV4 rope pool names, and the HiCache transfer-argument documentation. The
only conflict was the Mamba load-back test, which keeps main's argument
annotations without the .squeeze() this branch removed when the binding
started normalizing the request slot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bypass-fastfail run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants