Skip to content

[Unified Cache][1/N]: Support cache contract for external linker - #37091

Merged
hzh0425 merged 2 commits into
sgl-project:mainfrom
hzh0425:external-linker-cache-contract
Aug 30, 2026
Merged

hzh0425 merged 2 commits into
sgl-project:mainfrom
hzh0425:external-linker-cache-contract

Conversation

@hzh0425

@hzh0425 hzh0425 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

This is the first PR in the external linker series.

This PR is split from the original full PR: #35687

  • Add component-level transfer and load lifecycle contracts for external cache linkers.
  • Track adopted Full/SWA ranges during UnifiedTree insertion for deduplicated remote loads.
  • Add restorable-prefix metadata and external cache state to UnifiedTree.

This PR only introduces the cache-side contracts. Linker wiring and backends will follow in subsequent PRs.

Test Plan

  • pre-commit run --from-ref HEAD^ --to-ref HEAD

CI States

Latest PR Test (Base): ✅ Run #33297464622
Latest PR Test (Extra): ✅ Run #33297464442
Latest PR Test (AMD ROCm 7.2): ❌ Run #33297464616

@hzh0425

hzh0425 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/mem_cache/test_unified_radix_cache_unittest.py

@hzh0425

hzh0425 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-group radix_cache/unified_radix_tree

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/mem_cache/test_unified_radix_cache_unittest.py:

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_unified_radix_cache_unittest.py

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Results for /rerun-group radix_cache/unified_radix_tree:

🚀 4-gpu-h100 (4 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_hicache_pp_kl.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_cp.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dsv4.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_mamba.py

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dcp.py

🚀 8-gpu-h200 (3 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dsv4_pp.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_glm52.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_mimo.py

🚀 2-gpu-h100 (2 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py
cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_swa.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_hybrid_bitexact.py

Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
@hzh0425
hzh0425 force-pushed the external-linker-cache-contract branch from ee17743 to 01636dc Compare August 30, 2026 06:23
@hzh0425 hzh0425 changed the title [Unified Cache] Support cache contract for external linker [Unified Cache Linker][1/N]: Support cache contract for external linker Aug 30, 2026
@hzh0425 hzh0425 changed the title [Unified Cache Linker][1/N]: Support cache contract for external linker [Unified Cache][1/N]: Support cache contract for external linker Aug 30, 2026
@hzh0425

hzh0425 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator Author
image

@hzh0425
hzh0425 merged commit c9eb475 into sgl-project:main Aug 30, 2026
216 of 237 checks passed
saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 31, 2026
nzr-niu pushed a commit to nzr-niu/sglang that referenced this pull request Sep 1, 2026
RolaoDenthu pushed a commit to RolaoDenthu/sglang that referenced this pull request Sep 1, 2026
EdwardXuy added a commit to EdwardXuy/sglang-Ascend that referenced this pull request Sep 1, 2026
DeepSeek-V4 needs --tp-size 16/--dp-size 16, so the 2-card linux-aarch64-a3-2 runner cannot host it; switch to linux-aarch64-a3-16. Replace the two 20260828 images with main-cann9.0.0-a3 (daily build, contains sgl-project#37091 -> expected RED) plus the 20260828-cann9.0.0-a3 snapshot (control -> expected GREEN).
EdwardXuy added a commit to EdwardXuy/sglang-Ascend that referenced this pull request Sep 1, 2026
The bundled image sglang cannot reproduce sgl-project#37091: the main image crashes at cuda-graph capture (aclnnHcPre missing in libopapi.so) and the 8/28 image predates the PR. Checkout upstream sglang main and override via PYTHONPATH so the recover_after_unevict result-param change is exercised, while overlaying the ascend test utils from this PR's checkout.
EdwardXuy added a commit to EdwardXuy/sglang-Ascend that referenced this pull request Sep 2, 2026
Both cann9.0.0 images fail at capture_decode_graph with 'aclnnHcPre not in libopapi.so': the DSV4 mHC pre-norm path calls torch.ops.custom.npu_hc_pre, which cann9.0.0 does not ship. Use the 20260828 cann9.1.0 snapshot (has HcPre) and keep the PYTHONPATH override to upstream main (contains sgl-project#37091).
StevenChenSE pushed a commit to StevenChenSE/sglang that referenced this pull request Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants