Repository navigation
Capture KV host transfer buffers - #43711
Merged
Merged
Conversation
ispobock
requested review from
Ying1123,
hanming-lu,
hnyls2002,
merrymercy and
xiezhq-hermann
as code owners
October 11, 2026 15:53
ispobock
added
run-ci-extra
ispobock
requested review from
alphabetc1,
huangtingwei9988 and
hzh0425
as code owners
October 11, 2026 15:53
Collaborator
Author
|
/rerun-test test/registered/unit/mem_cache/pool_host/test_kvcacheio_asymmetric.py test/registered/unit/mem_cache/test_asymmetric_mha_pool_host_unit.py test/registered/unit/mem_cache/test_hicache_dcp_host_pool.py test/registered/unit/mem_cache/test_hicache_host_register.py test/registered/unit/mem_cache/test_hicache_seaweedfs_storage.py test/registered/unit/mem_cache/test_hicache_staged_write_back_dispatch.py test/registered/unit/mem_cache/test_mem_pool_host.py test/registered/unit/mem_cache/test_mla_host_dedup_primitives.py |
Collaborator
Author
|
/rerun-test test/registered/hicache/test_hicache_variants.py test/registered/hicache/test_qwen35_hicache.py test/registered/hicache/test_hicache_spec_file_storage.py test/registered/e2e/hicache/test_hicache_unified_memory.py test/registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_dcp.py test/registered/kernels/ops/kvcache/test_hicache_strided_device_rows.py test/registered/kernels/ops/kvcache/test_mxfp8_host_allocation.py |
Contributor
|
🚀 🚀 🚀 🚀 🚀 |
Contributor
|
🚀 🚀 |
3 of 5 tasks
reger-men
added a commit
to reger-men/sglang
that referenced
this pull request
Oct 12, 2026
pool_host/base.py: import conflict only, main dropped the KVCache import, kept take_contiguous_run. alloc() and _merge_release_slots() are unchanged on main. pool_host/mla.py merged without a conflict but the layer_first kernel reload still passed device_pool.kv_buffer[device_layer_id] to bulk_reload. After sgl-project#43711 that name only exists on the kernel_ascend path, so point bulk_reload at the captured self.device_kv_buffers[layer_id] like the jit and transfer_kv_per_layer_mla calls next to it (this also picks the right buffer for packed MTP draft layers). Signed-off-by: adlashab <adlashab@amd.com>
Liwansi
added a commit
to Liwansi/sglang_vans
that referenced
this pull request
Oct 13, 2026
…n_allgather * 'main' of https://github.com/sgl-project/sglang: (236 commits) [NPU] Stabilize HiCache and registered CI tests (sgl-project#43308) [Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) (sgl-project#43799) [Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD (sgl-project#43802) [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (sgl-project#40323) [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (sgl-project#38603) [XPU] Support decode context parallelism (DCP) on Intel XPU (sgl-project#34355) Keep NIXL Mamba checkpoint keys separate by TP rank (sgl-project#43594) Capture KV host transfer buffers (sgl-project#43711) [CI] Move tests that do not need GB300 off the per-PR GB300 suite (sgl-project#43784) [HiCache] Preserve all MTP KV and recurrent state (sgl-project#40223) [HiCache] Support Mamba in buffer-only host memory mode (sgl-project#36345) [PD] Poison the deferred abort ack when a Mooncake send fails (sgl-project#43719) [Perf] Route BF16 mixed-batch decode tails through decode attention (sgl-project#43238) [Fix] Link cuda_ipc JIT against CUDA driver stubs dir (sgl-project#43764) docs: clarify prefill CP terminology and simplify wording (sgl-project#43787) [diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (sgl-project#41940) [Fix] Adapt to transformers 5.18 / 5.19 changes (sgl-project#43688) [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (sgl-project#42048) [Deps] Bump FlashInfer to 0.7.1 (sgl-project#43494) docs: sync LMSYS SGLang blog cards (sgl-project#43462) ...
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Capture MHA/MLA transfer buffers and pointer tables during host initialization. This removes repeated target/draft buffer selection from transfer callbacks and makes token-size calculation independent of initialization side effects.
Validation: 122 CPU tests and 34 subtests passed.
CI States
Latest PR Test (Base): ✅ Run #38153176669
Latest PR Test (Extra): ❌ Run #38153176492
Latest PR Test (AMD ROCm 10): ❌ Run #38153176661