Skip to content

[mem_cache][6/N] refactor: move MHA host-pool into pool_host/mha.py #30249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hzh0425 merged 2 commits into from
Jul 8, 2026
Merged

[mem_cache][6/N] refactor: move MHA host-pool into pool_host/mha.py #30249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5c6b17c
[mem_cache][6/N] refactor: move MHA host-pool family into pool_host/m…
alphabetc1 Jul 6, 2026
c63c63c
Merge branch 'main' into refactor/mem-cache-poolhost-mha
alphabetc1 Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion benchmark/hf3fs/bench_zerocopy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
)
from sglang.srt.mem_cache.allocator import TokenToKVPoolAllocator
from sglang.srt.mem_cache.memory_pool import MHATokenToKVPool
from sglang.srt.mem_cache.memory_pool_host import MHATokenToKVPoolHost
from sglang.srt.mem_cache.pool_host.mha import MHATokenToKVPoolHost

init_distributed_environment(
world_size=1,
Expand Down
6 changes: 2 additions & 4 deletions python/sglang/srt/disaggregation/decode_kvcache_offload_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,8 @@
MLATokenToKVPool,
ReqToTokenPool,
)
from sglang.srt.mem_cache.memory_pool_host import (
MLATokenToKVPoolHost,
get_mha_host_pool_cls,
)
from sglang.srt.mem_cache.memory_pool_host import MLATokenToKVPoolHost
from sglang.srt.mem_cache.pool_host.mha import get_mha_host_pool_cls
from sglang.srt.server_args import ServerArgs
from sglang.srt.utils.common import ceil_align

Expand Down
6 changes: 2 additions & 4 deletions python/sglang/srt/mem_cache/hiradix_cache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,8 @@
MiniMaxSparseKVPool,
MLATokenToKVPool,
)
from sglang.srt.mem_cache.memory_pool_host import (
MLATokenToKVPoolHost,
get_mha_host_pool_cls,
)
from sglang.srt.mem_cache.memory_pool_host import MLATokenToKVPoolHost
from sglang.srt.mem_cache.pool_host.mha import get_mha_host_pool_cls
from sglang.srt.mem_cache.radix_cache import (
RadixCache,
RadixKey,
Expand Down
4 changes: 3 additions & 1 deletion python/sglang/srt/mem_cache/hybrid_cache/hybrid_pool_assembler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@
HostPoolGroup,
LogicalHostPool,
MambaPoolHost,
MHATokenToKOnlyPoolHost,
MLATokenToKVPoolHost,
PoolEntry,
)
from sglang.srt.mem_cache.pool_host.mha import (
MHATokenToKOnlyPoolHost,
get_mha_host_pool_cls,
)
from sglang.srt.mem_cache.unified_cache_components import ComponentType
Expand Down
6 changes: 2 additions & 4 deletions python/sglang/srt/mem_cache/kv_cache_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,8 @@ def maybe_register_hicache_draft(
MHATokenToKVPool,
MLATokenToKVPool,
)
from sglang.srt.mem_cache.memory_pool_host import (
MLATokenToKVPoolHost,
get_mha_host_pool_cls,
)
from sglang.srt.mem_cache.memory_pool_host import MLATokenToKVPoolHost
from sglang.srt.mem_cache.pool_host.mha import get_mha_host_pool_cls

pool = draft_kv_pool
if isinstance(pool, HybridLinearKVPool):
Expand Down
Loading
Loading