[LucasWilkinson]

Purpose

Large HiSparse host pools (384–480 GiB) are pinned twice on 4 KiB pages: by cudaHostRegister and by NIXL/RDMA registration. In the shared TP pool, every rank also registers the whole pool. At 450 GiB that's about 118M pages per pass, and we've seen it wedge K8s nodes during startup.

This PR adds an opt-in host_pool_dir key on HiSparseConnector. When it's set, the host pool is a file-backed mmap in that directory, typically a hugetlbfs mount. With 1 GiB pages, 450 GiB becomes 450 pages, so pinning and MR translation tables become trivial.

{"kv_connector": "HiSparseConnector", "kv_role": "kv_both",
 "kv_connector_extra_config": {"host_pool_gib": 384, "host_pool_dir": "/dev/hugepages-1Gi"}}

On K8s: emptyDir: {medium: HugePages-1Gi} plus resources.limits.hugepages-1Gi: <pool size>. The node must reserve 1 GiB huge pages.

Changes:

• SharedOffloadRegion gets a shm_dir argument, defaulting to /dev/shm, so existing behavior doesn't change. The mapping is rounded to the filesystem page size (statvfs().f_bsize); callers still see exactly total_size_bytes. On hugetlbfs the tmpfs free-space check is skipped, because hugetlbfs statfs reports 0 free without a size= limit. mmap reserves the huge pages instead, and an ENOMEM is turned into a clear error. A hugetlbfs region always uses one backing file, because huge-page files can't be stitched together at arbitrary offsets the way num_segments does.
• HiSparse shared pool (TP>1, mp): passes shm_dir through. Each rank's prefault shard is rounded to the huge page size, so ranks don't fault the same huge page. It skips the psutil RAM check, because reserved huge pages don't count toward MemAvailable.
• HiSparse private pool: when host_pool_dir is set, allocates an unlinked, pre-faulted file mapping in that directory instead of torch.empty.

Not a duplicate: #45693 (draft, needs rebase) adds hugetlbfs support to the OffloadingConnector TieringOffloadingSpec. This PR targets the HiSparse host pool, which that PR doesn't touch. The two overlap only in SharedOffloadRegion, and the change there is kept minimal so they can be reconciled.

Status: mockup / draft. Not yet validated on a real hugetlbfs mount (see test plan).

Test Plan / Results

• pytest tests/v1/kv_offload/cpu tests/v1/worker/test_utils.py: 238 passed, 1 skipped.
• pytest tests/v1/core/test_kv_cache_planning.py tests/v1/core/test_kv_cache_utils.py -k "hisparse or host_pool": 14 passed.
• New unit tests:
  • Region mapping is rounded to a 2 MiB page size while exposing only the logical size, and the free-space check is skipped.
  • The private file-backed pool leaves no file behind and is registered at its exact size.
  • Shared-pool kwargs are covered with and without host_pool_dir.
• GPU check (H200): an 8 GiB file-backed private pool registered with cudaHostRegister in 3.6 s, and a D2H copy ran at 54 GB/s with correct data. This used tmpfs-backed files, because the dev box has no reserved huge pages and no root.
• TODO: repeat the check on a hugetlbfs mount with 1 GiB pages at 384+ GiB with NIXL enabled.

AI assistance (Claude Code) was used to write this change; it will be reviewed end-to-end before being marked ready.

🤖 Generated with Claude Code

[mergify]

Documentation preview: https://vllm--58852.org.readthedocs.build/en/58852/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork