Fixed incorrect host memory counting

[nvjullin]

Motivation

Since #40135, the HiCache host-pool budget is bounded by the cgroup headroom limit - memory.current. memory.current includes page cache charged to the cgroup: the checkpoint the server just read (size of the model) and, in containerized Slurm jobs, the unpacked container image (58 GiB in a fresh job). That cache is freely reclaimable, and host MemAvailable (psutil's available) already counts it as available, but the cgroup bound counts it as used. Right after a large model loads, the check under-reports allocatable memory by roughly the checkpoint size and rejects pools the kernel would provide by reclaiming the cache. For example, GLM-5.2-NVFP4 at tp1 x pp4 on one GB300 node with --hicache-size 135 fails on every rank:

ValueError: Not enough host memory available. Requesting 135.00 GB but only have 122.49 GB free.

The same recipe passes or fails depending on the node, because the charged cache differs per job.

Modifications

• _cgroup_memory_headroom now counts usage at each limited cgroup level as memory.current - (active_file + inactive_file), clamped at 0. On cgroup v1 it is memory.usage_in_bytes - (total_active_file + total_inactive_file). These counters cover the pages on the kernel's file-backed reclaim lists, which is what MemAvailable counts as reclaimable, so the host and cgroup bounds agree. Shared memory and tmpfs sit on the anonymous lists and stay counted as used. memory.stat's file and cache counters would wrongly count them as free.
• Added a unit test covering v2, v1 (hierarchical total_* counters rather than the local ones), a shared-memory decoy, and the clamp. It fails on the old code.

GPU-free validation in a Slurm job (cgroup v2, 350 GiB limit, no swap), comparing the old and new code side by side, in GiB:

Step	psutil available	Old cgroup headroom	New cgroup headroom
Baseline	358.1	323.7	348.7
Read 151 GiB of checkpoint shards	347.7	173.1	348.4
Add an 8 GiB file in /dev/shm	339.6	165.1	340.4

mlock stood in for pinned host memory: locking 315.4 GiB then succeeded, although the old bound allowed only 173.1 GiB. The kernel reclaimed the cache (175.3 to 22.8 GiB) and there was no out-of-memory kill. While the memory was held, the new headroom read 33.3 GiB against a predicted 32.0 GiB.

Accuracy Tests

N/A: model outputs are unaffected.

Speed Tests and Profiling

N/A: the only cost is one extra memory.stat read per limited cgroup level at host-pool init.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #36844629362
Latest PR Test (Extra): ⚠️ Not enabled -- add run-ci-extra label to opt in.
Latest PR Test (AMD ROCm 10): ❌ Run #36844629350

[nvjullin]

/tag-and-rerun-ci

[nvpohanh]

cc @xiezhq-hermann for vis

[nvpohanh]

All NV pipelines have passed. @xiezhq-hermann could you approve this? thanks!

[nvpohanh]

Or @ispobock @alphabetc1 could you review/approve this? Because is this, we can't use the latest nightly container for some runs. Thanks!

[nvpohanh]

superseded by #42420