[HiCache] Fix cgroup page-cache accounting and the sizing fallback when cgroup discovery fails

[metamergebot]

Motivation

Two problems in how HiCache sizes its host pools against the process cgroup.

1. Charged page cache counted as used. Since [HiCache] Auto-size the host pool to fit available host memory #40135 the host-pool budget is bounded by the cgroup headroom limit - memory.current (memory.usage_in_bytes on v1). Charged usage includes the page cache the process populated, most visibly the checkpoint it just read. That cache is reclaimable, and host MemAvailable (psutil's available) already treats it as free, but the cgroup bound treated it as used. Right after a large model loads, an explicitly sized host pool could therefore fail with Not enough host memory available although the kernel would have served it by reclaiming the cache, and the same command passed or failed depending on how much cache the job happened to have charged.

2. Auto-sizing against host RAM when the container limit is unreadable. [HiCache] Fall back to host memory when no cgroup fs is mounted #42120 made a missing cgroup filesystem size the pool against host available memory with a warning, and added SGLANG_HICACHE_HOST_MEMORY_BYTES as an override. For the default auto-sized pool that is the wrong failure mode: when the limit cannot be read, assuming the host's RAM is the container's allowance sizes a pool the limit may not hold, and the OOM kill arrives later and far from the cause. An explicit --hicache-size or --hicache-ratio is different: the user has stated what they want, so the probe is only a sanity check and a warned fallback is appropriate.

The page-cache accounting is the same fix as #42039; this PR carries it together with the sizing-policy change because both edit the same function and tests, and the second part depends on the memory.stat read the first one introduces.

Modifications

• mem_cache/host_memory.py
  • At each limited cgroup level, charged usage is now usage - (active_file + inactive_file) (total_active_file + total_inactive_file on v1, whose unprefixed counters omit descendants), clamped at zero. These are the file-LRU pages MemAvailable counts as reclaimable; shared memory sits on the anonymous LRU and stays charged, which is why memory.stat's file / cache totals are not used. A missing memory.stat for a known limit is an error like a missing usage file.
  • available_host_memory_bytes(*, allow_cgroup_fallback=False): when cgroup discovery fails (no cgroupfs mounted, a mounted hierarchy that does not contain the process, or an unreadable counter for a known limit), it raises RuntimeError asking for an explicit --hicache-size. With allow_cgroup_fallback=True it logs a warning and checks against host available memory instead. Readable limits are enforced in both modes.
  • The one-time "no cgroup filesystem" warning and the SGLANG_HICACHE_HOST_MEMORY_BYTES override are removed.
• mem_cache/pool_host/base.py: host_memory_budget_bytes(requested_bytes=0, *, auto_size=False); the explicit-size checks in the host pools keep the default and so get the warned fallback.
• mem_cache/hicache_auto_size.py: the auto-sizing path passes auto_size=True, so it requires successful cgroup discovery.
• environ.py: SGLANG_HICACHE_HOST_MEMORY_BYTES is unregistered. It was added in [HiCache] Fall back to host memory when no cgroup fs is mounted #42120, which is not in any release tag, so it is removed outright rather than deprecated; --hicache-size is the supported way to size the pool without cgroup discovery.
• test/registered/unit/mem_cache/test_host_memory.py: the synthetic procfs now writes memory.stat; a regression test covers the page-cache accounting on v2 and v1 (hierarchical counters, a shared-memory decoy, and the clamp when v1 usage trails the counters); the no-cgroupfs, unmatched-hierarchy and missing-counter cases assert the actionable error by default and the host-memory fallback when allowed. The tests for the removed override are dropped.

Behavior is unchanged when cgroup discovery succeeds and no page cache is charged, and when the process has no cgroup membership.

Accuracy Tests

N/A: model outputs are unaffected.

Speed Tests and Profiling

N/A: the only cost is one extra memory.stat read per limited cgroup level at host-pool init.

Validation: test/registered/unit/mem_cache/test_host_memory.py, test_hicache_auto_size.py and test_mem_pool_host.py::TestHostMemoryBudget pass locally; the new page-cache test fails on the previous accounting.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): 🚫 Run #37147602776
Latest PR Test (Extra): ⚠️ Not enabled -- add run-ci-extra label to opt in.
Latest PR Test (AMD ROCm 10): ❌ Run #37147602737

[metamergebot]

This port clears our bar on da2084359527c9851bbcb6720fc0ece1eb056759: every base-a and base-b job is green.
base-c and the accelerator suites are reported but not gated, because they run
on scarce hardware and upstream merges through them routinely.

gate 20✓✗2✂4 lint✓ build✓ A✓ B✓ C✗2⏳8 other 6✓✂6 · accel ✗amd,b200,npu,xpu

@merrymercy @xiezhq-hermann — could one of you take a look and merge it if you are happy? Nothing else is waiting on our side.