[HiCache] Read host memory once per TP group before splitting the budget

[alilxxey]

Motivation

Fixes #38156.

host_memory_budget_bytes() (#35540) splits psutil.virtual_memory().available - 10 GiB by ranks_per_host() so that co-located ranks do not oversubscribe the host. The split is only right for a reading taken before any co-located rank allocates. Each rank reads available when it reaches the guard, and TP ranks reach it seconds apart: pinning a large host pool is slow and nothing synchronizes the ranks between pools. A rank that arrives after k peers have pinned their pools sees available reduced by k × pool while still dividing by the full rank count, so the peers' pools are charged twice and the guard raises ValueError: Not enough host memory available on a host with room for all of them. With a sidecar pool (DSA indexer, MiniMax index-K, mamba) this repeats on the second guard after the first pool is pinned everywhere. On a tp8 host with 1772 GiB and --hicache-size 176 (8 × 164 GiB KV + 8 × 33 GiB indexer, 86 GiB to spare) the start is intermittent; see the issue for the arithmetic.

Modifications

• host_memory_sync_group(): the TP cpu_group of a multi-rank run, None otherwise. TP ranks of one pipeline stage build the same host pools in the same order, so a collective issued once per pool is matched on every rank; the world group gives no such guarantee (pipeline stages own different layers and build different pools), which is why [HiCache] Split the host-memory budget across co-located ranks #35540 avoided a collective there.
• host_memory_budget_bytes(): reduce the reading to the group-wide MIN over that group before subtracting the reserve and splitting. The collective is a barrier, so no rank allocates before all of them have read, and the minimum is the latest reading, which already reflects every pool the group built earlier. Same pattern as sync_fixed_hicache_size() in the same constructor (PP-group MIN) and get_available_gpu_memory(distributed=True). Single-rank and non-distributed behavior is unchanged; the six guards that call the helper are untouched.
• Unit tests (test_mem_pool_host.py, TestHostMemoryBudget): the budget is sized from the group-wide minimum with ReduceOp.MIN on an int64 tensor over the sync group; no collective and the local reading without a group; the sync group is the TP cpu_group for multi-rank runs and None for single-rank runs, before distributed init, and before the TP group is built. Also exercised end to end on a real 3-process gloo group with divergent per-rank readings over two pool rounds: every rank gets the same minimum-derived budget, and a single-process run with sglang's real _TP at world size 1 never issues the collective.

Known limits, unchanged by this PR: with several pipeline stages on one host the stages still race each other (the window shrinks to the stage); with a TP group spanning hosts the minimum may come from a peer host, which is conservative; ranks_per_host() still counts only the ranks of this process group, so data-parallel replicas launched as separate groups are not accounted for.

Accuracy Tests

Not applicable: this changes only the host-pool sizing guards, not model forward paths.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ❌ Run #33991104005
Latest PR Test (Extra): ❌ Run #33991103784
Latest PR Test (AMD ROCm 7.2): ❌ Run #33991103904