[CI] Stabilize GLM-5.2 PCP evaluation

[khluu]

Summary

• enable PyTorch expandable CUDA allocator segments for the GLM-5.2 TP1/PCP4 GSM8K evaluation;
• keep --max-num-batched-tokens 32768, which is needed to exercise the large-batch PCP path;
• scope the allocator change to the memory-tight TP1/PCP4 case; TP2/PCP2 is unchanged.

Root cause

The pytest-level error in Buildkite #81913 is only a 20-minute server-start timeout. The worker log shows the actual failure during the profile run:

CUDA out of memory. Tried to allocate 24.43 GiB.
15.36 GiB is free
23.97 GiB is reserved by PyTorch but unallocated

The allocation comes from FlashInfer CUTLASS MoE's FusedMoeRunner::getWorkspaceInfo. The same signature is present in Buildkite #80082, before the recent Kimi/K3 changes, so this is not a recent model-runner regression or a bad B200 node.

The TP1/PCP4 configuration became memory-tight when --max-num-batched-tokens 32768 was added in #49294. Successful runs such as Buildkite #80689 show that the workload's real peak fits: 159.66 GiB total non-KV memory on a 178.35 GiB B200. The intermittent OOM occurs because a roughly 24 GiB cached allocator segment cannot satisfy the next slightly larger 24.43 GiB FlashInfer workspace request. Expandable segments let that reserved memory grow/reuse instead of requiring a second contiguous allocation.

Duplicate-work check

No open PR was found for the GLM-5.2 PCP FlashInfer MoE startup OOM using searches for GLM-5.2 PCP OOM, lm-eval-pcp-4xb200, and expandable_segments GLM.

• [Bugfix][Attention] Size FlashInfer sparse MLA workspace for decode-context-parallel #50791 sizes a different FlashInfer sparse-MLA attention workspace for DCP.
• Revert "[Core] Add MRV2 virtual-batch PCP for MLA" (#46570) #49196 is an automated draft revert for an older PCP accuracy regression, not this profile-run MoE OOM.
• [Bug]: GLM-5 FP8 on H200 CUDA OOM in sparse_attn_indexer at High Concurrency #34553 concerns a GLM-5 sparse-attention indexer OOM under serving load, not the startup MoE workspace failure here.

Testing

• git diff --check
• parsed the YAML and verified both the allocator setting and retained 32K batch limit with Ruby's YAML parser
• repository pre-commit hooks run by git commit: passed
• GitHub pre-commit workflow: passed
• Buildkite #82253 — LM Eval PCP (4xB200): passed on dgxB200-14 with exit status 0
  • unchanged TP2/PCP2 control: GSM8K passed
  • TP1/PCP4 with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True: profile run passed with 130.49 GiB of weights and a 28.33–30.23 GiB torch peak, the server started, and all 1,319 GSM8K examples passed

The allocator-only change does not alter model inputs, kernels, or numerics. The B200 model evaluation passed on the first validation iteration.

AI assistance

AI assistance was used for Buildkite log analysis, root-cause isolation, implementation, and drafting this description.

• The human submitter has reviewed the changed line and CI evidence before marking this PR ready.

[khluu]

/runci

[khluu]

/ci run

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[khluu]

/ci run

[github-actions]

✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/82253

[LucasWilkinson]

LGTM; thank you!