Revert "[Bugfix][Spec Decode] Capture the widest uniform decode batch by default" (#50488)

[vllm-agent]

Auto-generated draft revert of #50488 ("[Bugfix][Spec Decode] Capture the widest uniform decode batch by default"), which is the HEAD commit of nightly build #86121 and broke 2 H200 spec-decode jobs.

A narrower fix is preferable if the author has one — e.g. clamping the new default to the previous 512 ceiling (or to available memory) instead of taking max_num_seqs * (1 + num_spec_tokens) unconditionally. Please close this in favour of such a fix-forward.

Root cause

The PR makes max_cudagraph_capture_size default to the widest uniform decode batch. Across every engine in the two failing jobs the value moved from a flat 512 to values driven by max_num_seqs * (1 + num_spec_tokens):

build	commit	max_cudagraph_capture_size	result
86035 / 86108 (baseline)	21fa2c5a / 6d4562c5	512 for every engine	passed
86121	d3d79ff	768, 1024, 2176, 6400, 6400	failed

:nvidia: (H200) Spec Decode N-Gram + Suffix — the two suffix configs (num_spec_tokens=24, max_num_seqs=256 → 6400) die during CUDA graph capture; the two configs that stayed at 768/1024 passed:

File "vllm/v1/attention/backends/flash_attn.py", line 1162, in forward
    flash_attn_varlen_func(
  out, softmax_lse, _, _ = torch.ops._vllm_fa3_C.fwd(
RuntimeError: torch_call_dispatcher( "aten::new_empty", ... ) API call failed
  at torch/csrc/stable/ops.h, line 939

:nvidia: (H200) Spec Decode AL DFlash Nightly — 2176 instead of 512 grows the CUDAGraph pool and peak activation, collapsing the KV cache budget:

	baseline (86035)	86121
CUDAGraph memory	0.8 / 1.24 GiB	1.76 / 4.92 GiB
peak activation	2.04 / 3.36 GiB	3.09 / 7.88 GiB
Available KV cache	6.78 / 5.46 GiB	5.74 / 0.94 GiB

ValueError: To serve at least one request with the model's max seq len (32768),
5.12 GiB KV cache is needed, which is larger than the available KV cache memory (0.94 GiB).

This is a default-behaviour change, so it is not CI-only: any spec-decode deployment on a memory-constrained GPU now either fails engine init or silently loses most of its KV cache.

Bisect

• Baselines 86035 (21fa2c5a) and 86108 (6d4562c5) both green, both reporting 512.
• 86121 runs exactly d3d79ff (this PR's squash commit) and is the first red.
• Failure correlates test-by-test with the enlarged capture size; the MI300 mirrors of both jobs pass, consistent with a memory-footprint regression rather than a functional break.

Dependents

d3d79ff is still the tip commit on both vllm/config/vllm.py and vllm/v1/worker/gpu/cudagraph_utils.py, so nothing is stacked on it. The revert applied cleanly with no conflicts and touches only the 6 files from the original PR.

---

Auto-generated by CI failure analyzer.

[khluu]

Narrow fix-forward is now draft PR #54418. It preserves #50488's dynamic-tier and safe off-stride coverage while keeping generated capture sizes under the platform default memory ceiling. Exact H200 validation: https://buildkite.com/vllm/ci/builds/86219

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @vllm-agent.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork