[Perf] Narrow DeepSeek V3.2 eager CUDA graph region

[WoosukKwon]

Purpose

The portable DeepSeek-V3.2 attention path currently decorates the entire fused attention method with eager_break_during_capture. Only the sparse indexer and backend forward_mqa call require that eager boundary, so query preparation, fused norm/RoPE, and other graph-safe work are replayed as direct launches on every layer.

This change:

• inlines the graph-safe part of _fused_attention into forward;
• narrows the eager region to _sparse_indexer_and_attn;
• keeps the UV projection in the eager region so it can consume the backend-owned attention output without an extra copy;
• preserves the existing broad eager path for the ROCm subclass.

A one-request kernel trace showed 333 fewer direct launches while retaining 79 CUDA graph launches. The launches moved into graphs were 78 fused norm/RoPE kernels, 78 W_UK BMMs, 99 query/indexer GEMMs, and 78 fused-Q kernels.

Non-duplication

I searched open PRs for DeepSeek-V3.2/GLM CUDA graph, sparse-indexer, and eager_break_during_capture changes. Related PRs such as #50005 (NVIDIA DCP correctness), #47355 (indexer side-stream overlap), and #47335 (ROCm fused indexer-Q) address different paths or concerns. None narrows the eager-break boundary in the portable DeepSeek-V3.2 attention implementation.

Test Plan

• Run the focused breakable-CUDA-graph tests.
• Run all applicable pre-commit hooks on both changed files.
• Serve GLM-5.2-NVFP4 with dummy weights through the portable model override and Rust frontend using TP4/EP4, FP8 KV cache, prefix caching disabled, and the default CUDA graph mode.
• Measure TTFT with 256 sequential requests after 8 warmups, each with an 8-token prompt and one output token.
• Profile the kernel launch trace.
• Compare the no-copy implementation against an alternative that copies the backend attention output into a static buffer and captures the UV BMM.

Test Result

.venv/bin/python -m pytest tests/v1/cudagraph/test_breakable_cudagraph.py -q
14 passed, 18 warnings in 5.90s

pre-commit run --files \
  vllm/models/deepseek_v32/attention.py \
  vllm/models/deepseek_v32/amd/rocm.py
All applicable hooks passed.

GLM-5.2 TTFT, Rust frontend, dummy weights, TP4/EP4, concurrency 1, prefix caching disabled:

Variant	Mean TTFT	Median TTFT	p99 TTFT	Throughput
Current main	57.42 ms	56.99 ms	60.89 ms	17.39 req/s
This change	28.72 ms	28.65 ms	30.76 ms	34.74 req/s

All 256 requests completed successfully in both runs.

An explicit PIECEWISE-only comparison found that copying the attention output and capturing the UV BMM was another 7.43% faster than the no-copy form (26.51 ms versus 28.64 ms mean TTFT). However, that variant failed with OOM during the FULL capture phase of the default FULL_AND_PIECEWISE initialization. This PR retains the no-copy form, which successfully initializes in the default mode and does not require backend output-buffer support.

The refactor does not change the attention equations or checkpoint loading, so no model-quality evaluation was required.

AI assistance

OpenAI Codex assisted with implementation, kernel-trace analysis, benchmarking, validation orchestration, and drafting this description. The human submitter must review every changed line and be able to explain and defend the eager-boundary and backend-output trade-offs before marking this PR ready for review.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR is described.
• The test plan is provided.
• The test results and before/after comparison are provided.
• No documentation update is needed for this internal execution-path refactor.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[WoosukKwon]

/ci run

[github-actions]

✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/82887