fix(cudagraph): synchronize auxiliary warmup streams

[voipmonitor]

Resulting behavior

CUDA graph setup synchronizes the accelerator after each uncaptured warmup forward and before torch.cuda.graph begins. The compilation monitor exposes whether graph preparation is active so model code can disable auxiliary overlap during uncaptured warmups while retaining overlap during serving and regular FULL capture.

Technical reason

A model forward may fork kernels onto auxiliary streams and enqueue only event waits on the compute stream. Stream ordering is sufficient for normal execution, but CUDA graph capture must not begin while an uncaptured auxiliary kernel still owns temporary allocator storage. Starting capture at that boundary can race allocator reuse and surface as an illegal memory access in a later kernel.

Compatibility

The additional synchronization runs only during graph construction. It does not add synchronization to graph replay or serving. Models that do not use auxiliary streams retain the same captured graph and runtime execution.

Validation

• The focused regression verifies warmup forward -> accelerator synchronization -> capture begin -> capture forward ordering.
• Repository hooks passed Ruff, formatting, MyPy, SPDX, import, configuration-default, and CUDA API checks.
• A TP4 GLM-5.3 composition using the monitor state for both KDA-gate and L2-prefetch streams completed three consecutive FULL CUDA graph startups for no-speculative, MTP3, and DFlash2 serving on NVIDIA RTX PRO 6000 Blackwell GPUs.

OpenAI Codex assisted with the failure isolation, implementation, and validation. The submitter reviewed the resulting source and evidence.

Summary by CodeRabbit

• Bug Fixes

  • Improved CUDA graph capture reliability by ensuring warmup operations on auxiliary streams finish before capture begins.
  • Added safeguards to prevent incomplete warmup work from affecting captured execution.

• Tests

  • Added coverage verifying the expected order of warmup execution, stream synchronization, and CUDA graph capture.

[coderabbitai]

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 1bb06b2b-4c7f-4c68-a0bc-cf6ff0f92506

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 2f8cf27b-ddbd-418a-8f85-9288e9d294d1

📥 Commits

Reviewing files that changed from the base of the PR and between 83cb22a and 7cf2edd.

📒 Files selected for processing (3)

• tests/v1/cudagraph/test_cudagraph_manager.py
• vllm/compilation/monitor.py
• vllm/v1/worker/gpu/cudagraph_utils.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds a CUDA graph capture state helper and synchronizes the accelerator after warmup execution. A new test verifies that synchronization occurs before graph capture.

Changes

CUDA graph capture flow

Layer / File(s)	Summary
CUDA graph capture state API vllm/compilation/monitor.py	Adds is_cudagraph_capturing_enabled() to return the current CUDA graph-capturing state.
Warmup stream synchronization vllm/v1/worker/gpu/cudagraph_utils.py, tests/v1/cudagraph/test_cudagraph_manager.py	Synchronizes accelerator streams after the warmup forward pass. The test verifies the order of warmup execution, synchronization, capture start, and capture forward execution.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 7cf2e

This change synchronizes uncaptured warmup work before CUDA graph capture, preventing auxiliary-stream work from crossing the capture boundary. The covered ordering change introduces no remaining merge-blocking risk.

Sequence Diagram(s)

sequenceDiagram
  participant CudaGraphManager
  participant forward_fn
  participant torch.accelerator
  participant graph_capture
  CudaGraphManager->>forward_fn: Run warmup forward
  forward_fn-->>CudaGraphManager: Warmup completes
  CudaGraphManager->>torch.accelerator: Synchronize streams
  torch.accelerator-->>CudaGraphManager: Synchronization completes
  CudaGraphManager->>graph_capture: Begin capture
  graph_capture->>forward_fn: Run capture forward

Loading

Suggested reviewers: lukealonso

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: synchronizing auxiliary warmup streams before CUDA graph capture.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[voipmonitor]

Independent integration qualification

Status: qualified at the PR head embedded in
voipmonitor/vllm:jovian-judgement-community-20260903-r20.

The focused regression verified the required graph-setup order: uncaptured
warmup forward, accelerator synchronization, graph-capture start, then captured
forward. A TP4 GLM-5.3 composition using auxiliary KDA-gate and L2-prefetch
streams completed three consecutive graph startups in no-speculation, MTP3,
and DFlash2 modes.

The published launcher defaults to CUDAGRAPH_MODE=FULL_AND_PIECEWISE. On four
stock-clock RTX PRO 6000 Blackwell Workstation Edition GPUs, R20 completed the
DCP1 and DCP4 C1, C8, and 32K-prefill matrix in all three serving modes. Each
DCP4 C8 run was followed by a C1 request, validating the transition between
captured request shapes. No auxiliary-stream graph-boundary failure was
reproduced.

The synchronization is confined to graph construction; serving and graph
replay do not gain a synchronization point.