[Rockyeast]

Overview

Inductor can emit programmatic dependent launch (griddepcontrol.wait / launch_dependents) in the Triton kernels it generates (torch._inductor.config.triton.enable_pdl), so each generated kernel starts while its predecessor drains instead of waiting for the launch gap. vLLM currently only picks this up if the user sets TORCHINDUCTOR_ENABLE_PDL=1 in the environment.

#40543 discussed enabling it: the measured gain is a few percent at small batch, the concern was host overhead for prefill-heavy, high-concurrency serving, and #40555 (gate on performance_mode == "interactivity", via the environment variable) went stale. This PR implements that gating through the Inductor options instead of the environment:

• VllmConfig.__post_init__ sets compilation_config.inductor_compile_config["triton.enable_pdl"] = True when performance_mode == "interactivity", the vLLM Inductor backend is in use (CompilationMode.VLLM_COMPILE with backend "inductor", checked after the mode is resolved from the optimization level) and the GPU is SM90+ (has_device_capability(90), the NVML check, so the frontend process does not initialize CUDA). An explicit inductor_compile_config value or a TORCHINDUCTOR_ENABLE_PDL environment variable (Inductor's own switch) wins.
• Going through inductor_compile_config keeps the option in the compile cache key ([Bugfix] Include inductor and functorch configs in compilation cache key #40627), so switching modes recompiles instead of reusing the other variant's kernels.
• balanced and throughput are unchanged.

Claims

• H200, TP=1, decode TPOT at concurrency 1: -3.1% on Qwen3-8B (BF16), -4.2% on Qwen3-8B-FP8, -3.5% on Qwen3-14B-FP8. At concurrency 32: -1.2% / -1.8% / -2.6%. At concurrency 8: -1.0% on the BF16 model but +1.4% / +0.9% on the two FP8 models (consistent over both runs), so the option is not a free win at every batch size, which is why it stays behind interactivity.
• TTFT: within run-to-run spread at concurrency 1 and 8 on all three models; at concurrency 32 unchanged on the BF16 model and +3% on Qwen3-14B-FP8 (711 / 714 -> 734 / 736 ms), consistent with the host-overhead concern in [Feature]: Dynamic PDL Enablement #40543 for the piecewise (prefill) part of a step.
• No behaviour change outside interactivity mode, and none for users who already set TORCHINDUCTOR_ENABLE_PDL or inductor_compile_config["triton.enable_pdl"].

Validation

E2E

H200 NVL, vLLM nightly 0.33.0rc1.dev421, vllm serve <model> --max-model-len 4096 --no-enable-prefix-caching (default FULL_AND_PIECEWISE CUDA graphs; Qwen3-8B with the SM90 skinny GEMM plans from #64979 applied in both arms), vllm bench serve --dataset-name random --random-input-len 1024 --random-output-len 256 --ignore-eos --seed 0; 32 prompts at concurrency 1, 8x the concurrency otherwise. PDL is toggled with TORCHINDUCTOR_ENABLE_PDL, which sets the same Inductor option this PR sets; VLLM_DISABLE_COMPILE_CACHE=1 in both arms. Two runs per arm, interleaved on two GPUs with the GPUs swapped. Median TPOT (ms), both runs:

model	conc.	PDL off	PDL on	change
Qwen3-8B	1	5.115 / 5.129	4.971 / 4.959	-3.1%
	8	6.387 / 6.396	6.342 / 6.315	-1.0%
	32	8.289 / 8.190	8.120 / 8.159	-1.2%
Qwen3-8B-FP8	1	3.791 / 3.798	3.642 / 3.625	-4.2%
	8	4.837 / 4.803	4.866 / 4.905	+1.4%
	32	6.747 / 6.750	6.629 / 6.629	-1.8%
Qwen3-14B-FP8	1	5.707 / 5.702	5.509 / 5.509	-3.5%
	8	7.052 / 7.056	7.123 / 7.108	+0.9%
	32	9.738 / 9.667	9.444 / 9.441	-2.6%

Tests

tests/compile/test_config.py: new test_interactivity_mode_enables_inductor_pdl (parametrized over the three modes and PDL support, with the default CompilationConfig so the mode is resolved from the optimization level), test_interactivity_mode_keeps_explicit_inductor_pdl, test_interactivity_mode_keeps_inductor_pdl_env, test_interactivity_mode_no_inductor_pdl_without_inductor.

Details

• Why only interactivity. [Feature]: Dynamic PDL Enablement #40543: the win is at small batch (the generated norm / activation / RoPE kernels are a few microseconds each and the launch gap is a visible fraction), while PDL adds host-side cost per launch outside CUDA graphs, i.e. in the piecewise (prefill) part of a mixed step. interactivity mode already optimizes for small-batch latency (fine-grained CUDA graph sizes), so it is the right place for this default. The H200 runs above show a small loss at concurrency 8 on the FP8 models and slightly higher TTFT at concurrency 32, so a wider default is not proposed here.
• Interaction with other PDL kernels. FlashInfer's norm kernels, the CuTe DSL skinny GEMMs and the Triton kernels from [Perf] Launch the Qwen GDN decode and Triton fused-MoE kernels with PDL #64846 already wait on their predecessor with PDL; with the Inductor kernels also launched with the attribute, the chain is no longer broken at every generated kernel (the "alternating pattern" noted in [Feature]: Dynamic PDL Enablement #40543).

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.