[Model] Add PW CUDA graph support for Inkling [2/N]

[WoosukKwon]

Summary

This is the second independently mergeable slice carved out of #48768. It adds breakable (piecewise) CUDA graph support for Inkling on Model Runner V2:

• automatically enables breakable CUDA graphs for InklingForCausalLM and InklingForConditionalGeneration
• marks the Inkling attention boundary as eager and makes short-convolution metadata safe for uniform batched execution
• preserves padded request counts and static attention metadata during piecewise capture/replay
• keeps warmup and capture metadata independent
• adds MRV2 CudaGraphManager coverage for fresh capture metadata

Inkling is assumed to always run with Model Runner V2. The legacy vllm/v1/worker/gpu_model_runner.py and vllm/v1/cudagraph_dispatcher.py paths are unchanged.

This PR intentionally contains no LoRA support and no Inkling MTP support. The small autoregressive speculator CUDA-graph utility adjustment is generic MRV2 metadata plumbing required to keep that existing runner compatible with the shared breakable graph path; this PR does not enable or test Inkling speculative decoding.

Duplicate-work check

I searched the open PRs for breakable Inkling, piecewise Inkling CUDA graph, and PW CUDA graph Inkling. The only match was #48768 because its description records this carve-out. Before publishing this PR, the corresponding breakable-CUDA-graph files and config hunk were removed from #48768, so the diffs do not duplicate each other.

Testing

• pre-commit run
  • all hooks passed
• .venv/bin/python -m pytest tests/models/inkling/test_contract_validation.py -q
  • 4 passed
• .venv/bin/python -m pytest tests/v1/cudagraph/test_breakable_cudagraph.py::test_piecewise_capture_builds_fresh_metadata_for_both_passes -q
  • 1 passed (run with the existing precompiled extension and this worktree's Python source overlaid)

Model evaluation

Served thinkingmachines/Inkling-NVFP4 on four GB200 GPUs with TP4, Model Runner V2, and without MTP/speculative decoding:

export VLLM_USE_V2_MODEL_RUNNER=1
export FLASH_ATTENTION_CUTE_DSL_CACHE_ENABLED=1
export CUDA_VISIBLE_DEVICES=0,1,2,3
export HF_HOME=/tmp/hf-woosuk

.venv/bin/vllm serve thinkingmachines/Inkling-NVFP4 \
  --tokenizer-mode inkling \
  --reasoning-parser inkling \
  --tool-call-parser inkling \
  --enable-auto-tool-choice \
  --tensor-parallel-size 4 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.8 \
  --kernel-config.enable_flashinfer_autotune=False \
  --trust-remote-code

Both PIECEWISE (51/51) and FULL (51/51) CUDA graph capture completed successfully. Four concurrent GSM8K prompts were then sent through /v1/chat/completions with max_tokens=2048 and temperature=0. Result: 4/4 correct, with all requests finishing normally.

The lower memory utilization and 8192-token test length leave KV-cache headroom for the approximately 551 GiB checkpoint; the default 0.9 utilization did not leave enough free memory for KV allocation on this host.

AI assistance

AI assistance was used to help carve out the existing implementation, add tests, run validation, and prepare this PR. The human submitter will review every changed line and is responsible for understanding and defending the change end-to-end.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}