[Fix] Misc Fixes in ViT CUDA Graph

[b-mu]

Purpose

1. Previously max_batch_size = max_budget // min_budget could exceed min_budget, causing prepare_encoder_cudagraph_capture_inputs to compute per_image_output = token_budget // max_batch_size = 0 for small budgets, leading to a reshape crash on empty tensors in Qwen3_VisionPatchEmbed.forward. Fixed by capping to min(max_budget // min_budget, min_budget) if both budgets and max batch size are auto-inferred. For the paths where either budget or max batch size are provided by the user, we adjust the other (i.e. the one that is auto-inferred) to satisfy the invariant: max_batch_size <= min_budget.
2. Use ceiling so that when token_budget is not divisible by max_batch_size, input buffer is not under-allocated.
3. Disable CUDA graph when pruning is enabled and for images in Qwen3-VL.

Test Plan

4×GB200 NVLink (TP=4, ViT DP=4), Qwen3-VL-32B-Instruct, random-mm dataset (synthetic)

• Eager (baseline):

  vllm bench mm-processor \
    --model Qwen/Qwen3-VL-32B-Instruct \
    --dataset-name random-mm \
    --random-mm-base-items-per-request 20 \
    --random-mm-num-mm-items-range-ratio 0.5 \
    --random-mm-bucket-config '{"(224,224,1)": 0.2, "(336,336,1)": 0.3, "(448,448,1)": 0.2, "(672,672,1)": 0.2,
  "(1008,1008,1)": 0.1}' \
    --num-prompts 1000 --num-warmups 200 \
    --max-model-len 16384 --dtype bfloat16 --seed 42 \
    --mm-encoder-attn-backend FLASH_ATTN \
    --tensor-parallel-size 4 --mm-encoder-tp-mode data

• ViT full CUDA Graph:

  vllm bench mm-processor \
    --model Qwen/Qwen3-VL-32B-Instruct \
    --dataset-name random-mm \
    --random-mm-base-items-per-request 20 \
    --random-mm-num-mm-items-range-ratio 0.5 \
    --random-mm-bucket-config '{"(224,224,1)": 0.2, "(336,336,1)": 0.3, "(448,448,1)": 0.2, "(672,672,1)": 0.2,
  "(1008,1008,1)": 0.1}' \
    --num-prompts 1000 --num-warmups 200 \
    --max-model-len 16384 --dtype bfloat16 --seed 42 \
    --mm-encoder-attn-backend FLASH_ATTN \
    --tensor-parallel-size 4 --mm-encoder-tp-mode data \
    --compilation-config '{"cudagraph_mm_encoder": true, "encoder_cudagraph_max_frames_per_batch": 1}'

Test Result

Encoder Forward Latency (mean):

Config	FA4 BF16	FLASHINFER FP8
Eager (baseline)	38.4 ms	39.5 ms
ViT full CUDA Graph	29.7 ms (+22.7%)	31.5 ms (+20.2%)

Encoder Forward Latency (median):

Config	FA4 BF16	FLASHINFER FP8
Eager (baseline)	25.4 ms	29.8 ms
ViT full CUDA Graph	23.9 ms (+5.8%)	25.3 ms (+15.1%)

Encoder Forward Latency (P99):

Config	FA4 BF16	FLASHINFER FP8
Eager (baseline)	194.0 ms	244.4ms
ViT full CUDA Graph	155.6 ms (+19.8%)	140.3ms (+42.6%)

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request refactors the initialization logic for EncoderCudaGraphManager to enforce an invariant that max_batch_size must be less than or equal to the smallest token budget. This prevents potential reshape crashes during CUDA graph capture. The changes include adding validation for user-specified configurations and ensuring model-provided budget ranges are valid (positive and min <= max). New tests have been added to cover various configuration paths and validate these invariants. However, the review highlights a critical issue: user-provided encoder_cudagraph_token_budgets are not validated for positivity, which could lead to a ZeroDivisionError if a non-positive budget is supplied.

[gemini-code-assist]

User-provided encoder_cudagraph_token_budgets are not validated for positivity. If a user provides a non-positive budget (e.g., [0, 128]), min(self.token_budgets) could be zero or negative. This can lead to self.max_batch_size being set to zero, which will cause a ZeroDivisionError later during CUDA graph capture preparation.

You should add validation to ensure all user-provided budgets are positive. A similar check is needed in the elif user_budgets: block.

            self.token_budgets = sorted(user_budgets)
            if self.token_budgets[0] <= 0:
                raise ValueError(
                    f"Invalid encoder_cudagraph_token_budgets: {user_budgets}. "
                    "All budget values must be positive."
                )
            self.max_batch_size = user_max_images
            min_tok = self.token_budgets[0]

[wangshangsam]

This is a fair point, but I feel that the better way to handle this is through pydantic (cuz ultimately this is an input validation problem), e.g., in the definition of CompilationConfig:

from pydantic.types import PositiveInt

@config
class CompilationConfig:
    ...
    encoder_cudagraph_token_budgets: list[PositiveInt]

[b-mu]

I have added the check in vllm/config/compilation.py.

[gemini-code-assist]

Similar to the other block, user-provided encoder_cudagraph_token_budgets are not validated for positivity here. This can lead to a ZeroDivisionError if a non-positive budget is provided.

                self.token_budgets = sorted(user_budgets)
                if self.token_budgets[0] <= 0:
                    raise ValueError(
                        f"Invalid encoder_cudagraph_token_budgets: {user_budgets}. "
                        "All budget values must be positive."
                    )
                self.max_batch_size = min(
                    max_budget // min_budget,
                    self.token_budgets[0],
                )

[b-mu]

This has been addressed by checking in vllm/config/compilation.py.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @b-mu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[b-mu]

@Isotr0py I have rebased this branch, and added two other minor fixes for bugs found when I was reviewing #40830. Could you take another look?

[Isotr0py]

LGTM, requested force merge.