Make microbatch optimization (DBO) work with general models

[0xjunhao]

Purpose

Currently, microbatch optimization only works for text models, and the model must expose an internal model attribute (e.g.

vllm/vllm/model_executor/models/deepseek_v2.py

Lines 1339 to 1341 in 5bf3c42

	self.model = self.model_cls(
	vllm_config=vllm_config, prefix=maybe_prefix(prefix, "model")
	)

)

Otherwise, vLLM fails to start. This PR extends microbatch optimization to general models, removing the requirement for text-only models and the need for an internal model attribute.

Test Plan

vllm serve Qwen/Qwen3.5-35B-A3B --trust-remote-code --gpu-memory-utilization 0.8 -dp 2 --enable-expert-parallel --all2all-backend deepep_low_latency --enable-dbo

Test Result

Before the fix:

For multimodal models, it gives the following error:
#34210

For models without an internal model attribute, it gives the following error:

(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_ubatch_wrapper.py", line 494, in __call__
(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880]     return self._run_ubatches(ubatch_metadata, self.model)
(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880]                                                ^^^^^^^^^^
(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_ubatch_wrapper.py", line 164, in __getattr__
(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880]     raise AttributeError(
(Worker pid=5836) (Worker_DP1_EP1 pid=5836) ERROR 03-23 20:25:23 [multiproc_executor.py:880] AttributeError: Attribute model not exists in the runnable of cudagraph wrapper: Qwen3_5MoeForConditionalGeneration(

After the fix

The server starts and runs successfully.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request successfully extends microbatch optimization to general models by removing the dependency on a model attribute and handling optional input_ids and inputs_embeds. The changes are logical and well-aligned with the goal. I have one suggestion to improve consistency and prevent a potential runtime error, which is detailed in my comment.

[LucasWilkinson]

Thanks for the contribution, LGTM!

[mergify]

Hi @0xjunhao, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[0xjunhao]

Seems this is the only error and it's unrelated to this PR.

ERROR   -  mkdocstrings: Couldn't load inventory https://psutil.readthedocs.io/en/stable/objects.inv through handler 'python': HTTP Error 404: Not Found

[SageMoore]

Looks reasonable. Thanks for the contribution!