[Doc] Fix multimodal torch.compile troubleshooting to not use removed VLLM_TORCH_COMPILE_LEVEL

[DaoyuanLi2816]

Purpose

The "Compilation Errors" troubleshooting section in docs/design/torch_compile_multimodal.md tells users to disable compilation like this:

VLLM_TORCH_COMPILE_LEVEL=0 vllm serve <model> --compilation-config='{"compile_mm_encoder":"false"}'

VLLM_TORCH_COMPILE_LEVEL no longer exists anywhere in the codebase — torch.compile control was moved to CompilationConfig / the -O optimization levels. The env var is silently ignored, so following this step does not actually disable compilation: the model still runs compiled, defeating the "verify the model works without compilation" instruction.

Change

Use the documented way to turn off torch.compile + CUDA graphs:

vllm serve <model> --enforce-eager --compilation-config='{"compile_mm_encoder":"false"}'

Per docs/design/debug_vllm_compile.md, --enforce-eager (→ enforce_eager=True) is "Turn off torch.compile and CUDAGraphs". I kept the explicit compile_mm_encoder: false to preserve the original intent of also skipping the MM encoder.

One-line doc change.

Not a duplicate

Per AGENTS.md duplicate-work checks (on main @ 53b88d1d):

grep -rn VLLM_TORCH_COMPILE_LEVEL docs/ vllm/   # only the doc line above; no code definition
gh issue list --repo vllm-project/vllm --state all --search "VLLM_TORCH_COMPILE_LEVEL"  # nothing about this doc
gh pr list    --repo vllm-project/vllm --state all --search "VLLM_TORCH_COMPILE_LEVEL"  # none

The only open PR touching torch_compile_multimodal.md historically is #30549 (WhisperEncoder), which does not modify this troubleshooting section.

Test plan

# Confirm the env var is dead (no consumers anywhere):
grep -rn "VLLM_TORCH_COMPILE_LEVEL" . --include='*.py'   # (excluding the doc) -> no matches

pre-commit run --files docs/design/torch_compile_multimodal.md

typos, markdownlint-cli2, and the other applicable hooks pass on the changed file. (update-dockerfile-graph errors with Executable /bin/bash not found — a Windows-host limitation unrelated to this doc; it runs on Linux CI.)

---

AI-assisted (Claude Code); reviewed end-to-end by the submitter.

[mergify]

Documentation preview: https://vllm--44378.org.readthedocs.build/en/44378/

[DaoyuanLi2816]

Hi @hmellor — could you take a look when you have a moment? This is a small doc-correctness fix: the multimodal torch.compile troubleshooting section tells users to run VLLM_TORCH_COMPILE_LEVEL=0 ..., but that env var no longer exists, so the documented "disable compilation" step silently does nothing. The PR swaps it for --enforce-eager (the documented way to turn off torch.compile + CUDA graphs).

It's been open ~3 days with no reviewer auto-assigned — looks like docs/design/*.md isn't under a CODEOWNERS rule. Thanks!

[DaoyuanLi2816]

@DarkLight1337 — would you have a moment for this small doc fix? It's a 1-line swap: the multimodal torch.compile troubleshooting section tells users to run VLLM_TORCH_COMPILE_LEVEL=0 vllm serve ..., but that env var was removed, so the documented "disable compilation" step silently does nothing. The PR replaces it with --enforce-eager (the documented way).

Same shape as #44128 (which sfeng33 helped merge). CI is green except gates that need a ready label. Thanks!

[DarkLight1337]

I think we should use the direct replacement --compilation-config.mode=0. It is slightly different from --enforce-eager as it doesn't disable CUDA graphs

[DaoyuanLi2816]

Thanks @DarkLight1337, updated. Since the command already passed --compilation-config='{"compile_mm_encoder":"false"}', I folded mode into the same JSON to keep it a single argument:

vllm serve <model> --compilation-config='{"mode":0,"compile_mm_encoder":"false"}'

As you noted, mode=0 (CompilationMode.NONE) is the faithful replacement for the removed VLLM_TORCH_COMPILE_LEVEL=0 since it leaves CUDA graphs intact, unlike --enforce-eager. Happy to switch to the --compilation-config.mode=0 dotted form instead if you prefer that style.