Skip to content

[Bugfix][Frontend] Parse VLLM_USE_MODELSCOPE via vllm.envs - #5428

Merged
Gaohan123 merged 2 commits into
vllm-project:mainfrom
yashkgp:fix/modelscope-env-truthiness
Jul 27, 2026
Merged

Gaohan123 merged 2 commits into
vllm-project:mainfrom
yashkgp:fix/modelscope-env-truthiness

Conversation

@yashkgp

@yashkgp yashkgp commented Jul 26, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Fixes #5427.

omni_snapshot_download() checked VLLM_USE_MODELSCOPE for truthiness instead of
parsing it, so every non-empty value enabled ModelScope, including 0, False,
false, no and off.

vLLM only treats the literal string "true" (case-insensitive) as enabled, see
vllm/envs.py @ v0.25.0:

"VLLM_USE_MODELSCOPE": lambda: (
    os.environ.get("VLLM_USE_MODELSCOPE", "False").lower() == "true"
),

omni_snapshot_download() is the first call in OmniBase.__init__, so with
VLLM_USE_MODELSCOPE=0 both offline Omni(...) and vllm-omni serve took the
ModelScope branch. modelscope is not a vLLM-Omni dependency (it is only
installed in docker/Dockerfile.xpu), so a default install failed at startup with
ModuleNotFoundError: No module named 'modelscope'. Where it is installed,
weights were fetched from ModelScope despite the opt-out.

vllm_omni/model_executor/model_loader/weight_utils.py#L9 already reads the flag
through vllm.envs, so the two paths disagreed inside one process. This routes
the call site through vllm.envs as well. VLLM_USE_MODELSCOPE=true behaves as
before.

Test Plan

Added tests/entrypoints/test_omni_snapshot_download.py, marked core_model and
cpu so the existing L1 Simple · Other Test lane in
.buildkite/cuda/test-ready.yml collects it, no CI wiring changes needed. It
stubs the ModelScope module into sys.modules, otherwise the wrong branch raises
ModuleNotFoundError instead of being observable, then asserts which backend
omni_snapshot_download() selects for each flag value.

pytest -sv tests/entrypoints/test_omni_snapshot_download.py -m 'core_model and cpu'

Repro for the original bug:

VLLM_USE_MODELSCOPE=0 python -c "
from vllm_omni.entrypoints.omni_base import omni_snapshot_download
omni_snapshot_download('Qwen/Qwen3-Omni-30B-A3B-Instruct')
"

vLLM Version: v0.25.0

vLLM-Omni Commit: 2bb6189

Test Result

On main, 6 of the 10 cases fail:

FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[0]
FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[1]
FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[False]
FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[false]
FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[no]
FAILED tests/entrypoints/test_omni_snapshot_download.py::test_non_true_values_do_not_enable_modelscope[off]
6 failed, 4 passed in 0.02s

E         - huggingface
E         + modelscope

With the fix:

10 passed in 0.01s

ruff check, ruff format --check and tools/pre_commit/check_test_marks.py are
clean on both changed files.

I do not have a CUDA box, so that pytest run was done in a CPU sandbox that pairs
the real vllm/envs.py from v0.25.0 with the source of omni_snapshot_download()
taken from this branch, and from main for the "before" numbers. The test file is
the same one in this PR. The full tests/entrypoints/ suite needs CI.

omni_snapshot_download() read the raw environment variable, so every
non-empty value was truthy. Setting VLLM_USE_MODELSCOPE=0 (or False/no/off)
to explicitly opt out therefore selected the ModelScope download path, which
raises ModuleNotFoundError on a default install because modelscope is not a
vLLM-Omni dependency.

vLLM enables the flag only for the literal string "true" (case-insensitive),
and vllm_omni/model_executor/model_loader/weight_utils.py already reads it
through vllm.envs. Route this call site through vllm.envs too so both agree.

Signed-off-by: Yash Jain <yashjainjain1704@gmail.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@Gaohan123 Gaohan123 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks

@Gaohan123 Gaohan123 added the ready label to trigger buildkite CI label Jul 27, 2026
@Gaohan123
Gaohan123 enabled auto-merge (squash) July 27, 2026 04:00
@Gaohan123
Gaohan123 merged commit 911ea0c into vllm-project:main Jul 27, 2026
6 of 9 checks passed
nagisa-kunhah pushed a commit to nagisa-kunhah/vllm-omni that referenced this pull request Jul 30, 2026
khairulkabir1661 pushed a commit to khairulkabir1661/vllm-omni that referenced this pull request Sep 25, 2026
…ect#5428)

Signed-off-by: Yash Jain <yashjainjain1704@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: VLLM_USE_MODELSCOPE=0 still routes model download through ModelScope

2 participants