[SuperMarioYL]

Purpose

--stage-overrides is documented as a JSON object mapping each stage_id to a dict of per-stage overrides, e.g. '{"0": {"gpu_memory_utilization": 0.8}, "2": {"enforce_eager": true}}' (see the flag help in entrypoints/cli/serve.py). Today only the JSON syntax is validated (json.loads in async_omni_engine); the decoded object's shape is never checked before it is consumed in load_stage_configs_from_model:

for stage_id_str, overrides in stage_overrides.items():
    for key, val in overrides.items():
        cli_overrides[f"stage_{stage_id_str}_{key}"] = val

This blindly assumes a dict-of-dicts, with two bad outcomes for a syntactically-valid payload:

• A top-level list/scalar, or a stage_id -> scalar mapping, raises a bare AttributeError: '<type>' object has no attribute 'items' deep in config loading — naming neither the --stage-overrides flag nor the offending value:
  • --stage-overrides '[1,2]' → AttributeError: 'list' object has no attribute 'items'
  • --stage-overrides '{"0": 5}' → AttributeError: 'int' object has no attribute 'items'
• A non-numeric stage id is silently dropped: --stage-overrides '{"abc": {"k": 1}}' produces a dead override key stage_abc_k that the downstream parser _STAGE_OVERRIDE_PATTERN = re.compile(r"^stage_(\d+)_(.+)$") (config/stage_config.py) can never match, so the user's override vanishes with no error.

This PR validates the payload shape at the single consume site (load_stage_configs_from_model, reached by both the CLI and engine paths) and raises a ValueError that names the --stage-overrides flag and the offending value. Behavior is unchanged for well-formed input.

Resolves the input-validation gap noted while triaging stage-override handling.

Test Plan

Added TestValidateStageOverrides in tests/entrypoints/test_utils.py (CPU-only, no model load — the validator runs before any config/model access):

python -m pytest tests/entrypoints/test_utils.py -q -k ValidateStageOverrides

Covers: None/empty no-op; well-formed string-keyed and int-keyed payloads pass; top-level list/scalar/string rejected; per-stage scalar/list/string value rejected; non-integer stage ids ("abc", "stage0", "0.5", "-1", True) rejected; and an end-to-end assertion that load_stage_configs_from_model(model, stage_overrides=[1, 2]) now fails fast with a clear ValueError instead of the previous cryptic AttributeError.

Test Result

All new cases pass; the end-to-end case is red on main (raises AttributeError) and green here (raises ValueError naming --stage-overrides). ruff check and ruff format are clean on both changed files.

[SuperMarioYL]

Gentle ping on this one 🙂 — it's a small, self-contained bugfix that validates the --stage-overrides JSON shape up front and fails with a clear, flag-named error instead of a deep stack trace later. CI is green and the PR is mergeable. Happy to rebase or tweak the error-message wording if you'd prefer a different shape — would appreciate a look whenever you have a moment.

[vllm-omni-review-bot]

Omni ReviewBot: no human activity for 53 days

@SuperMarioYL this pull request has had no human commit, comment or review since 2026-07-17. Per repository policy it may be closed if it stays inactive.

To keep it moving, any one of these is enough: push an update, reply to the open blocker, or post the current plan and timeline.

[SuperMarioYL]

Closing this one — it's superseded by #4281, which landed parse_stage_overrides() with the same --stage-overrides JSON validation (plus stricter stage-id checks). Thanks for building the fuller version; happy to contribute again.