[apex-mochen]

vLLM Version: 0.30.0 · vLLM-Omni commit: 5ce6f5dc49ead82b24bb5e1c19849cea67c51d3e (this branch is one commit on top of it)
Test Plan / Result: the exact commands and their pass counts are tabulated under "Tests" below; the base-vs-head negative control is under "Negative control".

---

Problem

vllm-omni inherits vLLM's serve args through make_arg_parser, so
--fingerprint-mode and --fingerprint-value are accepted (and documented in
--help), but omni never applies them: --fingerprint-mode none still returns
a full fingerprint, and --fingerprint-value is inert.

Upstream applies the mode once, before any serving class is constructed, at
the top of init_generate_state:

# vllm/entrypoints/generate/api_router.py:89-96
from vllm.entrypoints.serve.utils.fingerprint import set_default_fingerprint_mode

set_default_fingerprint_mode(
    getattr(args, "fingerprint_mode", "full"),
    getattr(args, "fingerprint_value", None),
)

because the mode lives in module globals and each serving class reads it exactly
once, at construction time:

# vllm/entrypoints/generate/base/serving.py:161-172
from vllm.entrypoints.serve.utils.fingerprint import get_system_fingerprint
...
    self.system_fingerprint: str | None = get_system_fingerprint(
        engine_client.vllm_config, ...
    )
except Exception:
    # Never fail server startup over the fingerprint.
    self.system_fingerprint = None

omni_init_app_state builds the whole generate-side state itself
(vllm_omni/entrypoints/openai/api_server.py:749) and never calls
init_generate_state, in any of its three construction paths:

• multi-stage — OmniOpenAIServingChat / OpenAIServingCompletion / … built
  inside omni_init_app_state;
• duplex — omni_init_app_state delegates to _init_duplex_app_state and
  returns before the generate branch (api_server.py:800-802), which then builds
  the chat handler in _init_duplex_chat (:727);
• pure diffusion — OmniOpenAIServingChat.for_diffusion (:813), which
  bypasses BaseServing.__init__ (cls.__new__(cls)).

So every served omni process keeps the process-global default full mode no
matter what the CLI says, while the same flag on the same model under
vllm serve behaves as documented.

This is the "accepted but silently ignored flag" class that omni's own CLI
already guards against elsewhere: serve.py:258-282 rejects seven parallelism
flags at parse time precisely because they "would silently disagree with those
sources of truth". These two flags are not rejected and not wired.

Reproduction

No GPU and no live server needed — the flag reaches (or fails to reach) module
state during init:

1. Restore vllm_omni/entrypoints/openai/api_server.py to upstream bytes
   (md5 7189287bf5789f4fad9a74bb7c492d1f, asserted before the run) and keep the
   three tests below.
2. pytest tests/entrypoints/openai_api/test_api_server_guards.py -k fingerprint

Before the fix the two mode assertions report the default fingerprint
(vllm-<version>-01234567) where None / omni-fingerprint is expected, and
the duplex-ordering test reports the default where omni-duplex is expected.

Fix

Apply the mode at the one point that dominates all three construction paths —
near the top of omni_init_app_state, above the served-model-name resolution and above both early returns (the duplex delegation and the pure-diffusion path)
(+13 lines, one import plus twelve):

    set_default_fingerprint_mode(
        getattr(args, "fingerprint_mode", "full"),
        getattr(args, "fingerprint_value", None),
    )

The call is a byte-for-byte copy of upstream's, including the getattr
defaults, so an args object without these attributes (older callers, tests)
behaves exactly like upstream's default rather than clearing the mode.

Tests

Three regression tests in tests/entrypoints/openai_api/test_api_server_guards.py,
all driving the real omni_init_app_state, plus one shared fixture that
snapshots and restores the module globals (_DEFAULT_MODE / _CUSTOM_VALUE) so
the mode cannot leak into other tests:

• test_cli_fingerprint_mode_reaches_the_serving_layer — control asserts the
  default really does stamp a fingerprint, then fingerprint_mode="none" must
  yield None and custom + fingerprint_value="omni-fingerprint" must yield
  that literal.
• test_fingerprint_mode_keeps_upstream_default_when_args_omit_it — with the
  attributes absent and a stale none / "stale-value" configured
  beforehand, init must restore upstream's full default (exercises both
  getattr fallback branches).
• test_fingerprint_mode_is_set_before_the_duplex_init_path — stubs
  _init_duplex_app_state and asserts the mode is already applied when that
  path runs, pinning the ordering the duplex early-return depends on.

Executed locally on this branch (CPU, WSL, vllm 0.30.0):

command	result
pytest .../test_api_server_guards.py -k fingerprint	3 passed
pytest .../test_api_server_guards.py	67 passed
pytest .../test_duplex_api_server.py .../test_omni_sleep_wakeup.py	56 passed
pytest .../test_video_server.py	140 passed, 7 failed — the same 7 fail on the unpatched base tree (missing optional dep cache_dit), unrelated to this change
ruff check / ruff format --check on both touched files	all checks passed / already formatted

Alternatives considered

• Reject the two flags at parse time, the way serve.py:258-282 rejects the
  seven parallelism flags. That would also remove the silent disagreement, but it
  makes an inherited upstream flag unusable rather than correct, and the flags
  are honest capability switches (they only shape the system_fingerprint
  field). Wiring them is the smaller change: no CLI surface changes, no new
  error message. If you would rather have the rejection, I am happy to redo it
  that way.
• Calling upstream's init_generate_state instead of copying this one
  statement is not available: it also constructs the whole generate-side
  surface (state.serving_chat, pooling, etc.), which omni_init_app_state
  deliberately builds itself. Applying the one global statement leaves that
  structure untouched.

Negative control

api_server.py is restored to its upstream bytes at file level (the driver
asserts the md5 7189287b… and that the added text is gone before running —
not git stash, which silently no-ops here), with the new tests left in place.
All three fail, each for the production reason:

test_cli_fingerprint_mode_reaches_the_serving_layer
    AssertionError: assert 'vllm-0.30.0-01234567' is None

test_fingerprint_mode_keeps_upstream_default_when_args_omit_it
    AttributeError: 'NoneType' object has no attribute 'startswith'
    # the stale `none` mode survives: the default is never restored

test_fingerprint_mode_is_set_before_the_duplex_init_path
    {'fingerprint': 'vllm-0.30.0-01234567'} != {'fingerprint': 'omni-duplex'}

Re-applying the patch turns them green again (3 passed), and no other test in the
file changes state.

Edge-case and branch review

• custom without --fingerprint-value → None, exactly as upstream (the mode
  is passed through; upstream does not validate the pair at this call site).
• Args object without either attribute → falls back to full / None, tested.
• Duplex and diffusion paths → covered by placing the call above the duplex
  early-return; the diffusion handler never reads the mode either way, which is
  upstream's behaviour for a handler upstream also does not construct.
• Re-entry: set_default_fingerprint_mode is a plain global assignment; the
  second init overwrites the first, no accumulation.
• Process-global scope is deliberate — it is upstream's design, and matching it
  is the point of this change.

Related PRs

Same wiring-drift family (an upstream behaviour that build_app assumes but
omni_init_app_state never performs). Each is an independent hunk and they can
be reviewed separately:

• [Bugfix] Reject mounted routes whose handler is never wired #8169 — mounted route whose handler is never wired (/tokenize,
  /detokenize in diffusion/duplex).
• [Bugfix][Frontend] Forward the serve options omni's handler wiring drops #8209 — serve options dropped on the way into the chat handler.
• [Bugfix] Register tool-call parser metrics in omni's init paths #8212 — tool-call parser metrics never registered (init_parser_metrics).
• [Bugfix] Wire the generative-scoring handler that build_app mounts for every server #8242 — serving_generative_scoring never set.
• this PR — set_default_fingerprint_mode never called.

If you would rather review these as a single "omni init parity" change set, say
so and I will fold them into one PR with a combined test matrix.

AI assistance

AI assistance was used to prepare this change. Every command and result above was
run locally on this branch's frozen source (md5
c09db4024190d44eb4b412ae463337ba) and is reported verbatim.

Compatibility

The flags already exist (they come from upstream's parser) and nothing is added,
removed or renamed. With neither flag passed, the mode stays full and every
response carries exactly the fingerprint it carries today; with a flag passed,
responses carry what the operator asked for, which is the point of the change.
The process-global scope is upstream's own design: matching it is what makes
vllm serve and vllm serve --omni agree.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/error_contracts.md, docs/design/module/profiling.md, docs/design/module/entrypoints.md.

Module owners: @alex-jw-brooks @linyueqian @NickCao

Routing: @alex-jw-brooks via module of the changed files, module named in the PR description, semantic router, CODEOWNERS; @linyueqian via module of the changed files, module named in the PR description, semantic router, CODEOWNERS; @NickCao via module of the changed files, module named in the PR description, semantic router, CODEOWNERS

@apex-mochen, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 520e5c08c356 produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[apex-mochen]

Author self-review (@apex-mochen), as the review bot asks.

What I checked

• Scope. 2 files, +175/-0. The source change is one import plus twelve lines
  inside omni_init_app_state; no other file, route, or serving class is touched.
• Root cause, read from upstream rather than inferred.
  set_default_fingerprint_mode is called exactly once upstream, at the top of
  init_generate_state (vllm/entrypoints/generate/api_router.py:89-96), and each
  BaseServing.__init__ reads the module global once and caches it
  (vllm/entrypoints/generate/base/serving.py:161-172). omni_init_app_state
  never calls that function — in any of its three construction paths (multi-stage,
  duplex delegation, diffusion cls.__new__) — so the inherited CLI flags are
  accepted by make_arg_parser and then ignored.
• Placement dominates all three paths. The call sits before the duplex
  delegation (api_server.py:800-802), before the diffusion path, and before the
  multi-stage construction. The ordering the duplex early-return depends on is
  pinned by test_fingerprint_mode_is_set_before_the_duplex_init_path, which
  observes the mode from inside a stubbed _init_duplex_app_state.
• Edge cases / branches. Args object without either attribute → the getattr
  defaults give upstream's full/None and do not clear a stale mode (test 2
  sets a stale none/"stale-value" first); custom without a value; repeated init
  (plain global assignment, overwrites, no accumulation). All three new branches
  are covered by the three tests.
• Cross-provider / backend impact. None: no provider, backend, model, or
  serving-class code changes. The copied statement is byte-for-byte upstream's,
  including the getattr defaults, so any caller passing a differently shaped
  args behaves exactly as upstream would.
• Negative control. api_server.py restored at file level to upstream bytes
  (7189287b…, asserted before the run that the added text is gone) with the new
  tests kept: all three fail, each for the production reason quoted in the body.
  Re-applying → 3 passed.
• Test runs, on this branch, CPU/WSL/vLLM 0.30.0. -k fingerprint 3 passed;
  guards file 67 passed; duplex + sleep-wakeup 56 passed; test_video_server.py
  140 passed / 7 failed — the identical 7 fail on the unpatched base (optional dep
  cache_dit missing), i.e. pre-existing, not a regression. ruff check and
  ruff format --check clean on both touched files; no CR bytes in either file
  after patching.
• Disclosure. AI assistance was used to prepare this change; I reviewed every
  line and ran every command reported above myself.

No GPU was used or needed.

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Direct under experiment vllm-omni-strict-5050-20260829.

[vllm-omni-review-bot]

Omni ReviewBot review

PR description

Omni accepted upstream --fingerprint-mode / --fingerprint-value via the shared serve parser but never applied them, because omni_init_app_state builds serving handlers itself and never calls upstream init_generate_state. This PR copies upstream’s one-shot set_default_fingerprint_mode(...) into the top of omni_init_app_state, above duplex early-return and pure-diffusion construction, so module globals match CLI before any handler caches system_fingerprint. With no flags, behavior stays full; with flags set, responses carry the operator-requested fingerprint. Three CPU regression tests pin multi-stage reachability, getattr defaults restoring a stale mode, and duplex ordering.

Change flow

flowchart TD
  A["[EXISTING] CLI --fingerprint-mode / --fingerprint-value"]:::existing --> B["[CHANGED] omni_init_app_state applies set_default_fingerprint_mode"]:::changed
  B --> C["[EXISTING] Module fingerprint globals"]:::existing
  C --> D["[EXISTING] Multi-stage / duplex / diffusion serving constructors"]:::existing
  B --> E["[NEW] fingerprint_* regression tests in test_api_server_guards.py"]:::new
  E --> C
  classDef existing fill:#e5e7eb,stroke:#6b7280,color:#111827
  classDef changed fill:#fef3c7,stroke:#d97706,color:#451a03,stroke-width:2px
  classDef new fill:#dcfce7,stroke:#16a34a,color:#052e16,stroke-width:2px
  classDef removed fill:#fee2e2,stroke:#dc2626,color:#450a0a,stroke-width:2px

Loading

CI at ba5b1abb2a50 (2026-09-28T20:25:20.259023+00:00): required check(s) blocking: buildkite/vllm-omni (missing), and -5 more.

No actionable findings.

[apex-mochen]

Hi maintainers, could someone review this PR and trigger the required Buildkite checks?

Pre-commit and the wheel build are passing, but I don't see a Buildkite status on the current head. Related PR #8242 is also awaiting review and CI.

Happy to make any changes requested. Thanks!

[apex-mochen]

Hi @alex-jw-brooks, software engineering student / open-source contributor here. This fix applies --fingerprint-mode / --fingerprint-value in omni's CLI entrypoints, where they're currently parsed but ignored. Code and tests are ready, checks pass. Could you take a look when you have a moment?

[hsliuustc0106]

[P2] Include the configured fingerprint in Omni chat responses

The mode is now set before handler construction, but OmniOpenAIServingChat.chat_completion_full_generator() overrides upstream's generator and constructs OmniChatCompletionResponse at vllm_omni/entrypoints/openai/serving_chat.py:2530 without system_fingerprint=self.system_fingerprint. Consequently, a valid non-streaming /v1/chat/completions request still returns system_fingerprint: null with --fingerprint-mode custom --fingerprint-value omni-fingerprint. Please propagate the cached fingerprint into the response and add a response-level regression test; the new tests only inspect the module global after initialization.

[hsliuustc0106]

Rechecked head ba5b1abb2a50ff08cdf26d39f70888b197934ec1; this finding still applies. Setting the global mode fixes initialization, but the non-streaming Omni chat response at serving_chat.py:2530 still omits system_fingerprint=self.system_fingerprint, so --fingerprint-mode custom --fingerprint-value omni-fingerprint still produces system_fingerprint: null on that path. The new tests inspect the global after initialization and do not assert the response field.

I would include the response propagation and a response-level regression test here so the accepted flags work for Omni chat clients. If this is intentionally limited to initialization wiring, please narrow the PR's response-level claims and explicitly document which routes remain unsupported, with a tracked follow-up.

This was a static re-review; I did not execute the fork's tests locally. The visible checks passed, but no Buildkite result is visible on this head and the merge gate remains BLOCKED.

[apex-mochen]

Thanks for re-checking — agreed, and done in this PR. b6cca317 passes system_fingerprint=self.system_fingerprint into the OmniChatCompletionResponse built by chat_completion_full_generator (as upstream's generator does), and adds a response-level test: it runs the real multistage init with --fingerprint-mode custom --fingerprint-value omni-fingerprint (and none) and asserts the field on the response returned by chat_completion_full_generator.

Evidence: reverting only serving_chat.py turns the custom case red (assert None == 'omni-fingerprint'); with the fix, -k fingerprint gives 5 passed and the full test_api_server_guards.py 69 passed. ruff 0.14.10 is clean and the branch merges cleanly with current main.

Scope note: streaming chunks (OmniChatCompletionStreamResponse) and the diffusion chat paths (handler built via for_diffusion, which bypasses BaseServing.__init__) still don't carry the fingerprint. I've left them out to keep this PR small and will open a tracked follow-up, unless you'd prefer them here.

[apex-mochen]

Thanks again for the recheck. Following up on the scope note in my previous reply: 520e5c08 also covers the streaming path, so both non-streaming and streaming chat responses now carry the configured fingerprint in this PR.

• Non-streaming (b6cca317, unchanged): OmniChatCompletionResponse in chat_completion_full_generator gets system_fingerprint=self.system_fingerprint.
• Streaming (520e5c08): same rule as upstream's ChatCompletionStreamResponse — the fingerprint goes on the terminal chunk (finish_reason set) when include_usage is off, and on the trailing usage chunk when it is on. Applied to the text, audio, image and fallback-stop chunks in chat_completion_stream_generator.

Tests: test_cli_fingerprint_reaches_the_streaming_chat_chunks runs the real stream generator for custom / none × include_usage on/off × ["text"] / ["text", "audio"] (the latter ends on the fallback stop chunk), and asserts exactly one chunk carries the fingerprint and that it is the final one. Reverting only serving_chat.py to b6cca317 turns the 4 custom streaming cases red; with the fix -k fingerprint gives 13 passed, the full test_api_server_guards.py 77 passed, and test_stream_finish_reason.py + test_serving_chat_metrics.py 19 passed. ruff 0.14.10 is clean.

Not changed: the diffusion chat paths (_create_diffusion_chat_completion). That handler is built by for_diffusion, which skips BaseServing.__init__, so there is no vllm_config / self.system_fingerprint to send, and upstream has no equivalent path. Happy to handle it separately if you want it covered.

Could you take another look when you have a moment?

[apex-mochen]

Hi @hsliuustc0106, thanks for the review!

Re: P2 fingerprint in Omni chat responses — agreed that the configured fingerprint should surface in responses. That's a broader behavior change than this PR's scope (which only wires the existing flags through the init paths). I'd prefer to keep this PR focused on the wiring fix and open a follow-up for the response-level fingerprint propagation. Let me know if you'd rather I bundle it here.

[vllm-omni-review-bot]

Omni ReviewBot: no human activity for 7 days

@apex-mochen this pull request has had no human commit, comment or review since 2026-10-12. Please confirm the current plan and next step. The author or a maintainer decides whether to change the PR state.

To keep it moving, any one of these is enough: push an update, reply to the open blocker, or post the current plan and timeline.

[vllm-omni-review-bot]

Omni ReviewBot: no human activity for 14 days

@apex-mochen this pull request has had no human commit, comment or review since 2026-10-12. Please consider marking this PR as draft until work can resume. The author or a maintainer decides whether to change the PR state.

To keep it moving, any one of these is enough: push an update, reply to the open blocker, or post the current plan and timeline.

[apex-mochen]

Closing this stale bugfix PR to keep the queue focused on core feature work. It is not a technical rejection: the branch and review history remain available, and it can be reopened if maintainers want it after the core work lands.