[richard-for-bugen]

Motivation

Python's --return-tokens-as-token-ids (FrontendArgs.return_tokens_as_token_ids) sets the server-wide default for rendering logprob tokens as token_id:{id} placeholders. The chat and completions serving classes resolve it per request in _create_chat_logprobs / _create_completion_logprobs:

should_return_as_token_id = (
    return_as_token_id if return_as_token_id is not None else self.return_tokens_as_token_ids
)

So an omitted or null request field takes the server flag, and an explicit true/false overrides it (a request can opt out with false on a server started with the flag). The rule covers output logprobs and completions echo/prompt logprobs.

Rust already honors the per-request return_tokens_as_token_ids field, but declared the flag Unsupported, so a server started with it failed at startup (both under vllm serve with VLLM_USE_RUST_FRONTEND=1, through --args-json, and under vllm-rs serve).

Change

• vllm-cmd: --return-tokens-as-token-ids moves from the Unsupported markers to a real bool on SharedRuntimeArgs (help text kept from Python), carried into ApiServerOptions. Like the other frontend booleans it has no --no-* spelling yet (the inverse-spelling item is parked separately); the inverse sets Python's default anyway.
• vllm-server: ApiServerOptions gains default_return_tokens_as_token_ids (renamed in review, 90e74ff). The chat and completions ResponseOptions keep the request field as Option<bool> instead of collapsing it to false during lowering, and the response stage (the collect and chunk-stream functions of each route, which already receive both ApiServerOptions and ResponseOptions) resolves it once with unwrap_or(<server flag>), as Python's serving classes do.

Layering. The consumer is the OpenAI logprob formatting in the route response stage (routes/openai/utils/logprobs.rs callers); the server-wide default is fixed at construction in ApiServerOptions, and DTO lowering stays request-only, so prepare_*_request (also used by the render routes) takes no server config. /inference/v1/generate is unaffected: Rust serves only output_mode="tokens", whose logprobs carry integer token IDs, and Python's flag only changes its text/derender levels. Open vllm-project#63173 (Python only) adds a parse_context override for derender; not applicable here.

Validation

• New route test server_return_tokens_as_token_ids_is_the_request_default: chat and completions, streaming and non-streaming, against the mock engine, for (server flag, request field) = (on, omitted), (on, null), (on, false), (off, omitted), (off, true); expects token_id:104 exactly when Python would. Red-green: ignoring the server flag fails it (/v1/chat/completions stream=false server_flag=true request={}); ignoring it only in the completions stream fails /v1/completions stream=true.
• New CLI test serve_passes_return_tokens_as_token_ids_into_config; SharedRuntimeArgs/FrontendConfig snapshots refreshed.
• Rust CI command set (fmt, cargo sort, cargo deny bans, clippy -D warnings, cargo nextest run --workspace --all-features): all green, 2296 passed, 13 skipped.
• E2E on a GB200 node (Slurm, exclusive), tests/entrypoints/openai/test_return_tokens_as_ids.py under VLLM_USE_RUST_FRONTEND=1 with vllm-rs built from this branch:
  • The file's server loads a LoRA through --lora-modules, which fails at startup on current main (add_lora ... Expected bool, got null); that is the LoRA wire-layout fix in [Rust Frontend] Drop tensorizer_config_dict from the LoRA request wire layout #29, not this change. With that commit applied on top for the run only:
  • test_chat_return_tokens_as_token_ids_completion[True] (server flag, request field omitted) and [False] (request field) pass.
  • test_completion_return_tokens_as_token_ids_completion[True] and [False] both fail identically on token_strs[1:] == top_logprob_keys: Rust builds each completions top_logprobs entry as a HashMap, so the sampled token is not reliably the first key. That is the separate completions.top_logprobs_key_order gap, present with or without the flag; the token strings themselves are token_id:N in both variants.
  • Python frontend, same checkout: 5 passed.

tests/entrypoints/openai/test_return_tokens_as_ids.py joins Rust CI through the parity e2e run PR once this, #29, and the completions top_logprobs key-order fix have merged (the chat tests need only this and #29).

Dedup

No open upstream PR implements the flag in Rust (searched return_tokens_as_token_ids, return-tokens-as-token-ids). Active vllm-project#62684 edits routes/openai/completions.rs for echo with return_token_ids, a different field; the hunks here touch only the option destructuring in that file.

Codex review

Verdict: approve with nits; no blocking findings (problem real, precedence matches Python's _create_*_logprobs, resolution placed at each response path that formats logprobs; Codex re-ran both new tests red on the base implementation).

• P3, the route test checked only sampled output tokens: fixed; it now collects top candidates too (chat top_logprobs: 2, completions logprobs: 1) and expects both h/H or both placeholders. Echoed prompt tokens were not added: the flag is resolved once per response function before any formatter runs, and the echo/prompt-only formatters already take that same resolved value.

🤖 Generated with Claude Code

https://claude.ai/code/session_012jJ3QaARmRe3jZJYTfHsCB

[BugenZhao]

Suggested change

	pub return_tokens_as_token_ids: bool,
	pub default_return_tokens_as_token_ids: bool,

[richard-for-bugen]

Applied in 90e74ff: ApiServerOptions.default_return_tokens_as_token_ids. The CLI field keeps the flag name --return-tokens-as-token-ids. Full Rust CI command set green (2296 passed).

[richard-for-bugen]

Promoted as vllm-project#63318.

[richard-for-bugen]

Merged upstream as vllm-project#63318.