Skip to content

[Rust Frontend] Python parity run 2026-11-03 (fields) - #111

Closed
richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-run-2026-11-03-fields
Closed

richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-run-2026-11-03-fields

Conversation

@richard-for-bugen

Copy link
Copy Markdown

Summary

Python parity run PR (area fields), based on upstream main at 39dbf51839. Items: 1.

vllm-project#63902 (merged 2026-11-03) made the Rust frontend serve the stateless Responses API, so its request fields are now category 2 surfaces. This run audits every field of Python's ResponsesRequest (vllm/entrypoints/openai/responses/protocol.py) against the Rust DTO (rust/src/server/src/routes/openai/responses/types.rs), which, like the chat DTO, lets serde ignore unknown top-level keys. One field Python honors was silently dropped; it is fixed here as a trivial implementation.

Item key Category Python change Consumer trace Rust action
responses.thinking_token_budget 2 vllm-project#63896 (5c19391400, 2026-10-29) ResponsesRequest.to_sampling_params → SamplingParams.thinking_token_budget → engine reasoning budget (validate_thinking_token_budget) trivial implementation: declared on the DTO and passed to the shared SamplingParams; vllm-text lower_sampling_params already validates and normalizes it for chat (-1 = unlimited, negative → 400)

Ownership and layering. The DTO field belongs to the Responses wire type, and its lowering to prepare_responses_request, which builds the same vllm_text::SamplingParams chat completions builds; validation and normalization stay in the shared vllm-text lowering, so the two routes cannot drift.

Audit of the remaining Python fields

Everything else Python's ResponsesRequest declares is either honored by Rust, rejected explicitly, or not a silent behavior drop:

  • Honored: model, input (message, function_call, function_call_output, reasoning items), instructions, tools (function), tool_choice (none/auto/required/function), parallel_tool_calls, max_output_tokens, max_tool_calls (echoed; it only limits built-in tools), metadata, reasoning.effort (including the enable_thinking default), include_reasoning, service_tier (echoed), stream, temperature, top_p, top_k, min_p, text.format, truncation, user, penalties, seed, stop, stop_token_ids, ignore_eos, skip_special_tokens, include_stop_str_in_output, logit_bias, watermarking, request_id, session_id, priority, cache_salt, chat_template_kwargs, structured_outputs, kv_transfer_params, ec_transfer_params, vllm_xargs.
  • Accepted no-ops, documented by [Rust Frontend] Add the stateless Responses API (streaming and non-streaming) vllm-project/vllm#63902: reasoning.summary, text.verbosity, include=["reasoning.encrypted_content"].
  • Explicit 400 or 404 today:
    • background=true, previous_response_id, item_reference items, previous_input_messages (Harmony), and prompt;
    • other include values, including message.output_text.logprobs, and non-zero top_logprobs;
    • enable_response_messages, spaces_between_special_tokens=false, media_io_kwargs, and mm_processor_kwargs;
    • allowed_tools tool choices, input_file parts, input_image.file_id, and encrypted reasoning items;
    • built-in and namespace tools, and other item types, rejected at deserialization.
  • store: not declared. A store=true request runs unstored, which is Python's implicit downgrade when VLLM_ENABLE_RESPONSES_API_STORE is unset. With the variable set, Python keeps a store; that case gets an env placeholder in the env run PR ([Rust Frontend] Python parity run 2026-10-14 (env) #10).
  • Echo-only fields (prompt_cache_key, prompt_cache_options, prompt_cache_retention, safety_identifier): Python accepts them, does not act on them, and echoes them on the response object, where its ResponsesResponse also carries input_messages/output_messages as null. Rust drops the request fields and its response object lacks those keys. That is response-shape drift (category 4), queued as its own item rather than placed here.
  • Render-only (mm_payload, return_mm_kwargs): ignored on /v1/responses in Python too.

Nested shapes match: function tools reuse the chat Function type (name, description, parameters, strict, defer_loading), and the input item and content part types are closed enums, so an unknown variant fails deserialization rather than being dropped.

Not in this PR (queued separately)

Needs Bugen

None.

Validation

New route test responses_thinking_token_budget_reaches_engine_core checks the engine request for absent, 256 and -1 (→ no budget). It fails with the field dropped (left: None, right: Some(256)).

Rust CI set from .buildkite/scripts/run-rust-frontend-cargo-ci.sh, on the tip:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check
cargo sort --workspace --check rust
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast

All green on the tip 4cc1c46a87: fmt, sort, deny (bans ok) and workspace clippy clean; nextest 2511 passed, 13 skipped.

The e2e run PR (#47) keeps test_thinking_token_budget.py::test_thinking_token_budget_responses_api skipped under the Rust frontend until this merges; it then gets enabled there.

Codex review (gpt-6.1-sol, high): approve with nits. It traced the field through prepare_responses_request → ChatRequestProcessor → lower_sampling_params → EngineCoreSamplingParams and confirmed the Python target is deliberate (vllm-project#63896 added Responses support, docs and a chat/Responses comparison test). One P3, fixed: the regression assertion ran inside the mock engine, so a mismatch stalled until nextest's timeout; the test now captures the budget through a channel and asserts after the response drains (red run fails in 0.07 s). Test-only follow-up, so no re-review.

Placeholders turn previously ignored inputs into explicit errors. This run has none: its one item makes a previously ignored field work.

Python's ResponsesRequest takes thinking_token_budget and puts it on the
request's SamplingParams (vllm-project#63896), like chat
completions. The Rust Responses DTO did not declare the field, so serde
dropped it and reasoning ran past the budget. Declare it and pass it to
the shared sampling params, where vllm-text validates and normalizes it
exactly as for chat (-1 means unlimited).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
@BugenZhao

Copy link
Copy Markdown
Owner

The e2e run PR (#47) keeps test_thinking_token_budget.py::test_thinking_token_budget_responses_api skipped under the Rust frontend until this merges; it then gets enabled there.

Let's fold this into e2e run PR

@richard-for-bugen

Copy link
Copy Markdown
Author

Folded into #47 as e39538d (this PR's commit, cherry-picked), with f577830 dropping the Rust skip on test_thinking_token_budget_responses_api, which now passes under the Rust frontend.

@richard-for-bugen
richard-for-bugen Bot deleted the richard/parity-run-2026-11-03-fields branch November 4, 2026 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant