Repository navigation
[Rust Frontend] Python parity run 2026-11-03 (fields) - #111
Closed
richard-for-bugen[bot] wants to merge 1 commit into
Closed
richard-for-bugen[bot] wants to merge 1 commit into
richard-for-bugen[bot] wants to merge 1 commit into
Conversation
Python's ResponsesRequest takes thinking_token_budget and puts it on the request's SamplingParams (vllm-project#63896), like chat completions. The Rust Responses DTO did not declare the field, so serde dropped it and reasoning ran past the budget. Declare it and pass it to the shared sampling params, where vllm-text validates and normalizes it exactly as for chat (-1 means unlimited). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
Owner
Let's fold this into e2e run PR |
Author
richard-for-bugen
Bot
deleted the
richard/parity-run-2026-11-03-fields
branch
November 4, 2026 08:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Python parity run PR (area
fields), based on upstreammainat39dbf51839. Items: 1.vllm-project#63902 (merged 2026-11-03) made the Rust frontend serve the stateless Responses API, so its request fields are now category 2 surfaces. This run audits every field of Python's
ResponsesRequest(vllm/entrypoints/openai/responses/protocol.py) against the Rust DTO (rust/src/server/src/routes/openai/responses/types.rs), which, like the chat DTO, lets serde ignore unknown top-level keys. One field Python honors was silently dropped; it is fixed here as a trivial implementation.responses.thinking_token_budget5c19391400, 2026-10-29)ResponsesRequest.to_sampling_params→SamplingParams.thinking_token_budget→ engine reasoning budget (validate_thinking_token_budget)SamplingParams;vllm-textlower_sampling_paramsalready validates and normalizes it for chat (-1= unlimited, negative → 400)Ownership and layering. The DTO field belongs to the Responses wire type, and its lowering to
prepare_responses_request, which builds the samevllm_text::SamplingParamschat completions builds; validation and normalization stay in the sharedvllm-textlowering, so the two routes cannot drift.Audit of the remaining Python fields
Everything else Python's
ResponsesRequestdeclares is either honored by Rust, rejected explicitly, or not a silent behavior drop:model,input(message,function_call,function_call_output,reasoningitems),instructions,tools(function),tool_choice(none/auto/required/function),parallel_tool_calls,max_output_tokens,max_tool_calls(echoed; it only limits built-in tools),metadata,reasoning.effort(including theenable_thinkingdefault),include_reasoning,service_tier(echoed),stream,temperature,top_p,top_k,min_p,text.format,truncation,user, penalties,seed,stop,stop_token_ids,ignore_eos,skip_special_tokens,include_stop_str_in_output,logit_bias,watermarking,request_id,session_id,priority,cache_salt,chat_template_kwargs,structured_outputs,kv_transfer_params,ec_transfer_params,vllm_xargs.reasoning.summary,text.verbosity,include=["reasoning.encrypted_content"].background=true,previous_response_id,item_referenceitems,previous_input_messages(Harmony), andprompt;includevalues, includingmessage.output_text.logprobs, and non-zerotop_logprobs;enable_response_messages,spaces_between_special_tokens=false,media_io_kwargs, andmm_processor_kwargs;allowed_toolstool choices,input_fileparts,input_image.file_id, and encrypted reasoning items;namespacetools, and other item types, rejected at deserialization.store: not declared. Astore=truerequest runs unstored, which is Python's implicit downgrade whenVLLM_ENABLE_RESPONSES_API_STOREis unset. With the variable set, Python keeps a store; that case gets an env placeholder in the env run PR ([Rust Frontend] Python parity run 2026-10-14 (env) #10).prompt_cache_key,prompt_cache_options,prompt_cache_retention,safety_identifier): Python accepts them, does not act on them, and echoes them on the response object, where itsResponsesResponsealso carriesinput_messages/output_messagesasnull. Rust drops the request fields and its response object lacks those keys. That is response-shape drift (category 4), queued as its own item rather than placed here.mm_payload,return_mm_kwargs): ignored on/v1/responsesin Python too.Nested shapes match: function tools reuse the chat
Functiontype (name,description,parameters,strict,defer_loading), and the input item and content part types are closed enums, so an unknown variant fails deserialization rather than being dropped.Not in this PR (queued separately)
include=["message.output_text.logprobs"],top_logprobs): explicit 400; Python testsresponses/test_basic.py::test_logprobs,responses/test_simple.py::test_streaming_logprobs.enable_response_messages: explicit 400;responses/test_simple.py::test_enable_response_messages.tools[].type="namespace"([Frontend] Support OpenAI Responses API namespace tools vllm-project/vllm#47024, [Bugfix] Handle malformed namespace tools vllm-project/vllm#53763, [Bugfix][Frontend] Fix Responses namespace aliases in named tool calling vllm-project/vllm#59723): rejected by serde with a generic message;responses/test_namespace_tool_separator.py.Needs Bugen
None.
Validation
New route test
responses_thinking_token_budget_reaches_engine_corechecks the engine request for absent,256and-1(→ no budget). It fails with the field dropped (left: None, right: Some(256)).Rust CI set from
.buildkite/scripts/run-rust-frontend-cargo-ci.sh, on the tip:All green on the tip
4cc1c46a87: fmt, sort, deny (bans ok) and workspace clippy clean; nextest 2511 passed, 13 skipped.The e2e run PR (#47) keeps
test_thinking_token_budget.py::test_thinking_token_budget_responses_apiskipped under the Rust frontend until this merges; it then gets enabled there.Codex review (
gpt-6.1-sol, high): approve with nits. It traced the field throughprepare_responses_request→ChatRequestProcessor→lower_sampling_params→EngineCoreSamplingParamsand confirmed the Python target is deliberate (vllm-project#63896 added Responses support, docs and a chat/Responses comparison test). One P3, fixed: the regression assertion ran inside the mock engine, so a mismatch stalled until nextest's timeout; the test now captures the budget through a channel and asserts after the response drains (red run fails in 0.07 s). Test-only follow-up, so no re-review.Placeholders turn previously ignored inputs into explicit errors. This run has none: its one item makes a previously ignored field work.