Repository navigation
[Frontend] Support min_p in the Responses API - #48084
Conversation
|
Documentation preview: https://vllm--48084.org.readthedocs.build/en/48084/ |
|
This pull request has merge conflicts that must be resolved before it can be |
The Responses API already exposes `top_k` and forwards it to `SamplingParams`, but its natural companion `min_p` is missing even though `SamplingParams` fully supports it. Add the `min_p` field to `ResponsesRequest`, resolve its default alongside `top_k`, and plumb it through `to_sampling_params`, mirroring the existing `top_k` handling so `/v1/responses` users get min-p sampling like the other endpoints. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: sungbin1015 <sbin@solbox.com>
e49aa01 to
0b9e493
Compare
|
Rebased onto the latest |
|
/ci run |
|
✅ @sungbin1015, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #93648 for commit |
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #93685 for commit
|
Purpose
The Responses API (
/v1/responses) already exposestop_kas a request fieldand forwards it to
SamplingParams, but its natural companionmin_pismissing — even though
SamplingParams(andSamplingParams.from_optional)fully support it, and the Completions/Chat endpoints both expose it.
This adds the
min_pfield toResponsesRequest, resolves its defaultalongside
top_kinto_sampling_params, and plumbs it through toSamplingParams.from_optional, mirroring the existingtop_khandling exactly.Users of
/v1/responsescan now use min-p sampling like every other generationendpoint.
Not a duplicate: searched open PRs/issues from several angles
(
responses min_p,ResponsesRequest min_p in:body,responses/protocol.py min_p in:body,min_p in:title,responses API sampling parameters).min_pis entirely absent fromresponses/protocol.py(no matches in the file), and no open PR adds samplingparameters to the Responses API. PR #45839 adds sampling params to the separate
Translation API only; this change is orthogonal.
Test Plan
Added a
min_passertion totest_basic_sampling_paramsand a newtest_min_p_defaultverifying the neutral default (0.0).Test Result
ruff check,ruff format, andmypyall pass via pre-commit.Related work
min_pstruck through, so it was dropped before merge; I did not find the reason recorded in that thread. If leavingmin_pout was a deliberate decision rather than scope trimming, please say so and I will close this.stop_token_idsto the sameResponsesRequest/to_sampling_params. It is a different field, but it edits neighbouring lines inresponses/protocol.pyandtest_sampling_params.py, so whichever lands second will need a small rebase. I will rebase this one if [Bugfix][Frontend] Honor stop_token_ids on the Responses API #56592 goes first.AI assistance disclosure: this PR was prepared with AI (Claude) assistance. I
have reviewed every changed line and ran the tests above myself.