[Bugfix][Rust Frontend] Preserve selected-token logprob mode - #56406
Merged
BugenZhao merged 2 commits intoSep 14, 2026
Merged
Conversation
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com>
alec-flowers
force-pushed
the
codex/grpc-selected-logprobs
branch
from
September 11, 2026 18:24
d4a0318 to
db83f67
Compare
alec-flowers
changed the title
alec-flowers
marked this pull request as ready for review
September 11, 2026 21:03
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Member
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Use top_n(0) for omitted and empty candidate selectors so logprob requests do not implicitly include the highest-scoring candidate. Clarify the comment and update the existing request-lowering regression. Validation: 48 gRPC tests passed; updated regression fails before the fix. cargo fmt --all -- --check and git diff --check passed. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Member
|
/ci run |
|
✅ @alec-flowers, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #88744 for commit |
BugenZhao
enabled auto-merge (squash)
September 14, 2026 06:13
Shreya-gaur
pushed a commit
to Shreya-gaur/vllm_private
that referenced
this pull request
Sep 14, 2026
…oject#56406) Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com>
ItsRoy69
pushed a commit
to ItsRoy69/vllm
that referenced
this pull request
Sep 15, 2026
…oject#56406) Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Clients can request log probabilities for specific token IDs at each generated position. The Rust gRPC converter always paired those IDs with
logprobs=1, so requests selecting more than one token failed validation.For example, requesting scores for two valid token IDs
[100, 101]produced:[100, 101][100, 101]logprobscount1None)The new behavior matches Python HTTP's selected-token mode. An unset numeric count does not disable scoring when selected IDs are present. It also avoids applying the ordinary top-N limit to selected IDs: for example, 21 selected IDs are valid with
max_logprobs=20, subject to the separate selected-ID limit and vocabulary checks.Empty and absent selectors now use
top_n(0), returning only the sampled token’s logprob. Numeric top-N requests keep their existing behavior. This aligns request conversion with Python; existing response-rank and batch-padding issues are separate.Validation
From
rust/:cargo test --locked -p vllm-server --lib grpc::convert::tests— 14 passed; regression fails without the fix.cargo test --locked -p vllm-text --lib logprob— 7 passed.cargo fmt --all -- --checkandgit diff --check— passed.max_logprobs=20, and all 42 compared scores match Python within 0.000001 absolute difference.This fixes Rust gRPC request conversion; related #49320 addresses Python HTTP empty-list validation. AI assistance was used for implementation and independent review.