[Rust Frontend][gRPC] Preserve skip_special_tokens decoding option - #52384
Merged
njhill merged 2 commits intoAug 15, 2026
Merged
Conversation
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
biswapanda
force-pushed
the
feat/grpc-skip-special-tokens
branch
from
August 14, 2026 19:22
78b6aba to
cf16afb
Compare
biswapanda
marked this pull request as ready for review
August 14, 2026 19:26
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
biswapanda
force-pushed
the
feat/grpc-skip-special-tokens
branch
from
August 14, 2026 20:10
bb8d02b to
c98c11b
Compare
connorcarpenter15
approved these changes
Aug 14, 2026
Member
|
/ci run |
|
✅ @biswapanda, CI is now available for this PR.
|
njhill
enabled auto-merge (squash)
August 14, 2026 23:59
|
✅ Triggered Buildkite CI #83993 for commit |
4 tasks
zufangzhu
pushed a commit
to zufangzhu/vllm
that referenced
this pull request
Aug 24, 2026
…llm-project#52384) Signed-off-by: Biswa Panda <biswa.panda@gmail.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Align the native Generate gRPC API with vLLM's existing Python serving APIs by carrying the per-request
skip_special_tokensdecoding option throughResponseOptions.The Python chat completions, completions, responses, and token-native APIs default this option to
true, but the gRPC request did not expose it. Consequently, a gRPC caller could not explicitly preserve tokenizer-defined special markers required by some reasoning and tool parsers.The protobuf field is
optionalso the server can distinguish omission from an explicitfalse. Omitted values retain the established Python default oftrue; explicitfalsevalues are preserved. This affects decoded text only and does not change sampling, generated token IDs, or EOS behavior./v1/chat/completionstrue/v1/completionstrue/v1/responsestrue/inference/v1/generatesampling_params.skip_special_tokenstrueTest Plan
TextDecodeOptions.Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model. No documentation change is required for this parity fix.