[Bugfix][Rust Frontend] Backport integer KV handoff conversion to v0.29 - #56348
Closed
alec-flowers wants to merge 1 commit into
Closed
alec-flowers wants to merge 1 commit into
alec-flowers wants to merge 1 commit into
Conversation
Backport the integer normalization from vllm-project#54814 with a focused KV handoff regression covering integer metadata and fractional expiry. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Backport the integral protobuf-number conversion from #54814 to
releases/v0.29.0. Main already contains the fix and handoff coverage; the release branch diverged before that commit. No overlapping release backport was found.v0.29.0's Rust gRPC receiver converts
pp_size: 1into JSON1.0. NIXL then fails atrange(remote_pp_size)withTypeError: 'float' object cannot be interpreted as an integer, preventing native prefill/decode transfer.This takes only the numeric conversion from #54814, preserving safe integral values recursively while leaving fractional expiry timestamps and out-of-range values as floats. One request-conversion regression checks the KV metadata delivered to the engine.
Test plan and results
grpc::convert::testspass:cargo +1.95 test --locked --manifest-path rust/Cargo.toml -p vllm-server --lib grpc::convert::testsrustfmt +1.95 --edition 2024 --check rust/src/server/src/grpc/convert.rsandgit diff --checkpass.failfails on the stock binary with the TypeError above and zero completion tokens. Replacing only the Rust binary with this backport passes with eight completion tokens and finish reasonlength.AI assistance was used to investigate, implement, and test this draft.