Skip to content

[Bugfix][Rust Frontend] Backport integer KV handoff conversion to v0.29 - #56348

Closed
alec-flowers wants to merge 1 commit into
vllm-project:releases/v0.29.0from
alec-flowers:agent/v029-grpc-integer-handoff
Closed

alec-flowers wants to merge 1 commit into
vllm-project:releases/v0.29.0from
alec-flowers:agent/v029-grpc-integer-handoff

Conversation

@alec-flowers

Copy link
Copy Markdown
Contributor

Purpose

Backport the integral protobuf-number conversion from #54814 to releases/v0.29.0. Main already contains the fix and handoff coverage; the release branch diverged before that commit. No overlapping release backport was found.

v0.29.0's Rust gRPC receiver converts pp_size: 1 into JSON 1.0. NIXL then fails at range(remote_pp_size) with TypeError: 'float' object cannot be interpreted as an integer, preventing native prefill/decode transfer.

This takes only the numeric conversion from #54814, preserving safe integral values recursively while leaving fractional expiry timestamps and out-of-range values as floats. One request-conversion regression checks the KV metadata delivered to the engine.

Test plan and results

  • The new regression fails after removing the normalization guard; with the backport, all 14 grpc::convert::tests pass:
    cargo +1.95 test --locked --manifest-path rust/Cargo.toml -p vllm-server --lib grpc::convert::tests
  • rustfmt +1.95 --edition 2024 --check rust/src/server/src/grpc/convert.rs and git diff --check pass.
  • Real native gRPC P/D validation: Qwen/Qwen3-0.6B, vLLM 0.29.0, Dynamo 1.5.0.dev20260910, NIXL 1.3.2, separate prefill/decode engines sharing an RTX 6000 Ada. The same 641-token prompt with eight requested tokens and KV load failure policy fail fails on the stock binary with the TypeError above and zero completion tokens. Replacing only the Rust binary with this backport passes with eight completion tokens and finish reason length.
  • No model, kernel, or sampling behavior is changed; validation covers the serving protocol regression rather than model quality.

AI assistance was used to investigate, implement, and test this draft.

Backport the integer normalization from vllm-project#54814 with a focused KV handoff regression covering integer metadata and fractional expiry.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working rust

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant