[taneem-ibrahim]

Purpose

Fixes #54116.

The /inference/v1/generate endpoint validated sampling_params for MessagePack compatibility before dispatch, but omitted kv_transfer_params and ec_transfer_params. Oversized integers in either field therefore failed later during engine serialization and surfaced as HTTP 500 InternalServerError.

This change validates all three values together before engine dispatch. Serialization errors now use the existing HTTP 400 BadRequestError response path, while valid generation remains unchanged.

Reproducer

Start the server:

PATH="$PWD/.venv/bin:$PATH" CUDA_VISIBLE_DEVICES=0 \
vllm serve Qwen/Qwen3-0.6B \
  --dtype bfloat16 \
  --max-model-len 1024 \
  --enforce-eager \
  --no-enable-prefix-caching

Send requests to /inference/v1/generate using:

{
  "model": "Qwen/Qwen3-0.6B",
  "token_ids": [1, 2, 3],
  "sampling_params": {
    "max_tokens": 1,
    "temperature": 0.0,
    "seed": 0
  },
  "stream": false,
  "kv_transfer_params": {
    "oversized_integer": 158541820065335999920888242752337176824
  }
}

Repeat with ec_transfer_params in place of kv_transfer_params, then omit both fields for the ordinary-generation control request.

Output on main

{"error_type": null, "has_choices": true, "probe": "ordinary_generation", "runtime_seconds": 0.031, "status": 200}
{"error_type": "InternalServerError", "has_choices": false, "probe": "kv_transfer_params", "runtime_seconds": 0.003, "status": 500}
{"error_type": "InternalServerError", "has_choices": false, "probe": "ec_transfer_params", "runtime_seconds": 0.001, "status": 500}

Output on branch

{"error_type": "BadRequestError", "probe": "kv_transfer_params", "runtime_seconds": 0.004, "status": 400}
{"error_type": "BadRequestError", "probe": "ec_transfer_params", "runtime_seconds": 0.001, "status": 400}
{"has_choices": true, "probe": "ordinary_generation", "runtime_seconds": 0.029, "status": 200}

Test Plan

PATH="$PWD/.venv/bin:$PATH" CUDA_VISIBLE_DEVICES=0 \
HF_HUB_CACHE=/tmp/vllm-hf-cache  \
.venv/bin/python -m pytest \
  tests/entrypoints/openai/test_openai_schema.py::test_openapi_stateless \
  -q --tb=short

1 passed, 27 subtests passed, 0 failed, 0 skipped
450.85s pytest; 461.96s wall

PATH="$PWD/.venv/bin:$PATH" CUDA_VISIBLE_DEVICES=0 \
HF_HUB_CACHE=/tmp/vllm-hf-cache  \
.venv/bin/python -m pytest \
  tests/entrypoints/scale_out/token_in_token_out/ \
  -q --tb=short

38 passed, 0 failed, 0 skipped
261.64s pytest; 272.57s wall

AI assistance disclosure

OpenAI Codex (GPT-5) assisted with drafting the code.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[yewentao256]

LGTM, thanks for the work!

[github-actions]

✅ @taneem-ibrahim, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[taneem-ibrahim]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86196 for commit bfa3e3f799e2.