[richard-for-bugen]

Motivation

• Python: truncate_prompt_tokens budgets against the context length only. -1 truncates the prompt to max_model_len - 1 (TokenizeParams.max_truncation_tokens, vllm/renderers/params.py), holding one token back for generation, and an explicit value is rejected only above max_model_len. max_tokens is an upper bound that get_max_tokens() clamps after tokenization, not a reservation ([Bugfix] Treat max_tokens as output upper bound, not input-space reservation vllm-project/vllm#57035). [Bugfix][Frontend] Fix auto truncation without an output token limit vllm-project/vllm#62010 fixed the last gap, a request without an output limit: it used to truncate to the full context and then fail with no room to generate.
• Rust today: TextRequestProcessor budgets max_model_len - max_tokens (max_model_len when max_tokens is absent):
  • truncate_prompt_tokens=-1 without max_tokens truncates to the full context, then resolve_max_tokens rejects the request as too long. Auto-truncation never helps such a request.
  • With max_tokens, it drops max_tokens prompt tokens that Python keeps, so the same request sees a different prompt.
  • An explicit value above max_model_len - max_tokens is rejected, while Python accepts it.
• This PR: -1 keeps max_model_len - 1 tokens, and an explicit value is checked against max_model_len, with Python's message. Generation then gets the remaining context through the existing resolve_max_tokens clamp.

Example with max_model_len=10, a 15-token prompt, and truncate_prompt_tokens=-1:

max_tokens	Before	After (= Python)
absent	400, prompt too long	9 prompt tokens, max_tokens=1
5	5 prompt tokens, max_tokens=5	9 prompt tokens, max_tokens=1

Ownership and layering

Truncation is owned by vllm-text (PromptTruncation::apply, called from TextRequestProcessor::prepare_prompt_tokens). Chat, completions, and gRPC all reach it, so the fix there covers every entry point that accepts truncate_prompt_tokens. Only generation paths set a truncation policy (/tokenize and /inference/v1/generate never do), so the one held-back token always applies. That matches Python, where only no-output requests such as pooling keep the full context. prepare_prompt_tokens no longer needs max_tokens, and the error variant is renamed to TruncatePromptTokensExceedsMaxModelLen to match what it now checks.

Validation

Commands from .buildkite/scripts/run-rust-frontend-cargo-ci.sh (style-clippy and test), run from the worktree root:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check   # ok
cargo sort --workspace --check rust                          # ok
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans   # ok
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings   # ok
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast
# 2177 tests run: 2177 passed, 13 skipped

Tests:

• auto_truncation_leaves_one_output_token (vllm-text): the table above through TextRequestProcessor::prepare. Fails before the fix: setting -1 back to the full context makes the max_tokens-absent case panic on the prompt-too-long error.
• input_budget_holds_back_one_token_for_output and fixed_limit_cannot_exceed_max_model_len replace the unit tests that pinned the old budget.

Python parity item chat+completions.truncate_prompt_tokens_budget (category 4), queued at the 2026-10-14 baseline and refreshed by vllm-project#62010.

🤖 Generated with Claude Code

[richard-for-bugen]

Promoted upstream as vllm-project#62642 (replayed onto current main, no conflicts; same change). Leaving this open until it merges.

[BugenZhao]

Merged upstream as vllm-project#62642 (97a82d3).