docs(inference): correct stale serving_tokens.py docstrings for vLLM 0.22 - #2878
Merged
Merged
Conversation
…0.22 The module docstring and inline comments described vLLM 0.20 and claimed that ``data_parallel_rank`` header routing and server-side ``max_tokens`` defaulting are "not in the upstream protocol" / "drop once vLLM patches upstream". The pin is now 0.22.0, and both have landed upstream in ``ServingTokens`` (verified against the installed vLLM source). Update the wording so it reflects reality: upstream now covers them and we keep equivalent guards; the genuinely prime-RL-specific piece is the compact ``routed_experts`` export. Also retarget the stale ``OpenAIServingChat`` analog references to upstream ``ServingTokens``. Comments/docstrings only — no code changes. (The ``usage`` field is still genuinely dropped by upstream's response schema, and the ``kv_transfer_params`` note is left untouched as unverified.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hallerite
marked this pull request as ready for review
June 25, 2026 04:40
samsja
approved these changes
Jun 25, 2026
hallerite
added a commit
that referenced
this pull request
Jun 25, 2026
Brings in the 5 v1-cleanup PRs (#2872 filter fix, #2874 messages.py delete, #2875 get_metrics removal, #2876 is_vlm removal, #2877 detection_index removal, #2878 serving_tokens docs) plus the verifiers bump to 0.1.15.dev394. Conflict resolutions: - pyproject.toml: take main's verifiers floor (dev394 > dev390). - deps/verifiers: take main's 20abcd91 (dev394) — strict descendant of our e306ba52 (dev390), already includes Trace.to_record(). - utils/client.py: keep our new prefill score() method, drop the dead get_metrics() that #2875 removed. - filters.py / FilterResult: detection_index dropped (#2877); our branch-stream fix (#2872) preserved. uv lock is a no-op; orchestrator unit suite green (99 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The module docstring + inline comments said vLLM 0.20 and claimed
data_parallel_rankheader routing and server-sidemax_tokensdefaulting are "not in the upstream protocol" / "drop once vLLM patches upstream". The pin is now 0.22.0, and both have landed upstream inServingTokens(verified against the installed vLLM source).Reworded so it reflects reality (upstream covers them; we keep equivalent guards; the genuinely prime-RL-specific piece is the compact
routed_expertsexport) and retargeted the staleOpenAIServingChatanalog references to upstreamServingTokens.Comments/docstrings only — no code changes. The
usagefield is still genuinely dropped by upstream's response schema, and thekv_transfer_paramsnote is left untouched (unverified). Surfaced by the verifiers-v1 redundancy audit; the override code for DP-rank/max_tokens is now arguably redundant too — a possible follow-up.Note
Low Risk
Documentation-only changes with no runtime or behavioral code edits.
Overview
Updates comments and docstrings only in
serving_tokens.pyso they match the vLLM 0.22 pin and current upstreamServingTokensbehavior.The module header and inline notes no longer claim that
data_parallel_rankrouting and server-sidemax_tokensdefaulting are missing from upstream or “drop once vLLM patches.” They now state that vLLM 0.22 already implements those paths and that prime-RL keeps equivalent guards; the prime-RL-specific extension called out is still compactrouted_expertsexport (plus existing notes likeusage/kv_transfer_paramsare unchanged).References that pointed at vLLM 0.20 and
OpenAIServingChatformax_tokensanalogies are retargeted to 0.22 and upstreamServingTokens.Reviewed by Cursor Bugbot for commit ff60e9b. Bugbot is set up for automated code reviews on this repo. Configure here.