Skip to content

docs(inference): correct stale serving_tokens.py docstrings for vLLM 0.22 - #2878

Merged
hallerite merged 1 commit into
mainfrom
chore/serving-tokens-doc-vllm-022
Jun 25, 2026
Merged

hallerite merged 1 commit into
mainfrom
chore/serving-tokens-doc-vllm-022

Conversation

@hallerite

@hallerite hallerite commented Jun 25, 2026

Copy link
Copy Markdown
Member

The module docstring + inline comments said vLLM 0.20 and claimed data_parallel_rank header routing and server-side max_tokens defaulting are "not in the upstream protocol" / "drop once vLLM patches upstream". The pin is now 0.22.0, and both have landed upstream in ServingTokens (verified against the installed vLLM source).

Reworded so it reflects reality (upstream covers them; we keep equivalent guards; the genuinely prime-RL-specific piece is the compact routed_experts export) and retargeted the stale OpenAIServingChat analog references to upstream ServingTokens.

Comments/docstrings only — no code changes. The usage field is still genuinely dropped by upstream's response schema, and the kv_transfer_params note is left untouched (unverified). Surfaced by the verifiers-v1 redundancy audit; the override code for DP-rank/max_tokens is now arguably redundant too — a possible follow-up.


Note

Low Risk
Documentation-only changes with no runtime or behavioral code edits.

Overview
Updates comments and docstrings only in serving_tokens.py so they match the vLLM 0.22 pin and current upstream ServingTokens behavior.

The module header and inline notes no longer claim that data_parallel_rank routing and server-side max_tokens defaulting are missing from upstream or “drop once vLLM patches.” They now state that vLLM 0.22 already implements those paths and that prime-RL keeps equivalent guards; the prime-RL-specific extension called out is still compact routed_experts export (plus existing notes like usage / kv_transfer_params are unchanged).

References that pointed at vLLM 0.20 and OpenAIServingChat for max_tokens analogies are retargeted to 0.22 and upstream ServingTokens.

Reviewed by Cursor Bugbot for commit ff60e9b. Bugbot is set up for automated code reviews on this repo. Configure here.

…0.22

The module docstring and inline comments described vLLM 0.20 and claimed that
``data_parallel_rank`` header routing and server-side ``max_tokens`` defaulting
are "not in the upstream protocol" / "drop once vLLM patches upstream". The pin
is now 0.22.0, and both have landed upstream in ``ServingTokens`` (verified
against the installed vLLM source). Update the wording so it reflects reality:
upstream now covers them and we keep equivalent guards; the genuinely
prime-RL-specific piece is the compact ``routed_experts`` export. Also retarget
the stale ``OpenAIServingChat`` analog references to upstream ``ServingTokens``.

Comments/docstrings only — no code changes. (The ``usage`` field is still
genuinely dropped by upstream's response schema, and the ``kv_transfer_params``
note is left untouched as unverified.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hallerite
hallerite marked this pull request as ready for review June 25, 2026 04:40
@hallerite
hallerite merged commit 62702ba into main Jun 25, 2026
22 checks passed
@hallerite
hallerite deleted the chore/serving-tokens-doc-vllm-022 branch June 25, 2026 06:10
hallerite added a commit that referenced this pull request Jun 25, 2026
Brings in the 5 v1-cleanup PRs (#2872 filter fix, #2874 messages.py
delete, #2875 get_metrics removal, #2876 is_vlm removal, #2877
detection_index removal, #2878 serving_tokens docs) plus the verifiers
bump to 0.1.15.dev394.

Conflict resolutions:
- pyproject.toml: take main's verifiers floor (dev394 > dev390).
- deps/verifiers: take main's 20abcd91 (dev394) — strict descendant of
  our e306ba52 (dev390), already includes Trace.to_record().
- utils/client.py: keep our new prefill score() method, drop the dead
  get_metrics() that #2875 removed.
- filters.py / FilterResult: detection_index dropped (#2877); our
  branch-stream fix (#2872) preserved.

uv lock is a no-op; orchestrator unit suite green (99 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants