Skip to content

[Bugifx] Missing Renderer for fastokens mode#41984

Merged
vllm-bot merged 3 commits intovllm-project:mainfrom
EmbeddedLLM:bugfix-fastokens
May 8, 2026
Merged

[Bugifx] Missing Renderer for fastokens mode#41984
vllm-bot merged 3 commits intovllm-project:mainfrom
EmbeddedLLM:bugfix-fastokens

Conversation

@tjtanaa
Copy link
Copy Markdown
Collaborator

@tjtanaa tjtanaa commented May 7, 2026

Purpose

Fix error of #41741

(APIServer pid=8195)   File "/app/vllmaitercheck/vllmnew/vllm/renderers/registry.py", line 87, in renderer_from_config
(APIServer pid=8195)     return RENDERER_REGISTRY.load_renderer(renderer_mode, config, tokenizer)
(APIServer pid=8195)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=8195)   File "/app/vllmaitercheck/vllmnew/vllm/renderers/registry.py", line 68, in load_renderer
(APIServer pid=8195)     renderer_cls = self.load_renderer_cls(renderer_mode)
(APIServer pid=8195)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=8195)   File "/app/vllmaitercheck/vllmnew/vllm/renderers/registry.py", line 55, in load_renderer_cls
(APIServer pid=8195)     raise ValueError(f"No renderer registered for {renderer_mode=!r}.")
(APIServer pid=8195) ValueError: No renderer registered for renderer_mode='fastokens'.

Test Plan

Evaluate Qwen3-8B

#!/bin/bash

vllm serve Qwen/Qwen3-8B --tokenizer-mode fastokens \
--port 8001 \
2>&1 | tee -a serverqwen38bfastokens.log
#!/bin/bash

MODEL=Qwen/Qwen3-8B 
lm_eval --model local-completions --model_args model=$MODEL,base_url=http://0.0.0.0:8001/v1/completions,num_concurrent=100,max_retries=10,max_gen_toks=2048,timeout=60000,trust_remote_code=True --batch_size auto --tasks gsm8k --num_fewshot 5 --output_path . 2>&1 | tee -a eval.log

Test Result

Qwen3-8B

local-completions ({'model': 'Qwen/Qwen3-8B', 'base_url': 'http://0.0.0.0:8001/v1/completions', 'num_concurrent': 100, 'max_retries': 10, 'max_gen_toks': 2048, 'timeout': 60000}), gen_kwargs: ({}), limit: None, num_fewshot: 5, batch_size: auto
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|_  |0.8969|_  |0.0084|
|     |       |strict-match    |     5|exact_match|_  |0.8976|_  |0.0083|

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds "fastokens" to the renderer registry in vllm/renderers/registry.py, mapping it to the HfRenderer. There are no review comments to address, and I have no feedback to provide.

@tjtanaa tjtanaa marked this pull request as ready for review May 7, 2026 16:56
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@tjtanaa tjtanaa changed the title [Bugifx] Missing Rendere for fastokens mode [Bugifx] Missing Renderer for fastokens mode May 7, 2026
Comment thread vllm/renderers/registry.py Outdated
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) May 8, 2026 02:47
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label May 8, 2026
@vllm-bot vllm-bot merged commit 445d747 into vllm-project:main May 8, 2026
51 of 54 checks passed
libinta pushed a commit to libinta/vllm that referenced this pull request May 8, 2026
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Signed-off-by: Libin Tang <libin.tang@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants