Skip to content

[Bugfix] Fix local speculators with dots in the name from classifying as custom_class - #48754

Merged
simon-mo merged 1 commit into
vllm-project:mainfrom
neuralmagic:fix-local-non-custom-proposer
Jul 16, 2026
Merged

simon-mo merged 1 commit into
vllm-project:mainfrom
neuralmagic:fix-local-non-custom-proposer

Conversation

@mgoin

@mgoin mgoin commented Jul 15, 2026

Copy link
Copy Markdown
Member

Purpose

Fix speculative config so draft model names with dots (e.g. local GLM-5.2-speculator.dspark folder) are not misclassified as custom_class import paths.

Test Plan

Test Result

Before:

vllm serve zai-org/GLM-5.2-FP8 --max-model-len 20k --spec-model GLM-5.2-speculator.dspark --spec-method dspark --spec-tokens 7 -tp 4
...
ImportError: Cannot import module 'GLM-5.2-speculator' for custom proposer 'GLM-5.2-speculator.dspark': No module named 'GLM-5'

After: loads normally, same as using RedHatAI/GLM-5.2-speculator.dspark for the model


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

… as custom_class

Signed-off-by: mgoin <mgoin64@gmail.com>
@mgoin
mgoin requested a review from WoosukKwon as a code owner July 15, 2026 16:35
@mgoin mgoin added the bug Something isn't working label Jul 15, 2026
@mgoin
mgoin requested a review from ProExpertProg as a code owner July 15, 2026 16:35
@mgoin mgoin added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 15, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify

mergify Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Hi @mgoin, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

@yewentao256 yewentao256 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work!

@simon-mo
simon-mo merged commit 3a5e88e into vllm-project:main Jul 16, 2026
92 of 96 checks passed
@mgoin
mgoin deleted the fix-local-non-custom-proposer branch July 16, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed speculative-decoding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants