Skip to content

[Model] add GLM-5.3-Flash support - #53906

Merged
ywang96 merged 38 commits into
vllm-project:mainfrom
ZJY0516:glm-release
Sep 3, 2026
Merged

ywang96 merged 38 commits into
vllm-project:mainfrom
ZJY0516:glm-release

Conversation

@ZJY0516

@ZJY0516 ZJY0516 commented Aug 26, 2026

Copy link
Copy Markdown
Member

Purpose

@JaredforReal did most of the work for this day0 support 933876c, thanks!

add support for https://huggingface.co/zai-org/GLM-5.3-Flash

Please use docker image to run this model, see https://recipes.vllm.ai/zai-org/GLM-5.3-Flash

Only the first commit 933876c is fully verified

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Assisted-by: OpenAI Codex
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@yangqinghao-cmss

yangqinghao-cmss commented Sep 7, 2026

Copy link
Copy Markdown
image

This cannot be used. How can such a problem be resolved?

@drakosha

drakosha commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Same request on SM90 here (2x H200 NVL, TP2, no DP/EP, GLM-5.3-Flash NVFP4 on main):
你好 returns a normal greeting, finish_reason: stop, and prompt_tokens is 13,
the same count your run shows. So your chat template and tokenization are fine, the
prompt does reach the engine.

At temperature: 0 what you got is not a sampling artifact either: the model emits
memorized training text and then degenerates into one repeated phrase, which is what
a healthy LM does when the context has no influence on it. Suspect the attention and
state path or weight loading, not sampling.

Your fingerprint is tp4-dp2-ep, ours is plain tp2. Bisect one knob at a time and
stop at the first sane answer: drop EP, then DP, then MTP if enabled, then
--kv-cache-dtype auto. Worth posting with the result: the Using ... attention backend startup line, the kv_cache_dtype from the engine config dump, and which
commit d9105ea80 is. If the checkpoint is a third-party quantization, try
zai-org/GLM-5.3-Flash first, since fluent but context-blind output is also what a
bad quantized load looks like.

This PR is merged, so a new issue will get more eyes than a comment here.

AI assistance was used for this comment.

@gaby

gaby commented Sep 9, 2026

Copy link
Copy Markdown

@ZJY0516 @ywang96 Any idea why support for GLM-5.3-Flash was not added to the 0.29.0 release?

See #56007

@ZJY0516

ZJY0516 commented Sep 9, 2026

Copy link
Copy Markdown
Member Author

@ZJY0516 @ywang96 Any idea why support for GLM-5.3-Flash was not added to the 0.29.0 release?

See #56007

sorry about this, the branch cut is before merging model support PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build glm k3 kimi kv-cache-manager kv-connector mrv2 Model Runner V2 specific multi-modality Related to multi-modality (#4194) new-model Requests to new models nvidia quantization ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm scheduler speculative-decoding torch.compile

Projects

Status: Done
Status: Done
Status: Done

Development

Successfully merging this pull request may close these issues.