Repository navigation
[Qwen4Exp] Clean up HC and PLE implementations for Qwen3.8-Flash-Next - #61454
Merged
gau-nernst merged 18 commits intoOct 18, 2026
Merged
Conversation
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
peakcrosser7
requested review from
AndreasKaratzas,
DarkLight1337,
dllehr-amd,
gau-nernst,
hongxiayang,
tjtanaa and
ywang96
as code owners
October 13, 2026 16:34
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
gau-nernst
reviewed
Oct 14, 2026
6 tasks done
ZJY0516
reviewed
Oct 14, 2026
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
gau-nernst
reviewed
Oct 18, 2026
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
gau-nernst
approved these changes
Oct 18, 2026
Contributor
|
@peakcrosser7 Can you help fix DCO? I will run CI afterwards. Thanks! |
Contributor
Author
OK, I noticed the two revert commits didn't include a sign-off. I'll fix them. |
peakcrosser7
force-pushed
the
qwen4/cleanup_ple_hc
branch
from
October 18, 2026 06:21
b62243f to
661e235
Compare
Contributor
Author
DCO fixed. |
|
✅ Triggered Buildkite CI #95667 for commit |
kimseunghyun-kr
pushed a commit
to kimseunghyun-kr/vllm
that referenced
this pull request
Oct 18, 2026
Resolve the conflict in vllm/models/qwen4_exp/nvidia/ops/qsa_indexer.py: keep this branch's per-shard logits width for the sharded selector and use main's round_up (vllm-project#61454). Signed-off-by: kimseunghyun <hellonypcteam@gmail.com>
harjothkhara
added a commit
to harjothkhara/vllm
that referenced
this pull request
Oct 18, 2026
Resolve conflict in vllm/models/qwen4_exp/common/ngram_embedding.py: keep main's reworked Qwen4ExpPLEEmbeddingMethod.from_quant_config (vllm-project#61454, vllm-project#56273) and keep this PR's strip_model_root_prefix() call ahead of every prefix-based quant lookup, including the new NVFP4 path. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: harjoth <harjoth.khara@gmail.com>
michael-gpu
pushed a commit
to practical-inference/vllm-ampere-extended
that referenced
this pull request
Oct 20, 2026
…vllm-project#61454) Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com> Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
mjkvaak-amd
added a commit
to mjkvaak-amd/vllm-project
that referenced
this pull request
Oct 22, 2026
…spatch vllm-project#61454 renamed GatedResidual.lora_rank to hc_lowrank; the two-stage combine_and_mix dispatch still read the old attribute and failed at the first compiled forward on bases with vllm-project#61454. Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
mjkvaak-amd
added a commit
to mjkvaak-amd/vllm-project
that referenced
this pull request
Oct 22, 2026
…spatch vllm-project#61454 renamed GatedResidual.lora_rank to hc_lowrank; the two-stage combine_and_mix dispatch still read the old attribute and failed at the first compiled forward on bases with vllm-project#61454. Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Closed
5 of 6 tasks
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Clean up Qwen4Exp HC/PLE implementations and refine PLE quantization selection, with regression coverage for the affected paths.
Claims
PLEVocabParallelEmbedding, using the embedding's own TP shard metadata.lora_ranktohc_lowrankand localloravariables tolowrankon NVIDIA and AMD. Use clearer names for token counts and tensor dimensions in HC and PLE launch functions.round_upfor PLE vocabulary padding, QSA RoPE-position alignment, QSA ring capacity, and prefill logits alignment.Fp8Configusage.Validation
Correctness tests:
Result: 123 passed, 2 skipped. The skipped cases require ROCm. Coverage includes HC and PLE kernel correctness, checkpoint shard loading, FP8 handling, pinned-host lookup and prefetch, and quantization selection.
Static checks: Ruff lint and formatting checks passed for all 12 changed files.
git diff --checkalso passed.Serving smoke validation: Qwen3.8-Flash-Next started and served HTTP 200 requests with BF16, TP2/DP1, CPU-offloaded PLE, and MTP with 3 speculative tokens. The run recorded nonzero MTP acceptance.
CI coverage: Existing CPU CI covers PLE quantization selection and shard-loading tests. The H200 job covers HC operations and PLE convolution correctness.
End-to-end model accuracy evaluations and before/after performance benchmarks were not run. ROCm hardware execution and quantized-checkpoint serving validation remain pending.
Details
The unquantized PLE fallback assumes the checkpoint contains an unquantized embedding table. NVFP4 PLE decoding and format conversion remain outside this change.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.