[Bugfix][Worker] Unblock Qwen3.5-9B align/all perf validation#7
Closed
lHrHenry233 wants to merge 2 commits intomainfrom
Closed
[Bugfix][Worker] Unblock Qwen3.5-9B align/all perf validation#7lHrHenry233 wants to merge 2 commits intomainfrom
lHrHenry233 wants to merge 2 commits intomainfrom
Conversation
- support mamba all-mode across config/scheduler/model runner and qwen3.5 patch flow\n- extend causal_conv1d host/kernel to handle 2D token-level cache indices\n- add UT/e2e coverage for prefill/decode all-mode paths and token snapshot writeback Signed-off-by: lHrHenry233 <lHrHenry233@users.noreply.github.com>
- make draft proposer import optional to avoid hard dependency when MTP draft module is absent - remove stale NPUInputBatch argument max_num_blocks_per_req for branch compatibility - cast recurrent state to bf16 before npu_recurrent_gated_delta_rule to satisfy NPU op dtype constraints These fixes are required to keep Qwen3.5-9B prefix-caching perf runs stable and comparable between mamba_cache_mode=align and all. Signed-off-by: lHrHenry233 <lHrHenry233@users.noreply.github.com>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
lHrHenry233
pushed a commit
that referenced
this pull request
Apr 10, 2026
…(v3.1) - Port upstream _causal_conv1d_fwd_kernel as NPU Triton kernel - Handles initial/final/intermediate conv state in-kernel - Supports APC block boundary state writes - NPU adaptations: removed .cache_modifier, kept debug_barrier - Rewrite causal_conv1d_fn to dispatch to new Triton kernel - Rewrite gdn.py conv1d path: split decode/prefill like upstream - Decode: causal_conv1d_update_npu with block params - Prefill: causal_conv1d_fn with APC params (new kernel) - Fix SSM #6: _build_initial_state only zeros prefill sequences - Fix SSM #7: _write_final_states adds slot >= 0 validation - Fix SSM #8: _scatter_intermediate_states adds unaligned offset - Update all 36 UTs to pass with new num_computed_tokens_all field Alignment status vs upstream #26807: #1 conv1d prefill kernel: FIXED (kernel ported) #3 causal_conv1d_fn params: FIXED (rewritten) #4 intermediate conv state: FIXED (kernel internal) #6 SSM zeroing scope: FIXED #7 _write_final_states guard: FIXED #8 SSM scatter alignment: FIXED #9 causal_conv1d_fn signature: FIXED #2 decode pre-copy: KEEP (NPU needs it) #5 SSM decode index: OK (correct approach) #10 conv layout hardcoded: DEFERRED Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
This PR addresses two runtime blockers found while validating Qwen3.5 Mamba prefix-caching performance under
mamba_cache_mode=alignvsallon single-card Ascend (32G HBM).Background and motivation
During end-to-end perf runs on
Qwen/Qwen3.5-9B, the service hit two production-path failures:stateas FP32) caused runtime 500 intorch_npu.npu_recurrent_gated_delta_rule, which expects BF16 in this path.Additionally, there was one constructor compatibility mismatch in this branch:
NPUInputBatch.__init__()does not acceptmax_num_blocks_per_req, but caller still passed it.These issues blocked stable apples-to-apples perf comparison of align/all mode, so this PR focuses on minimal, targeted fixes.
Code changes
1) Make draft proposer import optional (decouple from mandatory MTP draft module)
vllm_ascend/worker/model_runner_v1.pyAscendDraftModelProposerimport intry/except ModuleNotFoundError.draft_proposer_typestuple dynamically inisinstance/assertchecks.2) Remove stale constructor arg for branch compatibility
vllm_ascend/worker/model_runner_v1.pymax_num_blocks_per_req=max_num_blocksfromNPUInputBatch(...)call.NPUInputBatch.__init__signature in this branch does not accept that argument.TypeErrorand restore request execution path.3) Enforce BF16 recurrent state for NPU gated delta rule
vllm_ascend/patch/worker/patch_qwen3_next.pytorch_npu.npu_recurrent_gated_delta_rule, castssm_stateto BF16 when needed in both spec and non-spec decode branches.Does this PR introduce any user-facing change?
How was this patch tested?
Local verification
python3 -m py_compile vllm_ascend/worker/model_runner_v1.py vllm_ascend/patch/worker/patch_qwen3_next.pyRuntime validation (Ascend NPU)
Qwen/Qwen3.5-9B--enable-prefix-caching --enforce-eager--mamba-cache-mode align--mamba-cache-mode all/v1/completionssuccessfully after fixes.allfaster thanalignin this environment.Risk assessment
Scope is narrow and targeted to error paths uncovered in real runtime.
No architectural behavior changes beyond compatibility and dtype safety.
vLLM version: v0.16.0
vLLM main: vllm-project/vllm@4034c3d