[Bugfix][Worker] Runtime compatibility for Qwen3.5 perf path#10
Closed
lHrHenry233 wants to merge 2 commits intomainfrom
Closed
[Bugfix][Worker] Runtime compatibility for Qwen3.5 perf path#10lHrHenry233 wants to merge 2 commits intomainfrom
lHrHenry233 wants to merge 2 commits intomainfrom
Conversation
- make draft proposer import optional to avoid startup hard failure when module is absent - remove stale max_num_blocks_per_req argument from NPUInputBatch call - cast recurrent state to bf16 before npu_recurrent_gated_delta_rule in qwen3_next patch This PR focuses on runtime stability and compatibility and intentionally excludes scheduler/config or kernel-level optimization changes. Signed-off-by: lHrHenry233 <lHrHenry233@users.noreply.github.com>
github-actions
bot
added
documentation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Follow-up adjustment after upstream API drift: NPUInputBatch now accepts max_num_blocks_per_req, so keep passing it to preserve block-limit semantics while retaining other runtime compatibility fixes in PR10. Signed-off-by: lHrHenry233 <lHrHenry233@users.noreply.github.com>
lHrHenry233
pushed a commit
that referenced
this pull request
Apr 10, 2026
…(v3.1) - Port upstream _causal_conv1d_fwd_kernel as NPU Triton kernel - Handles initial/final/intermediate conv state in-kernel - Supports APC block boundary state writes - NPU adaptations: removed .cache_modifier, kept debug_barrier - Rewrite causal_conv1d_fn to dispatch to new Triton kernel - Rewrite gdn.py conv1d path: split decode/prefill like upstream - Decode: causal_conv1d_update_npu with block params - Prefill: causal_conv1d_fn with APC params (new kernel) - Fix SSM #6: _build_initial_state only zeros prefill sequences - Fix SSM #7: _write_final_states adds slot >= 0 validation - Fix SSM #8: _scatter_intermediate_states adds unaligned offset - Update all 36 UTs to pass with new num_computed_tokens_all field Alignment status vs upstream #26807: #1 conv1d prefill kernel: FIXED (kernel ported) #3 causal_conv1d_fn params: FIXED (rewritten) #4 intermediate conv state: FIXED (kernel internal) #6 SSM zeroing scope: FIXED #7 _write_final_states guard: FIXED #8 SSM scatter alignment: FIXED #9 causal_conv1d_fn signature: FIXED #2 decode pre-copy: KEEP (NPU needs it) #5 SSM decode index: OK (correct approach) #10 conv layout hardcoded: DEFERRED Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
This PR is a runtime compatibility follow-up to keep the serving path stable with minimal review scope.
Changed files (2):
vllm_ascend/worker/model_runner_v1.pyvllm_ascend/patch/worker/patch_qwen3_next.pyMain changes
model_runner_v1.pyisinstance/assertto support both environments.max_num_blocks_per_reqinNPUInputBatchcall to match current interface.patch_qwen3_next.pystateis BF16 before callingnpu_recurrent_gated_delta_rule(both spec and non-spec paths) to avoid dtype runtime errors.Why this split
This PR only addresses runtime compatibility, without mixing scheduler semantics or kernel-level optimization changes.
Validation
python3 -m py_compile vllm_ascend/worker/model_runner_v1.py vllm_ascend/patch/worker/patch_qwen3_next.py