Repository navigation
fix(qwen3.8): patch GDN+MTP async wild-write crash on dual-fast (#1052) - #1061
Merged
Merged
Conversation
On a Qwen3-Next hybrid (GDN linear attention) with the MTP drafter + prefix caching + async scheduling (vLLM v0.27.1 default), the engine died with a CUDA illegal memory access in gdn_attn.py (Xid 31 MMU fault, VIRT_WRITE) within 6-13k generated tokens of agent-shaped traffic. Cause: gpu_model_runner.py::synchronize_input_prep waits prepare_inputs_event (recorded BEFORE the forward + spec-decode fused-align postprocess) but not num_accepted_tokens_event (recorded AFTER it), so under async scheduling the next step's _update_states block-table mutation raced the still-in-flight postprocess -> stale state-block index -> wild GPU write. Fix: also wait num_accepted_tokens_event at the top of synchronize_input_prep, closing the write-after-read window. It relocates a wait align-mode already did later, so TPS is neutral. Scoped to spec decode (event None otherwise) and async (the method is a no-op without prepare_inputs_event). Delivered as an idempotent, anchor-checked install_script patch (vllm-gdn-mtp-async-spec-order, shared from the qwen3.6-27b tree like pr48375), wired on vllm/qwen38-27b-dual-fast, registered in patches.yml. Ours, not an upstream PR (diagnosis + reproducer posted on vllm#50021 and vllm#52873). Validated on the reference 2x3090 (v0.27.1): async-ON survived 3/3 boots to 33-36k generated tokens each (crossing ctx 32k + a compaction) where 5/5 unpatched arms died 6-13k; real delivery path confirmed; bench neutral (79.1 narr / 108.5 code / 1756 prefill >= baseline). Does NOT address the separate acceptance-collapse bug (vllm#52873). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes the CUDA illegal-memory-access crash (Xid 31 MMU fault,
VIRT_WRITE) onvllm/qwen38-27b-dual-fastreported in #1052 — engine dies within 6–13k generated tokens of agent-shaped traffic when the MTP drafter + prefix caching + async scheduling are all on (all default on this compose under vLLM v0.27.1).Root cause
In
gpu_model_runner.py::synchronize_input_prep, the next step waitsprepare_inputs_event(recorded before the forward + spec-decode fused-align postprocess) but notnum_accepted_tokens_event(recorded after it). Under async scheduling the next step's_update_statesblock-table mutation raced the still-in-flight postprocess reading those buffers → stale state-block index → wild GPU write. Confirmed by isolation: drafter-off,--no-async-scheduling, andCUDA_LAUNCH_BLOCKING=1each avoid it → a cross-stream race, not an index bug.Fix
Two lines: also wait
num_accepted_tokens_eventat the top ofsynchronize_input_prep. It relocates a wait align-mode already performed later inside_prepare_inputs, so there is no net GPU stall. Scoped to spec decode (the event isNoneotherwise) and async (the method is a no-op withoutprepare_inputs_event).Shipped as an idempotent, anchor-checked
install_scriptpatch (vllm-gdn-mtp-async-spec-order, shared from the qwen3.6-27b tree likepr48375), wired onvllm/qwen38-27b-dual-fast, registered inpatches.yml. Anchor drift → boot refused.Attribution / scope
Wired on dual-fast only (the validated slug); the fix is engine-wide in
gpu_model_runner.py, so broader MTP-compose rollout is a follow-up decision.Validation (reference 2×3090, v0.27.1)
switch.sh→ compose mount → entrypoint logs[gdn-async-order] applied; 16k soak clean.scripts/testssweep green except a pre-existingtest-locale-utf8failure onmaster(concurrency-probePYTHONUTF8, [bug] half the script layer breaks on non-UTF-8 locales — 42/84 tests fail under LC_ALL=C #779 — unrelated).🤖 Generated with Claude Code