[FIX_FOR_VLLM_CUSTOM=1c3633acafd6bbde1f3636cee9799e3ab0879d13] Restore SharedExperts stream-sync in dp1 MoE fast path (+1 more) - #1732
Merged
iboiko-habana merged 1 commit intoAug 13, 2026
Conversation
…e SharedExperts stream-sync in dp1 MoE fast path (vllm#52024) Root cause: vllm PR #52024 reverted the shared-experts dual-stream decode work, removing SharedExperts.maybe_forward_async and the MoERunner._apply_quant_method shared_experts_overlapping kwarg. The HPU dp_size==1 fast path in patched_fused_moe_forward still called the removed API, raising AttributeError: 'SharedExperts' object has no attribute 'maybe_forward_async'. Upstream: vllm-project/vllm#52024 Fix: mirror the reverted MoERunner._forward_impl - call the runner's _maybe_sync_shared_experts_stream and drop the removed shared_experts_overlapping kwarg from the _apply_quant_method call. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
requested review from
PatrykWo,
adobrzyn,
afierka-intel,
iboiko-habana,
jbyczkow,
mgawarkiewicz-intel,
michalkuligowski and
xuechendi
as code owners
August 13, 2026 13:33
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Gaudi MoE dp_size==1 fast path in patched_fused_moe_forward to match the upstream vLLM MoERunner behavior after the referenced upstream revert, primarily by restoring runner-level shared-expert stream sync handling and removing an no-longer-supported _apply_quant_method kwarg.
Changes:
- Restore the dp1 fast-path shared-experts stream sync step via
self._maybe_sync_shared_experts_stream(shared_experts_input). - Remove forwarding of the
shared_experts_overlappingargument intoself._apply_quant_method(...)to match the updated upstream signature.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
✅ CI PassedAll checks passed successfully against the following vllm commit: |
iboiko-habana
approved these changes
Aug 13, 2026
iboiko-habana
pushed a commit
that referenced
this pull request
Aug 31, 2026
…e-detect MoE shared-experts overlap API (#1773) This PR consolidates 1 hourly-CI fix against vllm@`488b6da105222f4f8130b3aae4a0e67cfc61f522`. ## Bug 1: Feature-detect MoE shared-experts overlap API - **State machine id**: moerunner_dual_stream_relanded_sync_attr_missing - **Commit**: 326038e ### Root cause vllm#52033 (2c7d7dd64a) re-landed the dual-stream shared-experts decode work that vllm#52024 had reverted, deleting MoERunner._maybe_sync_shared_experts_stream and re-introducing SharedExperts.maybe_forward_async plus the shared_experts_overlapping kwarg on MoERunner._apply_quant_method. patched_fused_moe_forward in vllm_gaudi/ops/hpu_fused_moe.py still called the deleted method, so every MoE forward raised AttributeError: 'MoERunner' object has no attribute '_maybe_sync_shared_experts_stream' and 19 of 59 jobs went red in CI run 33314657602 (run_unit_tests, run_pd_disaggregate_test and 17 MoE e2e legs). The preceding run 33291778445 was green at the same vllm-gaudi commit, so the break is purely upstream-side. This is the third flip of the same upstream API: vllm#51838 introduced it (adapted in #1723), vllm#52024 reverted it (adapted in #1732), and vllm#52033 has now re-landed it. ### Culprit Regression introduced by [PR #52033](vllm-project/vllm#52033). ### Fix route the aux-stream launch through _launch_shared_experts_overlap, which mirrors upstream MoERunner._forward_impl: it feature-detects _maybe_sync_shared_experts_stream and SharedExperts.maybe_forward_async, and only forwards shared_experts_overlapping when an async launch actually happened. maybe_forward_async gates on current_platform.is_cuda_alike(), so on Gaudi it returns False, the kwarg is omitted, and the path stays correct on both API shapes - a fourth flip in either direction will not re-break the hourly suite. Verified on a fresh Gaudi3 four-card pod at the pinned vLLM SHA: the two MoE unit tests CI reported red fail at baseline with the same AttributeError and pass with the fix, and the full run_unit_tests suite shows no MoE-related failures. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR consolidates 2 fixes against vllm@
1c3633acafd6bbde1f3636cee9799e3ab0879d13. Both symptoms are the same ~10-line shared-experts fast-path block and are co-resolved by one commit (697d798).Bug 1: Restore SharedExperts stream-sync in dp1 MoE fast path
Root cause
Upstream vLLM PR #52024 reverted PR #51838, restoring the shared-experts stream-synchronization path in the dp1 MoE fast path. The plugin MoERunner override had adapted to the #51838 shape and forwarded a shared_experts_overlapping keyword into MoERunner._apply_quant_method(); after the revert that parameter no longer exists upstream, so every MoE job raised TypeError: _apply_quant_method() got an unexpected keyword argument 'shared_experts_overlapping'.
Upstream PR
vllm-project/vllm#52024
Revert of vllm#51838, restoring the shared-experts stream-sync fast path.
Fix
Restore the shared-experts stream-sync block in the dp1 MoE fast path so the override matches post-revert upstream and stops forwarding the removed shared_experts_overlapping kwarg into MoERunner._apply_quant_method(). Single ~10-line block; both symptoms are co-resolved by this one commit.
Bug 2: Restore SharedExperts.maybe_forward_async in dp1 MoE fast path
Root cause
Same upstream revert (vLLM PR #52024 reverting #51838). Post-revert, the shared-experts fast path again dispatches through SharedExperts.maybe_forward_async, but the plugin override had dropped that call, so MLA/MoE jobs raised AttributeError: 'SharedExperts' object has no attribute 'maybe_forward_async'.
Upstream PR
vllm-project/vllm#52024
Revert of vllm#51838, restoring the shared-experts stream-sync fast path.
Fix
Same commit: the restored fast-path block calls SharedExperts.maybe_forward_async again, matching post-revert upstream and clearing the AttributeError across MLA/MoE/PD jobs.