Conversation
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This stacked follow-up adds Model Runner V2 routed-expert capture support on top of
xuanyu/enable-fi-monolithic-moe-rrfrom vllm-project#44214.Before this change, Model Runner V2 only bound
BaseRoutercapture callbacks. Monolithic MoE kernels perform routing inside the fused expert kernel, so supported monolithic kernels from vllm-project#44214 never received the routed-expert capture callback and R3 results could be missing or all zero.This change:
BaseRoutercapture path;fused_expertsimplementation through the V2 MoE quantization path;supports_routing_replay_capture()is true;This PR is intentionally stacked on both vllm-project#38163 (Model Runner V2 expert-index capture) and vllm-project#44214 (FlashInfer monolithic replay support). The requested base branch is
xuanyu/enable-fi-monolithic-moe-rrinxuanyu-mistral/vllm.Validation
git diff --checkpasses.vllm/vllm-openai:nightlycontainer.AI assistance was used. A human submitter should review every changed line and verify the stacked dependencies before merging.