Repository navigation
[Fix] State the configured MoE-DP width in the weight-cache fingerprint - #41806
Merged
Merged
Conversation
This was referenced Sep 30, 2026
ch-wan
force-pushed
the
cheng/hot-fix/tensorcast-world-placement
branch
from
September 30, 2026 04:24
96e0a09 to
f573c0e
Compare
ch-wan
requested review from
Ying1123,
alphabetc1,
hanming-lu,
hnyls2002,
huangtingwei9988,
hzh0425,
ispobock,
merrymercy,
xiezhq-hermann and
yizhang2077
as code owners
September 30, 2026 04:24
ch-wan
force-pushed
the
cheng/hot-fix/weight-cache-moe-dp-fingerprint
branch
from
September 30, 2026 04:24
ea4c1ed to
29921f3
Compare
ch-wan
force-pushed
the
cheng/hot-fix/tensorcast-world-placement
branch
from
September 30, 2026 20:24
f573c0e to
eed1566
Compare
Base automatically changed from
cheng/hot-fix/tensorcast-world-placement
to
main
September 30, 2026 20:25
The engine filled `CacheConfig.moe_dp_size` with `get_moe_cp_size()`, the width of the MoE communicator, while the daemon fills it with the configured `moe_dp_size`. `CacheConfig.matches()` compares for equality, so whenever `attn_cp_size > moe_dp_size` -- where the MoE-DP group aliases the attention-CP group and is wider than configured -- every fetch failed with a config mismatch. Both sides now state the configured width. The ranks already agreed: both read `moe_dp_rank` from the published placement, which takes the attention-CP rank under the same aliasing. The weight-cache daemon e2e test gains an attention-CP case.
ch-wan
force-pushed
the
cheng/hot-fix/weight-cache-moe-dp-fingerprint
branch
from
September 30, 2026 20:25
29921f3 to
49b65a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of a stack (oldest at bottom):
Motivation
The engine fills
CacheConfig.moe_dp_sizewithget_moe_cp_size(), the width of the MoE communicator. The daemon fills it with the configuredmoe_dp_size.CacheConfig.matches()compares for equality. Wheneverattn_cp_size > moe_dp_size, the MoE-DP group aliases the attention-CP group and is wider than configured, so every fetch from the daemon fails withDaemon config mismatch!.Modifications
moe_dp_size, as the daemon does. The ranks already agree: both sides readmoe_dp_rankfrom the published placement, which takes the attention-CP rank under the same aliasing.test_weight_cache_daemon.pygainsTestWeightCacheDaemonQwen3MoeAttnCP: Qwen3-30B-A3B served through the daemon with--attn-cp-size 2 --enable-prefill-cp --cp-strategy zigzag. The 2-GPU suite'sest_timerises to cover it.Accuracy Tests
H200:
Config mismatch: {'moe_dp_size': (1, 2)}and the client fails to load. On this PR: 3 passed, about two minutes.--tp 2 --attn-cp-size 2 --enable-prefill-cp --cp-strategy zigzag --cuda-graph-backend-prefill=disabled: weights load over IPC, GSM8K (200 questions) 0.910.test/registered/unitat this PR's head, compared withmain: no new failures.Speed Tests and Profiling
Not applicable.
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): 🚫 Run #36772507415
Latest PR Test (Extra): 🚫 Run #36772507303
Latest PR Test (AMD ROCm 10): ❌ Run #36772507201