[karverma-amd]

Motivation

On the per-rank fused shared-slot path the shared expert is appended after the
gate, by fused_append_remap_shared_experts_deepep. But select_experts also
passes num_fused_shared_experts down into the gate, so both of them emit a
shared marker.

Two things go wrong, neither of which raises:

A routed expert is lost. The gate is asked for
K_routed = top_k - num_fused_shared_experts slots and then spends one of them
on its own marker, so a top-6 routing silently becomes top-5.

An id lands out of range. The gate's marker is written at id num_experts,
and the DeepEP per-rank remap then shifts it one past the end of the expert
space — 384 -> 392 for 384 routed experts on EP8, where the valid ids are
0..391.

Modification

Pass 0 to the gate when has_per_rank_fused_shared_slots() holds, so the
append is the single producer of the shared slot. Every other path keeps the
existing value, so this is a no-op off the per-rank shared-slot path.

14 lines in one file.

Accuracy

DeepSeek-V4-Pro, MI355X (gfx950), TP8 + DP8 attention + EP8/MoRI with
--enforce-shared-experts-fusion, which is the regime where the per-rank
shared-slot path is live. The arms differ only by this fix; each server asserts
which side it is on before measuring.

GSM8K, 1319 questions, parallel 1319:

arm	accuracy	invalid
without fix	0.945	0.000
with fix	0.948	0.000

I would not read that gap as the effect. Four questions out of 1319 is inside
run-to-run noise for this benchmark, so the honest summary is that GSM8K cannot
resolve this bug either way — it is reported to show the fix does not regress,
not to claim it wins.

The case for the fix is the two defects themselves: a top-6 routing that is
actually running top-5, and an expert id written one past the end of the id
space. Both are silent, and neither is something a routing path should be doing
regardless of what an end-to-end score says.

Commands

SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton \
SGLANG_OPT_FUSE_COMPRESS_NORM_ROPE=1 \
SGLANG_OPT_NATIVE_BPRESHUFFLE_SCALE=1 \
SGLANG_OPT_USE_AITER_BATCHED_GEMM=1 \
AITER_BF16_FP8_MOE_BOUND=0 SGLANG_USE_AITER=1 \
SGLANG_SHARED_EXPERT_TP1=1 SGLANG_DP_SHARED_EXPERT_LOCAL=1 \
SGLANG_DP_USE_GATHERV=1 SGLANG_DP_USE_REDUCE_SCATTER=1 \
GPU_MAX_HW_QUEUES=5 MORI_SHMEM_MODE=ISOLATION MORI_SHMEM_HEAP_SIZE=8G \
MORI_EP_LAUNCH_CONFIG_MODE=AUTO \
python3 -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 --dp 8 --enable-dp-attention \
  --ep-size 8 --moe-a2a-backend mori --deepep-mode normal \
  --moe-dense-tp-size 1 --enable-dp-lm-head \
  --attention-backend dsv4 --page-size 256 --kv-cache-dtype fp8_e4m3 \
  --enforce-shared-experts-fusion \
  --context-length 16384 --chunked-prefill-size 8192 \
  --mem-fraction-static 0.78 \
  --max-running-requests 64 --cuda-graph-max-bs 64

python3 benchmark/gsm8k/bench_sglang.py --num-questions 1319 --parallel 1319

Checklist

• Accuracy measured on MI355X, 1319 questions, one arm per side of the fix
• Each arm asserted which side of the fix it was on before measuring
• No-op off the per-rank fused shared-slot path
• Single file, no new configuration

---

CI States

Latest PR Test (Base): ✅ Run #33272334686
Latest PR Test (Extra): ❌ Run #33272334475
Latest PR Test (AMD ROCm 7.2): ❌ Run #33272334891

[HaiShaw]

/tag-and-rerun-ci