[qianlihuang]

Overview

Archive the H200 SM90 NVFP4 experiment using flashinfer.megamoe, the legacy Torch-symmetric-memory API. Held experiment; no upstream submission planned. Prefer evaluating FlashInfer #6999's unified flashinfer.moe_ep.MoEEpLayer implementation when it is ready.

This snapshot temporarily reuses --moe-backend flashinfer_moe_ep_cutedsl to select the legacy adapter. It does not integrate vllm-project#6999. The unvalidated independent-backend rename is not included.

Claims

• Opt-in NVFP4-weight / FP8-activation MegaMoE on EP4/EP8 H200, with BF16 MTP experts retaining Triton + AG/RS. The qualified kernel family is fused.
• Same-NVFP4 decode A/B/A, C128: +6.72% output throughput, -18.16% TPOT p50, with +26.81% TTFT p50. These are medians of per-run metrics, six baseline runs versus three candidate runs.
• Prefill A/B only: 21.13 → 22.28 req/s (+5.45%). The final baseline arm was stopped at the user's request. No complete prefill A/B/A or accuracy-parity claim.

Validation

8× H200 on one node for decode; GPUs 0–3 for prefill. Same nvidia/GLM-5.3-Flash-NVFP4 checkpoint, revision da920bb0b9f4a06727223a349e55468e38352348, in both arms. Baseline: HUMMING + allgather/reducescatter. Candidate: FI legacy MegaMoE + internal dispatch/combine, with BF16 MTP still using AG/RS.

Published snapshot ff86daa9a4b9b932a0e636c5e8894b32ad3ada0d. Source base 18343393b4da41a7e8027efbc4c7e77c28301bf0; native wheel 4c12fce79d33abe2cfcf56c7cbb0c5eccb9c276e (CUDA/Rust source unchanged between these revisions); FlashInfer source 47a51d76d023dbf145ac3b7111e783e015a19033. Torch 2.14.0+cu130. The experiment base branch is pinned because the personal fork's main is older.

Decode: TP1/DP8/EP8, MRV2, default CUDA graphs, DecodeBenchConnector, ISL131072/OSL512, MTP3 synthetic unconditional acceptance [0.8,0.6,0.5], C8/32/64/128/192, N=2C, seed0. One warm-up and three timed passes per arm, serial A/B/A. All45 timed runs completed with zero failed requests and exact token counts; finite512-logprob gates passed. Profiles are separate from the timing runs. ReplaySSM is omitted in both arms because the NVIDIA conditional wrapper is not on main's allowlist.

C128 overall metric	Baseline, median [range], n=6	FI, median [range], n=3
Output tok/s	7040.49 [6586.89, 7257.96]	7513.38 [7378.31, 7564.19]
TPOT p50, ms	13.87 [13.56, 14.29]	11.35 [11.33, 11.63]
TTFT p50, ms	1440.10 [1279.81, 1745.73]	1826.18 [1810.18, 1984.16]

Five-second samples during all 45 timing windows show eight GPU contexts, with their PIDs matching each arm's server log, and SM clocks of 1980 MHz. These are point samples, not proof of continuous exclusivity. Earlier co-tenant-affected node18 performance runs are excluded.

Prefill: TP2/DP2/EP4, natural MTP3, MRV2/default graphs, ISL2048/OSL1. Warm-up 64 requests; three 1000-request unlimited-rate runs (seeds 102/104/106); Poisson 11/15/19/21/23/25RPS. A and B completed; the return-to-baseline arm was interrupted during startup. This is preliminary A/B evidence only.

Profile: DP0 decode contains 29 baseline versus 31 FI MTP iterations. Per iteration, 42 pairs of target Humming GEMMs become 42 FI fused launches. AG/RS pairs fall 45→3, preserving the BF16 MTP path. Cumulative kernel-time sums are not wall-clock savings.

Correctness: the measured source passed 28 model tests and 3 distributed tests covering EP4/EP8, TP2+SP/DP2/EP4, ragged/idle ranks, and graph replay at 16/257 rows. The published cleanup adds a constructor non-None assertion, equivalent test API aliases, and help/docs wording; no arithmetic change. These final cleanup files were not rerun on GPUs after the experiment was held. Pre-commit passed on the published snapshot. Final VLLM_GPU_SYNC_CHECK=error serving validation was queued and canceled before execution.

Operator and regression test commands

.venv/bin/python -m pytest -q -s tests/models/test_deepseek_v4_fi_moe_ep.py
# 28 passed
.venv/bin/python -m pytest -q -s tests/kernels/moe/test_flashinfer_moe_ep.py -k sm90_nvfp4
# 3 passed; EP4, EP8 and TP2/DP2/EP4

Recorded RED cases before fixes included BF16 MTP selecting pass-through stages, idle-rank buffer allocation rejecting M=0, and explicit ubatch_size=2 being accepted. Representative output:

assert isinstance(stages, MoEPrepareAndFinalizeNaiveDPEPModular)
E AssertionError: assert False
# stages was PassThroughPrepareAndFinalize

assert M_full > 0 and M_chunk > 0
E AssertionError

Stable FlashInfer lacks flashinfer.megamoe; the hardware tests skip when this API is absent. Manual H200 runs used the source revision above. No upstream CI result is claimed.

Server and Rust client commands

Both arms use identical flags; omit the final --moe-backend option for the baseline. The backend choice also binds internal passthrough; do not add an A2A CLI option.

MODEL=/gpfs/yiliu/models/nvidia-GLM-5.3-Flash-NVFP4-da920bb0
export VLLM_USE_V2_MODEL_RUNNER=1

# Decode
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 vllm serve "$MODEL" \
  --served-model-name glm53flash --host 127.0.0.1 --port 8264 \
  --tensor-parallel-size 1 --data-parallel-size 8 --enable-expert-parallel \
  --max-num-seqs 64 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.9 \
  --no-enable-prefix-caching --limit-mm-per-prompt '{"image":0,"video":0}' \
  --kv-transfer-config '{"kv_connector":"DecodeBenchConnector","kv_role":"kv_both"}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"rejection_sample_method":"synthetic","synthetic_acceptance_rates":[0.8,0.6,0.5]}' \
  --moe-backend flashinfer_moe_ep_cutedsl

C=128
vllm-rs bench serve --backend vllm --base-url http://127.0.0.1:8264 \
  --model "$MODEL" --served-model-name glm53flash --tokenizer "$MODEL" \
  --dataset-name random --random-input-len 131072 --random-output-len 512 \
  --prompt-token-ids --ignore-eos --request-rate inf --max-concurrency "$C" \
  --num-prompts "$((2 * C))" --seed 0 --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,99 --disable-tqdm --save-result

# Prefill / natural-MTP quality server
CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve "$MODEL" \
  --served-model-name glm53flash --host 127.0.0.1 --port 8264 \
  --tensor-parallel-size 2 --data-parallel-size 2 --enable-expert-parallel \
  --limit-mm-per-prompt '{"image":0,"video":0}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --moe-backend flashinfer_moe_ep_cutedsl

# warm-up: --num-prompts 64 --seed 100; timed seeds: 102,104,106
vllm-rs bench serve --backend vllm --base-url http://127.0.0.1:8264 \
  --model "$MODEL" --served-model-name glm53flash --tokenizer "$MODEL" \
  --dataset-name random --random-input-len 2048 --random-output-len 1 \
  --prompt-token-ids --ignore-eos --request-rate inf --num-prompts 1000 --seed 102 \
  --percentile-metrics ttft,e2el --metric-percentiles 50,90,99 --disable-tqdm --save-result

# Unmodified repository evaluator; DEP8 quality uses natural MTP3 without DecodeBenchConnector
python3 tests/evals/gsm8k/gsm8k_eval.py --port 8264

Raw Rust C128 outputs: all nine timed runs (overall metrics)

These excerpts stop before ITL and the distinct steady-state subsection. Warm-up/profile runs are excluded.

0-base, pass0

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  18.35     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              13.95     
Output token throughput (tok/s):         7142.76   
Peak output token throughput (tok/s):    3553.00   
Peak concurrent requests:                198.00    
Total token throughput (tok/s):          1835689.34
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1581.07   
Median TTFT (ms):                        1354.78   
P50 TTFT (ms):                           1354.78   
P90 TTFT (ms):                           2807.78   
P99 TTFT (ms):                           2930.79   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          13.51     
Median TPOT (ms):                        13.89     
P50 TPOT (ms):                           13.89     
P90 TPOT (ms):                           16.26     
P99 TPOT (ms):                           17.20

0-base, pass1

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  18.71     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              13.68     
Output token throughput (tok/s):         7006.14   
Peak output token throughput (tok/s):    3200.00   
Peak concurrent requests:                186.00    
Total token throughput (tok/s):          1800578.03
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1437.49   
Median TTFT (ms):                        1279.81   
P50 TTFT (ms):                           1279.81   
P90 TTFT (ms):                           2410.71   
P99 TTFT (ms):                           2615.15   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          14.32     
Median TPOT (ms):                        14.29     
P50 TPOT (ms):                           14.29     
P90 TPOT (ms):                           16.09     
P99 TPOT (ms):                           16.89

0-base, pass2

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  18.53     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              13.82     
Output token throughput (tok/s):         7074.84   
Peak output token throughput (tok/s):    3545.00   
Peak concurrent requests:                179.00    
Total token throughput (tok/s):          1818233.49
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1634.56   
Median TTFT (ms):                        1525.43   
P50 TTFT (ms):                           1525.43   
P90 TTFT (ms):                           2654.42   
P99 TTFT (ms):                           3180.66   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          13.76     
Median TPOT (ms):                        13.86     
P50 TPOT (ms):                           13.86     
P90 TPOT (ms):                           15.99     
P99 TPOT (ms):                           17.37

1-mega, pass0

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  17.76     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              14.41     
Output token throughput (tok/s):         7378.31   
Peak output token throughput (tok/s):    3839.00   
Peak concurrent requests:                189.00    
Total token throughput (tok/s):          1896225.98
--------------------Time to First Token---------------------
Mean TTFT (ms):                          2018.56   
Median TTFT (ms):                        1984.16   
P50 TTFT (ms):                           1984.16   
P90 TTFT (ms):                           3326.88   
P99 TTFT (ms):                           3747.33   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          11.77     
Median TPOT (ms):                        11.63     
P50 TPOT (ms):                           11.63     
P90 TPOT (ms):                           14.08     
P99 TPOT (ms):                           17.00

1-mega, pass1

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  17.33     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              14.77     
Output token throughput (tok/s):         7564.19   
Peak output token throughput (tok/s):    3970.00   
Peak concurrent requests:                186.00    
Total token throughput (tok/s):          1943997.20
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1991.21   
Median TTFT (ms):                        1826.18   
P50 TTFT (ms):                           1826.18   
P90 TTFT (ms):                           3303.88   
P99 TTFT (ms):                           3586.66   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          11.41     
Median TPOT (ms):                        11.35     
P50 TPOT (ms):                           11.35     
P90 TPOT (ms):                           13.75     
P99 TPOT (ms):                           15.06

1-mega, pass2

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  17.45     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              14.67     
Output token throughput (tok/s):         7513.38   
Peak output token throughput (tok/s):    3969.00   
Peak concurrent requests:                194.00    
Total token throughput (tok/s):          1930939.82
--------------------Time to First Token---------------------
Mean TTFT (ms):                          2019.70   
Median TTFT (ms):                        1810.18   
P50 TTFT (ms):                           1810.18   
P90 TTFT (ms):                           3269.83   
P99 TTFT (ms):                           4130.59   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          11.56     
Median TPOT (ms):                        11.33     
P50 TPOT (ms):                           11.33     
P90 TPOT (ms):                           13.73     
P99 TPOT (ms):                           15.47

2-base, pass0

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  18.88     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              13.56     
Output token throughput (tok/s):         6942.83   
Peak output token throughput (tok/s):    3561.00   
Peak concurrent requests:                183.00    
Total token throughput (tok/s):          1784306.24
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1672.28   
Median TTFT (ms):                        1636.80   
P50 TTFT (ms):                           1636.80   
P90 TTFT (ms):                           2444.24   
P99 TTFT (ms):                           2955.30   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          13.66     
Median TPOT (ms):                        13.56     
P50 TPOT (ms):                           13.56     
P90 TPOT (ms):                           16.03     
P99 TPOT (ms):                           17.13

2-base, pass1

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  19.90     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              12.87     
Output token throughput (tok/s):         6586.89   
Peak output token throughput (tok/s):    3177.00   
Peak concurrent requests:                191.00    
Total token throughput (tok/s):          1692830.52
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1783.29   
Median TTFT (ms):                        1745.73   
P50 TTFT (ms):                           1745.73   
P90 TTFT (ms):                           2842.74   
P99 TTFT (ms):                           3109.36   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          14.15     
Median TPOT (ms):                        13.99     
P50 TPOT (ms):                           13.99     
P90 TPOT (ms):                           16.23     
P99 TPOT (ms):                           17.84

2-base, pass2

================= Serving Benchmark Result =================
Successful requests:                     256       
Failed requests:                         0         
Maximum request concurrency:             128       
Benchmark duration (s):                  18.06     
Total input tokens:                      33554432  
Total generated tokens:                  131072    
Request throughput (req/s):              14.18     
Output token throughput (tok/s):         7257.96   
Peak output token throughput (tok/s):    3568.00   
Peak concurrent requests:                198.00    
Total token throughput (tok/s):          1865294.75
--------------------Time to First Token---------------------
Mean TTFT (ms):                          1464.25   
Median TTFT (ms):                        1299.46   
P50 TTFT (ms):                           1299.46   
P90 TTFT (ms):                           2542.55   
P99 TTFT (ms):                           2816.98   
----------Time per Output Token (excl. 1st token)-----------
Mean TPOT (ms):                          13.51     
Median TPOT (ms):                        13.62     
P50 TPOT (ms):                           13.62     
P90 TPOT (ms):                           15.57     
P99 TPOT (ms):                           17.04

Raw Rust prefill saturation outputs: three runs per arm

0-base, seed102

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  47.38     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              21.11     
Output token throughput (tok/s):         21.11     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          43248.13  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          23937.15  
Median TTFT (ms):                        24020.28  
P50 TTFT (ms):                           24020.28  
P90 TTFT (ms):                           42569.03  
P99 TTFT (ms):                           46679.71

0-base, seed104

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  47.32     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              21.13     
Output token throughput (tok/s):         21.13     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          43297.96  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          23937.07  
Median TTFT (ms):                        23927.66  
P50 TTFT (ms):                           23927.66  
P90 TTFT (ms):                           42602.76  
P99 TTFT (ms):                           46720.64

0-base, seed106

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  47.09     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              21.23     
Output token throughput (tok/s):         21.23     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          43508.67  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          23783.27  
Median TTFT (ms):                        23908.58  
P50 TTFT (ms):                           23908.58  
P90 TTFT (ms):                           42531.70  
P99 TTFT (ms):                           46654.96

1-mega, seed102

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  44.88     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              22.28     
Output token throughput (tok/s):         22.28     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          45656.92  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          22670.26  
Median TTFT (ms):                        22661.30  
P50 TTFT (ms):                           22661.30  
P90 TTFT (ms):                           40387.24  
P99 TTFT (ms):                           44294.16

1-mega, seed104

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  44.93     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              22.26     
Output token throughput (tok/s):         22.26     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          45607.52  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          22669.64  
Median TTFT (ms):                        22751.89  
P50 TTFT (ms):                           22751.89  
P90 TTFT (ms):                           40404.60  
P99 TTFT (ms):                           44304.38

1-mega, seed106

================= Serving Benchmark Result =================
Successful requests:                     1000      
Failed requests:                         0         
Benchmark duration (s):                  44.82     
Total input tokens:                      2048000   
Total generated tokens:                  1000      
Request throughput (req/s):              22.31     
Output token throughput (tok/s):         22.31     
Peak output token throughput (tok/s):    24.00     
Peak concurrent requests:                1000.00   
Total token throughput (tok/s):          45720.74  
--------------------Time to First Token---------------------
Mean TTFT (ms):                          22645.81  
Median TTFT (ms):                        22714.01  
P50 TTFT (ms):                           22714.01  
P90 TTFT (ms):                           40468.27  
P99 TTFT (ms):                           44390.25

Raw GSM8K output — no accuracy-parity claim

1319 questions, 5-shot, natural MTP3. DEP8 candidate scores range 0.908–0.919 over six runs; prefill includes 0.902. Latencies in these raw evaluator outputs are not serving benchmark speedups.

DEP8 baseline

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.917
Invalid responses: 0.000
Total latency: 108.678 s
Questions per second: 12.137
Total output tokens: 120847
Output tokens per second: 1111.973

DEP8 candidate, run1

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.908
Invalid responses: 0.000
Total latency: 68.398 s
Questions per second: 19.284
Total output tokens: 120807
Output tokens per second: 1766.247

DEP8 candidate, run2

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.914
Invalid responses: 0.000
Total latency: 69.011 s
Questions per second: 19.113
Total output tokens: 121909
Output tokens per second: 1766.527

DEP8 candidate, run3

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.919
Invalid responses: 0.001
Total latency: 69.323 s
Questions per second: 19.027
Total output tokens: 121140
Output tokens per second: 1747.467

DEP8 candidate, run4

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.908
Invalid responses: 0.001
Total latency: 69.986 s
Questions per second: 18.847
Total output tokens: 121998
Output tokens per second: 1743.173

DEP8 candidate, run5

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.919
Invalid responses: 0.001
Total latency: 68.679 s
Questions per second: 19.205
Total output tokens: 121511
Output tokens per second: 1769.264

DEP8 candidate, run6

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.914
Invalid responses: 0.000
Total latency: 69.491 s
Questions per second: 18.981
Total output tokens: 121133
Output tokens per second: 1743.134

Prefill baseline

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.922
Invalid responses: 0.000
Total latency: 73.509 s
Questions per second: 17.943
Total output tokens: 121808
Output tokens per second: 1657.052

Prefill candidate, run1

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.917
Invalid responses: 0.000
Total latency: 51.888 s
Questions per second: 25.420
Total output tokens: 121251
Output tokens per second: 2336.773

Prefill candidate, run2

Running GSM8K evaluation: 1319 questions, 5-shot

Results:
Accuracy: 0.902
Invalid responses: 0.001
Total latency: 46.541 s
Questions per second: 28.340
Total output tokens: 120896
Output tokens per second: 2597.604

Details

• This archive intentionally keeps the tested temporary selector and existing modular experts wrapper. It is not the intended upstream architecture. A future integration should qualify [CI/Build] bump minimum cmake version vllm-project/vllm#6999 through the actual flashinfer.moe_ep API, rather than extend this legacy API detour.
• Weights remain NVFP4 in memory, but the SM90 kernel promotes FP4/local scales into E4M3 for computation and quantizes activations/intermediates. Exact power-of-two scale rebalancing does not imply bitwise model equivalence with HUMMING.
• Auto/split repeatedly stalled in serving; the standalone replay with random weights/activations did not reproduce it. Fused-only completed the recorded quality/benchmark runs. Root cause is unresolved; no FI-only bug or general hang-freedom claim.
• EPLB, microbatching/DBO, runtime weight transfer, input router-weight placement, EP2 and unsupported expert dimensions/scales are rejected. The default backend is unchanged.
• vLLM #62603 covers block-FP8 through MoE-EP, a separate precision/API path. #62073 overlaps the idle-rank prerequisite. No duplicate upstream PR is being opened. Old FP8 performance numbers are reference only: different vLLM revisions and ReplaySSM settings prevent a controlled FP8/NVFP4 comparison.
• All experiment services and queued validation were stopped on request. The complete decode evidence, partial prefill evidence and profiles are preserved locally. No further optimization is scheduled.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill.
• AI assistance was used during the creation of this PR.
• Design Fit: held experiment; unified MoE-EP integration is the preferred follow-up.
• Testing and Validation: decode A/B/A complete; prefill return baseline and final cleanup/sync-check verification canceled.
• Code Quality and Style: local hooks passed; documentation included.
• Pull Request Contents: raw commands/results, precision differences, and limitations included.