[DSV4.1] Avoid PDL for small target routers on SM103

[BBuf]

Motivation

On 4×GB300, the small DeepSeek-V4.1 Flash target router occasionally has long latency while overlapping with mHC statistics and shared-expert work. Using ordinary stream dependencies for this router improves BS=1 decode throughput without changing routing arithmetic or removing those overlap streams.

Modifications

Limit the launch-policy change to SM103, sqrtsoftplus, 384 experts, top-6, and 1–8 rows. BS=1 with DSPARK block size 5 verifies 6 target rows. The draft router and larger batches keep their existing PDL behavior.

The existing use_pdl value controls both launch_pdl and the kernel's gdc_wait / gdc_launch_dependents instructions. Set it to false for the measured shape, so all three are disabled together and CUDA's ordinary stream ordering supplies the dependency. Scoring, top-k selection, tie-breaking, normalization and packing are unchanged. No new flag, environment variable, or CUDA stream.

This is an independent PR on dsv4.1, not stacked on #39704.

Accuracy Tests

On GB300, the full test/registered/kernels/ops/moe/test_moe_fused_gate.py file passes: 52 passed, including 6 new graph-replay cases. The new cases compare PDL and ordinary launches bitwise for weights, expert IDs and packed outputs, with changing in-graph producers/consumers, FP32/BF16 bias inputs, equal-score ties and varying live/padded rows. The test can exercise both launch variants on Hopper CI by mocking only the dispatch architecture; Triton still targets the actual GPU.

PYTHONPATH="$PWD/python" CUDA_VISIBLE_DEVICES=0 \
  python -m pytest -q test/registered/kernels/ops/moe/test_moe_fused_gate.py

Full GSM8K/AIME/GPQA evaluations have not been rerun for this PR. Kernel equality is not presented as a new task-accuracy result. Pre-commit checks pass for both touched files.

Speed Tests and Profiling

Base: 00d7d516d047292263aff7ed3dd2449668d0f7c7 (dsv4.1). Candidate: f4a4da0c16bdc4cd0749890126aceb6c0a2e4f5a (this PR only).

Version	Fresh start 1 median	Fresh start 2 median	All 40 measured requests, median
Base	1168.53 tps	1168.16 tps	1168.16 tps
This PR	1202.94 tps	1219.13 tps	1210.46 tps

Pooled median improvement: +3.62%. Individual-run medians improve by 2.94% and 4.36%. All four runs have median reported acceptance length 5.5054 (range 5.4468–5.5956).

The 40-request ranges are 1131.47–1216.70 tps for base and 1159.78–1251.30 tps for this PR. These results describe this fixed synthetic workload; no BS=32/64 gain is claimed for this small-row policy.

Hardware/software: 4×NVIDIA GB300 (SM103), TP4 / EP1; original DeepSeek-V4.1 Flash checkpoint; PyTorch 2.13.0+cu130, Triton 3.7.1, FlashInfer 0.6.18, sglang-kernel 0.4.7, sgl-deep-gemm 0.2.0, CUTLASS DSL 4.6.2.

BS=1, fixed seed-42 random-token input of 4096 tokens, 1024 output tokens, temperature 0 and ignore-EOS. DSPARK block size 5; acceptance is simulated with target length 5.5, not measured natural acceptance. Each fresh server has 1 discarded warm-up request and 20 measured requests; all measured requests are included. Order: base → candidate → candidate → base. No profiler during speed measurements.

Throughput is measured by the streaming client: (final completion tokens − first event tokens) / (last event timestamp − first event timestamp). This excludes TTFT and the first streamed token group; it is not a server-log throughput value or whole-request throughput. The same prompt, cache flushes, launch parameters and dependencies are used for both variants.

Server command and exact client workload

Run from either checkout with the same environment; MODEL points to the original Flash checkpoint.

#!/usr/bin/env bash
set -euo pipefail
# Invoke from the selected checkout; set MODEL to the original Flash checkpoint.
export PYTHONPATH="$PWD/python"
export CUDA_VISIBLE_DEVICES=0,1,2,3
export MAX_JOBS=16
export SGLANG_SPEC_TP_SYNC=rng
export SGLANG_RAGGED_VERIFY_MODE=static
export SGLANG_SIMULATE_ACC_LEN=5.5
export SGLANG_SIMULATE_ACC_METHOD=match-expected
python -m sglang.launch_server \
  --model-path "$MODEL" \
  --served-model-name deepseek-ai/DeepSeek-V4.1-Flash \
  --tp 4 --ep-size 1 --trust-remote-code \
  --moe-a2a-backend none --moe-runner-backend flashinfer_mxfp4 \
  --mem-fraction-static 0.80 --max-total-tokens 33554432 \
  --chunked-prefill-size 4096 \
  --cuda-graph-bs-decode 1 2 4 8 16 32 64 \
  --max-running-requests 128 --speculative-algorithm DSPARK \
  --speculative-dspark-block-size 5 --skip-server-warmup \
  --reasoning-parser deepseek-v41 --random-seed 42 \
  --decode-log-interval 10 --host 127.0.0.1 --port 30021

Save the following as client_repro.py, then run python client_repro.py "$MODEL". Restart the server for each repeat. The input-ID SHA256 assertion detects a mismatched tokenizer/workload.

"""BS=1 fixed-prompt decode throughput; run with the PR's server command."""
import hashlib
import json
import random
import statistics
import sys
import time
from pathlib import Path

import requests
from tokenizers import Tokenizer

model = Path(sys.argv[1])
url = 'http://127.0.0.1:30021'
tokenizer = Tokenizer.from_file(str(model / 'tokenizer.json'))
special = {i for i, t in tokenizer.get_added_tokens_decoder().items() if t.special}
vocab = sorted(set(tokenizer.get_vocab().values()) - special)
rng = random.Random(42)
ids = [vocab[rng.randrange(len(vocab))] for _ in range(4096)]
assert hashlib.sha256(json.dumps(ids, separators=(',', ':')).encode()).hexdigest() == 'c37b49585df8fb22e125ae467003a00fff94a3f010cc7cddc03a3b463d047614'
requests.post(url + '/freeze_gc', timeout=60).raise_for_status()
measurements = []
for rep in range(21):
    requests.post(url + '/flush_cache?timeout=30', timeout=60).raise_for_status()
    first = last = None
    with requests.post(url + '/generate', json={
        'input_ids': ids,
        'sampling_params': {'temperature': 0, 'max_new_tokens': 1024,
                            'ignore_eos': True, 'stream_interval': 1},
        'stream': True,
    }, stream=True, timeout=(30, 1800)) as response:
        response.raise_for_status()
        for line in response.iter_lines():
            if not line.startswith(b'data: ') or line[6:] == b'[DONE]':
                continue
            chunk = json.loads(line[6:])
            now = time.perf_counter()
            assert 'error' not in chunk, chunk
            meta = chunk['meta_info']
            count = meta.get('completion_tokens', 0)
            if count and first is None:
                first = (now, count)
            last = (now, count)
    assert first is not None and last[1] == 1024
    assert meta['prompt_tokens'] == 4096 and meta.get('cached_tokens', 0) == 0
    tps = (last[1] - first[1]) / (last[0] - first[0])
    print(rep, tps, meta.get('spec_accept_length'), flush=True)
    if rep:
        measurements.append(tps)
print('median:', statistics.median(measurements))
print('all:', json.dumps(measurements))

Checklist

• Format both touched files with pre-commit.
• Extend the existing kernel tests.
• Document the launch scope, reproduction command and measurement limitations above.
• Provide kernel correctness and fresh-server performance results.
• Follow the existing kernel wrapper style.

---

CI States

Latest PR Test (Base): ❌ Run #35301367356
Latest PR Test (Extra): ❌ Run #35301367205
Latest PR Test (AMD ROCm 10): ❌ Run #35301367271