[mikasenghaas]

Note

This PR is not meant to be merged. The branch is the prime-rl fork that our DeepSeek V4 Flash SFT runs pin. It stays on top of main and carries only the fixes that are not upstream yet. We rebase it onto main on our own schedule. When a fix lands upstream, we drop it from this branch.

Last synced with main at d5f29c072 (vLLM 0.30.0) on 2026-09-30. Earlier pinned heads are tagged dsv4f-sft/2026-09-27, dsv4f-sft/2026-09-28, and dsv4f-sft/2026-09-29.

Summary

The branch carries these commits on top of main:

• DeepSeek V4 attention-sink loading: loads the attention sinks through vLLM's weight loaders. Before this fix, a layerwise weight update silently discarded the new sink values, so inference kept serving the boot values. vLLM 0.30.0 still writes the sinks with a bare slice copy.

• Abort stale requests on pause: /pause accepts a mode and defaults to abort. The orchestrator pauses with abort instead of keep, so orphaned requests cannot block a weight update or resume on old-policy KV state.

• inference.fp8_ue8m0_weight_scales: quantizes online block-FP8 weights with power-of-two block scales. Checkpoints that were dequantized from FP8 then convert back to FP8 exactly.

• SwiGLU clamp on the Triton FP8 fast path: vLLM's fused silu_and_mul_per_block_quant path ignores DeepSeek V4's swiglu_limit. The patch routes clamped models to the clamped activation path.

• inference.fp8_stochastic_weight_rounding: rounds online FP8 weights stochastically. The served weight then equals the BF16 trainer weight in expectation, so it tracks updates smaller than one FP8 bin. Includes unit tests.

• Local vLLM and renderer plugins: inference.vllm_plugins lists module:function or path.py:function targets. prime-rl calls each target in every vLLM process, for example to register a local tokenizer mode. SFT accepts renderers plugin configs (renderers#165), so a local renderer can train without an upstream release.

• [[data.source]] for SFT: each source is a single (dataset, subset, split), with an optional weight, columns, and renderer table. data.columns maps dataset columns onto messages, prompt, completion, tools, and per-sample renderer fields (for example renderer = { depth = "task_depth" }). Renderer arguments apply in this order: [renderer], then the source's renderer table, then non-null per-sample columns. The legacy name/subsets/splits/probabilities fields resolve to the same sources with the same interleave order and metric names, so resumed runs keep their sample order. Sources load only their mapped columns. Columns whose types differ between sources are stored as JSON so the sources can interleave.

• Numpy interleave indices: load_sft_dataset builds interleave_datasets' probability-weighted indices with vectorized numpy instead of datasets' per-draw Python loop. The indices are identical, since Generator.choice with p is a searchsorted over rng.random, so resumed runs keep their order. A 45-subset all_exhausted mix (21M rows) goes from about 15 minutes per rank to seconds.

• Pad SFT packs with their own tokens: CatDataset padded packs with token 0. Identical pad tokens get identical hidden states, so the MoE router sends every pad token of a pack to the same experts, and the expert-parallel rank that hosts them receives the whole pad run. Pads now repeat the pack's own tokens (text packs only). They stay loss-masked and come after every real token, so the loss is unchanged.

Verification

• Ran ruff check and ruff format --check on each commit.
• After the rebase onto vLLM 0.30.0: applied every shared vLLM patch in one process, and ran pytest tests/unit/inference tests/unit/train/sft tests/unit/test_configs.py tests/unit/train/models/test_fp8_utils.py: 315 passed, 15 skipped.
• Ran pytest tests/unit/inference tests/unit/orchestrator/test_clients.py: 33 passed.
• Ran pytest tests/unit/test_configs.py tests/unit/train/sft: 252 passed, including 13 new source tests.
• Ran pytest tests/unit/train/sft: 82 passed, including exact index equality with interleave_datasets for both stopping strategies and pad-token content.
• Measured MoE routing on the exact packs of a DSV4-Flash SFT step that stalled a 256K run (EP16). With token-0 padding, one EP group's pack pair was 73% padding, and some EP ranks received up to 6.2x their balanced load (one rank 6.1x in the last layer). With content padding on the same packs, the worst rank got 2.1x and the loss was identical. The rerun passed that step without the memory-ceiling stall, with expert imbalance (max_vio) about halved and about 10% more throughput.
• Loaded a private 44-subset SFT dataset as [[data.source]] tables with a local renderer plugin. The per-sample renderer column was set on all 4,268 rendered rows.

🤖 Generated with Claude Code