Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 43 additions & 5 deletions models/deepseek-ai/DeepSeek-V4.1-Flash.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ meta:
provider: "DeepSeek"
description: "DeepSeek V4.1 Flash vision-language MoE (552B backbone; 8B active per prompt token, 16B per output token) combining sliding-window plus compressed sparse attention with a two-level indexer, engram n-gram memory, hyper-connections, and a DSpark multi-token draft head."
date_added: 2026-09-09
date_updated: 2026-09-17
date_updated: 2026-09-20
difficulty: advanced
tasks:
- text
Expand Down Expand Up @@ -169,6 +169,14 @@ strategy_overrides:
h100: 8
b300: 2
hardware_overrides:
blackwell:
extra_args:
- "--attention-config"
- '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
- "--kv-cache-dtype"
- "fp8"
- "--max-num-seqs"
- "128"
b300:
extra_args:
- "--engram-config"
Expand All @@ -181,6 +189,20 @@ strategy_overrides:
- "8192"
- "--max-num-seqs"
- "256"
single_node_dep:
hardware_overrides:
blackwell:
extra_args:
- "--kernel-config"
- '{"moe_backend":"deep_gemm_mega_moe"}'
- "--attention-config"
- '{"backend":"FLASHMLA_MEGA_ATTN_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
- "--kv-cache-dtype"
- "fp8"
- "--max-num-seqs"
- "128"
- "--engram-config"
- '{"embedding_across_dp":true}'
pd_cluster:
# Verified 1P1D on GB200 NVL4: one tray per role, TP4 each, NIXL transfer.
env:
Expand Down Expand Up @@ -462,11 +484,27 @@ guide: |
client. Validation evidence applies to that pinned benchmark configuration, not
every future `:nightly` image or the general command's inherited scheduler limits.

## 4xB300
## Blackwell TP and DEP

On B200, GB200 and GB300, **Tensor Parallel** uses TP4 with
`FLASHINFER_MLA_SPARSE_DSV41`. **Data + Expert Parallel** uses the hardware's
GPU count: DEP4 on GB200/GB300 trays, DEP8 on B200/B300 nodes. It enables
`--enable-expert-parallel`, `FLASHMLA_MEGA_ATTN_DSV41`,
`--kernel-config '{"moe_backend":"deep_gemm_mega_moe"}'`, and
`--engram-config '{"embedding_across_dp":true}'`.

Both configurations set `indexer_kv_dtype` to `mxfp4`, `indexer_sparse_logits`
to `true`, `--kv-cache-dtype fp8`, and `--max-num-seqs 128`. Reasoning uses
`--reasoning-parser deepseek_v41`.

B300 retains its **Tensor Parallel** override: TP2, Engram CPU offload,
the existing CUDA Graph capture configuration (`--max-cudagraph-capture-size 8190`),
`--max-num-batched-tokens 8192`, and `--max-num-seqs 256`. Its exact-GPU
settings take precedence over the shared Blackwell defaults. For high
interactivity, use `--tensor-parallel-size 4`.

The command builder defaults to TP2 for B300 (2 of the 8 GPUs). For high
interactivity, deploy the model using TP4: pass
`--tensor-parallel-size 4`.
These command settings were updated on 2026-09-20; they have not been rerun
on GPUs as part of this recipe update.

## KV cache offloading

Expand Down