Repository navigation

[omni, model, ckpt, dist, ops, config, docs] feat: support DeepSeek-V4.1-Flash in SeedOmni - #1370

Open

Coach257 wants to merge 165 commits into

omniv2from

szl.omniv2-deepseek-v4.1-flash

Open

[omni, model, ckpt, dist, ops, config, docs] feat: support DeepSeek-V4.1-Flash in SeedOmni#1370
Coach257 wants to merge 165 commits into
omniv2from
szl.omniv2-deepseek-v4.1-flash

Commits on Sep 16, 2026
- [model] feat: add deepseek_v41_ngram SeedOmni module
  
  Coach257
  and
  cursoragent
  committed
  5ba1aad
- [model] feat: add deepseek_v41 vision and text_encoder SeedOmni modules
  
  Coach257
  and
  cursoragent
  committed
  805a32d
- [model] feat: add deepseek_v41_llm config and Engram connection
  Coach257
  committed
  c6185c2
- [model] fix: declare gate_temp on DeepseekV41LlmConfig and note MoE router divergences
  
  Coach257
  and
  cursoragent
  committed
  52de97d
- [model] feat: add deepseek_v41 llm attention stack
  
  Coach257
  and
  cursoragent
  committed
  c489dcb
- [model] feat: train the deepseek_v41 indexer with an auxiliary KL
  
  Coach257
  and
  cursoragent
  committed
  b8b56b7
- [model] feat: add DeepSeek-V4.1 mHC wiring, MoE routing and decoder layer
  
  Coach257
  and
  cursoragent
  committed
  e725c2e
- [model] feat: add the deepseek_v41_llm backbone and its graph hooks
  
  Coach257
  and
  cursoragent
  committed
  8a03700
- [model] feat: wire the deepseek_v41 n-gram node into the training graph
  
  Coach257
  and
  cursoragent
  committed
  e0bbcab
- [model] feat: add deepseek_v41 checkpoint dequantization
  
  Coach257
  and
  cursoragent
  committed
  89f3f3a
- [model] feat: add the deepseek_v41 checkpoint converter
  
  Coach257
  and
  cursoragent
  committed
  482bcf0
- [model] docs: record the deepseek_v41 Step 2 landing
  
  Coach257
  and
  cursoragent
  committed
  3a93bb4
- Merge remote-tracking branch 'origin/omniv2' into szl.omniv2-deepseek-v4.1-flash
  Coach257
  committed
  1203fe9
- [docs] docs: record the dev box's proxy and nvm PATH facts
  
  Coach257
  and
  cursoragent
  committed
  c453d3e
- [omni, model, config] feat: add the DeepSeek-V4.1-Flash training config and the ngram freeze hook
  
  Coach257
  and
  cursoragent
  committed
  75bcf86
- [config] fix: point the v41 convert command at the shared entry point
  
  Coach257
  and
  cursoragent
  committed
  19a359f
- [omni, ckpt] fix: stage converter shards when the destination has no memory mapping
  
  Coach257
  and
  cursoragent
  committed
  80aa4ea
- [omni, model, ckpt] fix: separate the ngram module's two vocabulary sizes
  
  Coach257
  and
  cursoragent
  committed
  0a6d286
- [omni, ckpt] fix: dequantize the engram tables in row chunks, and accept their per-row scale
  
  Coach257
  and
  cursoragent
  committed
  62e146b
Commits on Sep 17, 2026
Commits on Sep 18, 2026
- [model] feat: add a quantized linear for DeepSeek-V4.1 inference
  
  Coach257
  and
  cursoragent
  committed
  48b79fa
- [seed_omni] feat: add config-only export to refresh omni checkpoint metadata
  
  Coach257
  and
  cursoragent
  committed
  5cf066b
- [seed_omni] feat: run DeepSeek-V4.1 dense projections from the released fp8 weights
  
  Coach257
  and
  cursoragent
  committed
  ee317d6
- [seed_omni] feat: run the grouped output projection and shared expert from fp8
  
  Coach257
  and
  cursoragent
  committed
  ed1d705
- [seed_omni] feat: run the routed experts from the released fp4 slabs
  
  Coach257
  and
  cursoragent
  committed
  5373ed2
- [seed_omni] feat: keep the n-gram tables in the fp8 they were released in
  
  Coach257
  and
  cursoragent
  committed
  3465ac3
- [seed_omni] feat: expert-parallel inference over the fp4 slabs, and its config
  
  Coach257
  and
  cursoragent
  committed
  e922902
- [seed_omni] fix: a module asking only for ep or emb was built as if unsharded
  
  Coach257
  and
  cursoragent
  committed
  bb0df84
- [seed_omni] fix: pad the n-gram tables into a row count that can be sharded
  
  Coach257
  and
  cursoragent
  committed
  009eac0
- [seed_omni] fix: shard a quantized weight's scales alongside the weight
  
  Coach257
  and
  cursoragent
  committed
  bdd0796
- [arguments,distributed] fix: let a module overlay's ep_size reach the mesh
  
  Coach257
  and
  cursoragent
  committed
  a080ce2
- [distributed,arguments,seed_omni] feat: shard inference by plan alone, without the FSDP wrap
  
  Coach257
  and
  cursoragent
  committed
  3906352
- [seed_omni,docs] feat: add a DeepSeek-V4.1 image chat script
  
  Coach257
  and
  cursoragent
  committed
  85b4ab1
- [seed_omni] fix: make the ranks agree on the sampled token
  
  Coach257
  and
  cursoragent
  committed
  8f21237
Commits on Sep 20, 2026
- [seed_omni] fix: make generation reproducible across ranks
  
  Coach257
  and
  cursoragent
  committed
  15cafc4
Commits on Sep 21, 2026
- [seed_omni] feat: dequant V4.1 at load and capture KV on batched prefill
  
  Coach257
  and
  cursoragent
  committed
  c80043a
- [seed_omni] feat: hang DSpark on generate and convert the tied MTP tensors
  
  Coach257
  and
  cursoragent
  committed
  bb99143
- [seed_omni,docs] feat: drop full-determinism and text-encoder emb from serving
  
  Coach257
  and
  cursoragent
  committed
  6469b1f
Commits on Sep 22, 2026
- [seed_omni] feat: skip DSpark hits and CED-prefill without changing training
  Coach257
  committed
  0495682
Commits on Sep 23, 2026
Commits on Sep 28, 2026
Commits on Sep 29, 2026
- [seed_omni] fix: train DeepSeek-V4.1 DSpark as decode's block slot 0
  Coach257
  committed
  00389fd
- [seed_omni] feat: add DeepSeek-V4.1 64-card configs that train every llm layer kind
  Coach257
  committed
  6164cbe
Commits on Oct 5, 2026
- [seed_omni] fix: accept data-layer uint8 image tensors in the DeepSeek-V4.1 image processor
  Coach257
  committed
  1b50a28
- [docs] chore: note that SeedOmni preprocessors get uint8 image tensors in training
  Coach257
  committed
  6deff18
Commits on Oct 6, 2026
Commits on Oct 7, 2026
Commits on Oct 8, 2026
- [omni] feat: log each SeedOmni V2 loss term as module.method.loss_name
  Coach257
  committed
  4116a85
Commits on Oct 13, 2026
Commits on Oct 16, 2026
Commits on Oct 17, 2026
- [omni, ckpt] fix: shard large carried tensors and consolidate HF source exports across ranks
  Coach257
  committed
  afcb266
Commits on Oct 19, 2026
- [omni, distributed] fix: average extra_parallel_only replicated grads in omni clip_grad_norm and align full128 text_encoder with FSDP2
  
  Coach257
  and
  cursoragent
  committed
  2526543
Commits on Oct 20, 2026
- [docs] docs: add DeepSeek-V4.1-Flash verification report with B tokens/day throughput and CP alignment
  
  Coach257
  and
  cursoragent
  committed
  3ee068e
- [omni, parallel] feat: add Ulysses sequence parallelism to deepseek_v41_vision
  
  Coach257
  and
  cursoragent
  committed
  1056f7a
- Merge branch 'omniv2' into szl.omniv2-deepseek-v4.1-flash
  
  Coach257
  and
  cursoragent
  committed
  fa81f0a
- [omni, parallel] feat: support ViT Ulysses SP + LLM CP in DeepSeek-V4.1
  
  Coach257
  and
  cursoragent
  committed
  2ee61e9
- Merge branch 'omniv2' into szl.omniv2-deepseek-v4.1-flash
  Coach257
  committed
  5f09fc2
- [omni, ops] refactor: migrate deepseek_v41 to module-level VeomniOp
  
  Coach257
  and
  cursoragent
  committed
  f92839d
- [docs] docs: update deepseek_v41 verification report for f92839dc2 VeomniOp and ViT SP + LLM CP
  
  Coach257
  and
  cursoragent
  committed
  e4d04e8
Commits on Oct 22, 2026
Commits on Oct 23, 2026
Commits on Oct 26, 2026
Commits on Oct 27, 2026
- [docs] docs: add DeepSeek-V4.1-Flash-1017 video SFT with 3L MTP, ViT GC, dyn_bsz, and before/after 8-GPU video inference results
  
  Coach257
  and
  cursoragent
  committed
  aaa00e4

Commits

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 20, 2026

Commits on Sep 21, 2026

Commits on Sep 22, 2026

Commits on Sep 23, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026

Commits on Oct 8, 2026

Commits on Oct 13, 2026

Commits on Oct 16, 2026

Commits on Oct 17, 2026

Commits on Oct 19, 2026

Commits on Oct 20, 2026