Add DeepSeek V4 - #45643

Merged

ArthurZucker merged 44 commits into

mainfrom

add-deepseek-v4

May 2, 2026

Commits on Apr 28, 2026

Add DeepSeek V4 (modular)
ArthurZucker
committed
26c62d0
Split V4 HCA / CSA caches and compressors into independent classes
ArthurZucker
committed
8b3c91a
Fix tests_generate / tests_tensor_parallel CI failures

ArthurZucker
and
claude
committed
b4b3a20

Commits on Apr 29, 2026

Address PR review feedback batch (comments 2-24)
ArthurZucker
committed
6baa653
Address PR review feedback batch (comments 25-42)
ArthurZucker
committed
838cc0c
Fix Fp8Dequantize.reverse_op to actually re-quantize on save
ArthurZucker
committed
f18a6b8
Address Arthur's review batch + revisit two of vasqu's comments
ArthurZucker
committed
921a8dc
Apply suggestions from code review
ArthurZucker
authored
6d82332
Move DeepseekV4Config out of modular + simplify __post_init__
ArthurZucker
committed
4ee8e47
Fix V4 TP failures: dynamic num_key_value_groups + FP8-safe GroupedLinear
ArthurZucker
committed
ea29922
Revert GroupedLinear F.linear loop, keep bmm
ArthurZucker
committed
092dcd6

Commits on Apr 30, 2026

Fix V4 GroupedLinear comment with real V4-Flash / V4-Pro config values
ArthurZucker
committed
def7199
up

ArthurZucker
and
Copilot
committed
631151d
up

ArthurZucker
and
Copilot
committed
fa0369f
small cleanup
ArthurZucker
committed
5f711e0
repo fixes
ArthurZucker
committed
b5998a0
nits

ArthurZucker
and
Copilot
committed
dcbe66b
more nits
ArthurZucker
committed
aecf319
nits, small thing left to do
ArthurZucker
committed
4fc49a0
update
ArthurZucker
committed
02356fe

Commits on May 1, 2026

update DeepseekV4HCACache
ArthurZucker
committed
8124d3e
more update
ArthurZucker
committed
45c4323
nits
ArthurZucker
committed
a48b51b
update
ArthurZucker
committed
46bc564
Merge branch 'add-deepseek-v4' of github.com:huggingface/transformers into add-deepseek-v4
ArthurZucker
committed
240647a
Merge branch 'add-deepseek-v4' of github.com:huggingface/transformers into add-deepseek-v4
ArthurZucker
committed
5eb7bec
update
ArthurZucker
committed
8f807c1
nits
ArthurZucker
committed
0c62b58
update
ArthurZucker
committed
e32e9a8
fixes
ArthurZucker
committed
5b125c6
nits
ArthurZucker
committed
07aca9f
Merge branch 'main' of https://github.com/huggingface/transformers into add-deepseek-v4
ArthurZucker
committed
913adcc
Fix CI: \N>1 backrefs in conversion mapping + drop irrelevant drift

ArthurZucker
and
claude
committed
7b5b54a
Drop TP plan from V4 attention path; keep EP-only on experts

ArthurZucker
and
claude
committed
7feb6a1
Address vasqu review batch + skip quantized-TP test
ArthurZucker
committed
d515ec5
Drop model. prefix from V4 conversion mapping + tester / parity cleanup
ArthurZucker
committed
65e564f

Commits on May 2, 2026

Silence Unrecognized rope_parameters keys warning
ArthurZucker
committed
a021e84
Disable flex_attention on V4; guard tensor-only mask pad

ArthurZucker
and
claude
committed
0e35cfd
Switch V4 to EP-only plan (gpt-oss style)

ArthurZucker
and
claude
committed
a55ffee
Fix V4 yarn rope validation under nested rope_parameters

ArthurZucker
and
claude
committed
d86d46d
Trim V4 config-attribute allowlist from 24 → 6

ArthurZucker
and
claude
committed
ff720b3
Inline V4 config legacy-kwarg + resolve_* helpers into __post_init__

ArthurZucker
and
claude
committed
c946087
Disable FlashAttention on V4: head_dim=512 exceeds the 256 cap

ArthurZucker
and
claude
committed
62a75c0
update
ArthurZucker
committed
08e4cf8