Merge GDN conv fusion (fused pre-gated-delta-rule kernels) - #13

Closed

wplf wants to merge 23 commits into

wplf:jinliangl/qwen35-vl-central-devfrom

yuzhongw-nvidia:yuzhongw/gdn_conv_fusion

Commits on May 21, 2026

[Dev] add support for deepep/hybridep dispatcher under thd format training (#4816)

HaochenYuan
and
claude[bot]
authored
0afbb98
[dev] Fix FSDP TP metadata for LinearCrossEntropyModule (#4888)
conver334
authored
98b595c
[Dev] Add MoE example recipes (#4890)
Dennis(Zhenhuan) Liu
authored
6b6fb95

Commits on May 22, 2026

[Dev] Remove duplicated `bias_act_func` from main to dev sync (#4927)
yaox12
authored
56481b0

Commits on May 27, 2026

Commits on May 28, 2026

[Dev] Add Qwen3 30B MoE recipes (#5012)
Dennis(Zhenhuan) Liu
authored
7b9593b
[DEV] fix(megatron-fsdp): reduce padding for grouped expert weights (#5013)

xuwchen
and
yaox12
authored
473145c
[Dev] fix(combined-1f1b): release loss-node input storage after combined backward (#4908)

Wohox
and
yaox12
authored
3e8ce1f
[dev] [fix] [DeepSeek-v4] fix dense loss and rope type in DSv4 Hybrid Attention (#5018)

hxbai
and
yuzhongw-nvidia
authored
35f36c7

Commits on May 29, 2026

Commits on Jun 2, 2026

Add fused GDN pre-gated delta rule path
yuzhongw-nvidia
committed
b3c1c7e
Refactor fused pre-GDR forward scopes
yuzhongw-nvidia
committed
11a6948
Merge QK l2 backward for fused GDN
yuzhongw-nvidia
committed
d542489
Clean up fused GDN backward elementwise launches
yuzhongw-nvidia
committed
5ab7855
Tidy fused GDN kernel style

Yuzhong Wang
authored and
yuzhongw-nvidia
committed
e9e1772
Add packed fused GDN path tests

Yuzhong Wang
authored and
yuzhongw-nvidia
committed
0bcfd98
Add packed THD fused GDN kernels

Yuzhong Wang
authored and
yuzhongw-nvidia
committed
b5e0b9c
Add fused mega pre-GDR interface

Yuzhong Wang
authored and
yuzhongw-nvidia
committed
0b2fd55
Implement mega fused pre-gated-delta-rule forward and backward kernels

authored and
yuzhongw-nvidia
committed
c90fc7e
Note --pre-gated-delta-rule-impl is auto-generated from config

authored and
yuzhongw-nvidia
committed
05bd2da