Add fused pre-gated-delta-rule (GDN conv fusion) - #14

Closed

wplf wants to merge 35 commits into

jinliangl/qwen35-vl-central-devfrom

jinliangl/gdn-conv-fusion

Commits on May 20, 2026

feat(examples/multimodal_dev): add Qwen3.5-VL training example

authored and
Dennis(Zhenhuan) Liu
committed
0915f61
fix(examples/multimodal_dev): correct CORD-V2 dataset SFT masking and preprocessing

wplf
and
claude
committed
aa529b3
chore(examples/multimodal_dev): tune Qwen3.5-VL launcher defaults

wplf
and
claude
committed
6d2e13c

Commits on May 21, 2026

chore(examples/multimodal_dev): bump copyright header to 2026

wplf
and
claude
committed
209a46f

Commits on May 25, 2026

test(examples/multimodal_dev): migrate to pack_or_pad_batch + add CP=1 vs CP=4 correctness

wplf
and
claude
committed
ec6d2d3
fix(examples/multimodal_dev): pack_or_pad_batch TP-broadcast correctness + small cleanups

wplf
and
claude
committed
4332813
style(examples/multimodal_dev): clean lint warnings on changed files (pylint 10/10)

wplf
and
claude
committed
0211665
style(examples/multimodal_dev/tests): apply black to _helpers.py

wplf
and
claude
committed
6399e29
[dev] feat(moe): add NVTX ranges to SharedExpertMLP for shared-expert overlap profiling
wplf
committed
dc835c2
fix(parallel_state): skip NCCL.Options under --fake-process-group
wplf
committed
b6a0063

Commits on May 28, 2026

add norm_out recompute in gdn

xuantengh
authored and
wplf
committed
7b20bf5
add gdn test

xuantengh
authored and
wplf
committed
d4c103a
rebase upstream

xuantengh
authored and
wplf
committed
4df5109
update test

xuantengh
authored and
wplf
committed
383eaad
rebase for thd layout

xuantengh
authored and
wplf
committed
71ed547
include a2a inside the recompute region

xuantengh
authored and
wplf
committed
7300256
fix(moe/fused_a2a): comment out false-positive IB QP depth guard
wplf
committed
e9be7eb
feat(multimodal): enable EP A2A overlap for MultimodalModel decoder-only path
wplf
committed
3c94703
feat(memory_snapshot): enable event recording (frames + device_traces)
wplf
committed
25cc5a3

Commits on May 29, 2026

Add optional mcore GDN optimized wrapper

byshiue
authored and
root
committed
ea9471f
Add Qwen3.5 MRoPE fusion benchmark support

BestJuly
authored and
root
committed
440575f
Fix THD mRoPE CP fallback consistency

BestJuly
authored and
root
committed
6773972
Address mRoPE THD review cleanup

BestJuly
authored and
root
committed
f6a8a17

Commits on May 30, 2026

Commits on Jun 2, 2026

Support discard-output recompute for MoE shared experts (overlap + non-overlap)

wplf
and
claude
committed
35e6f5a

Commits on Jun 3, 2026

docs(gdn): update mcore_gdn_opt install to NaN-fixed 12605c5

wplf
and
claude
committed
cbe72da
Add gdn_qkv discard-output recompute for GatedDeltaNet
wplf
committed
03d0ed3
Configure shared experts in test_fsdp_1f1b_memory_opt
wplf
committed
c87283b
Add gdn_qkv discard-output recompute unit test
wplf
committed
2cac187
Merge pull request #9 from wplf/jinliangl/gdn-qkv-recompute-clean
wplf
authored
b95366e
Merge pull request #10 from wplf/jinliangl/shared-experts-ckpt-clean
wplf
authored
39b9c73
Merge pull request #7 from BestJuly/lit/qwen35_opt_fla
wplf
authored
79f6060
Add fused pre-gated-delta-rule (GDN conv fusion)

yuzhongw-nvidia
authored and
wplf
committed
92e303f