[main] moe(feat): support chunkwise context parallelism for GDN - #5637

Draft

yuzhongw-nvidia wants to merge 544 commits into

NVIDIA:mainfrom

yuzhongw-nvidia:yuzhongw/fla_cp

This pull request is big! We're only showing the most recent 250 commits

Commits on Feb 5, 2026

[Dev] Fix EP Overlap missing record stream for shared expert (#3244)
Wohox
authored
ef336ca

Commits on Feb 6, 2026

Restore missing linear-cross-entropy option accidentally removed from arguments.py (#3266)

shjwudp
and
yaox12
authored
ec94d63

Commits on Feb 9, 2026

Fix reload_model_params failure when loading MoE models with explicit state_dict (#3243)

eternally-z
and
yaox12
authored
500e080
ci: Disable moe20 tests (#3312)
ko3n1g
authored
433c169
ci: Pin down setuptools to lt 82 (#3316)
ko3n1g
authored
fd4801e

Commits on Feb 10, 2026

Commits on Feb 11, 2026

Revert "[None][Fix] Prevent resource leak warnings (#3216)" (#3366)
ko3n1g
authored
28b130f

Commits on Feb 13, 2026

Commits on Feb 14, 2026

[dev] ci: Fix merge queue (#3385)
ko3n1g
authored
1dcf0da

Commits on Feb 23, 2026

[dev] `cp: Cherrypick CI changes` (#3543)
ko3n1g
authored
cd1c215

Commits on Feb 25, 2026

[Dev] Fix MoE aux loss tracker hang with MTP enabled (#3400)
Dennis(Zhenhuan) Liu
authored
aa86018
ci: Remove multi-approval action from dev branch (#3576)
chtruong814
authored
2b4b9c4

Commits on Feb 26, 2026

Merge branch 'main' into dev
FDecaYed
committed
0ab47fa
[dev] pull main 260220 (#3574)

authored
a1a73f8

Commits on Feb 27, 2026

[dev] fix(moe): fix the bug where gate was not sliced when kv_head < tp_size. (#3529)

authored
2e4a5d4

Commits on Feb 28, 2026

Add unit test for THD (#3608)
kunlunl
authored
d0e0cf0

Commits on Mar 2, 2026

[Dev] feat(checkpoint): zero-copy storage sharing in CheckpointWithoutOutput (#3641)

Dennis(Zhenhuan) Liu
and
claude
authored
bc9298c

Commits on Mar 3, 2026

[Dev] Add E2E support for THD format (#2924)

xiaoyao0115
and
kunlunl
authored
5c613ab

Commits on Mar 4, 2026

fix: skip FSDP DTensor boundary validation under fake process group (#3668)

Dennis(Zhenhuan) Liu
and
claude
authored
5dadaf1

Commits on Mar 5, 2026

Commits on Mar 6, 2026

[dev] feat(mHC): Add basic pytorch implementation of manifold hyper connection(mHC). (#2943)

authored
0b0074e
[Dev] Cherry-pick: M-FSDP: Cancel erroneous grad accumulation check (#3629) (#3729)

Dennis(Zhenhuan) Liu
and
shjwudp
authored
597f0d8
[dev] fix(moe): Fix DSA spec and rope. (#3402)
yuzhongw-nvidia
authored
3d097e5

Commits on Mar 10, 2026

Commits on Mar 11, 2026

Commits on Mar 12, 2026

Continue emerging optimizer refactoring (#3737)
skyw
authored
9374a4d

Commits on Mar 17, 2026

Fix emerging optimizer init_group for ckpt loading (#3897)
FDecaYed
authored
f47ad91
fix cg acess issue by using dict instead of list to iteratively acces… (#3867)

ilml
and
yaox12
authored
74124ba
Enhance rotary positional embedding version checks (#3887)

huvunvidia
and
yaox12
authored
51299c5
[DEV] fix(megatron-fsdp): build expt_device_mesh only for MoE models (#3832)

xuwchen
and
yaox12
authored
7c3eea6

Commits on Mar 18, 2026

[Fix][Dev] Missing Assertion for moe layer recomptue in A2A Overlap (#3916)
Wohox
authored
a9e5bf9

Commits on Mar 19, 2026

ci: Fix sso users check (#3937)
chtruong814
authored
ebf1508
Add more emerging optimizers (#3907)

skyw
and
FDecaYed
authored
8ae70d4

Commits on Mar 20, 2026

Support GEMM + Swiglu fused MLP (#3890)

ksivaman
and
yaox12
authored
c72c459

Commits on Mar 25, 2026

Commits on Mar 30, 2026

fix layerwise related merge error due to dev refactor
FDecaYed
committed
6823637
[Dev][feat] Support CUDA Graph capture offloading modules (#3219)

lhb8125
and
root
authored
0c306dc
update golden value for gpt3_moe_mcore_te_tp4_ep2_etp2_pp2_resume_torch_dist_dist_optimizer, respecting dev
FDecaYed
committed
9c0b6ef
Merge branch 'dev' into pull-request/4031
FDecaYed
committed
f36257c
Sync main into dev (#4031)

authored
4ef64eb

Commits on Apr 3, 2026

[Dev] Fix golden values mismatch and dependency error due to last pull main (#4123)

Dennis(Zhenhuan) Liu
and
claude
authored
2bb0d38
[Dev] Skip routed expert padding for graph-safe MoE (#4071)
zhongbozhu
authored
8d1fd3c

Commits on Apr 7, 2026

[DEV] Minor update optimizer (#4082)
skyw
authored
74751c9
TE fused grouped mlp with grouped bias and delayed wgrad (#4095)

ksivaman
and
yaox12
authored
ab6c0ff
[Dev][feat] Support overlapping A2A Combine backprop with wgrad GEMM (#3766)

Wohox
and
cspades
authored
37a4cee
[dev] feat(moe): Support packed sequence for gated delta net (GDN) (#2644)

yuzhongw-nvidia
and
kunlunl
authored
b6193a3
Revert unintended code owner change (#4172)
kunlunl
authored
07b9fb2
[Dev] feat: Dynamic CP (part 2) (#2000)

authored
2d6e946

Commits on Apr 8, 2026

Update golden values nightly (#4185)
ko3n1g
authored
16a985f
[Dev][MoE] Add a new score function to the router (#4193)
yaox12
authored
7fec528
[Dev] feat(fsdp): use TE general_gemm for mixed-precision wgrad in FSDP path (#3824)

Dennis(Zhenhuan) Liu
and
claude
authored
0b121cf
ci: update dev nightly golden values (#4201)

ko3n1g
and
claude
authored
06ecac3

Commits on Apr 9, 2026

Commits on Apr 10, 2026

[Dev] Update golden values for inference (#4246)
yaox12
authored
0410104
[dev] fix: Support mHC with cuda graph and activation offloading (#4190)

jingqiny-99
and
buptzyb
authored
7226640
Add training code to MCore wheel (#3573) (#4255)
maanug-nv
authored
7960a31

Commits on Apr 13, 2026

[dev] fix(ssm): handle alignment padding in GDN packed seq + CP (#4230)

authored
0f6fcb0
[Dev] Paged Stashing (#2690)

authored
c0c4fdc
[Dev] Fix TE version check for retain_pinned_cpu_buffers in cpu offload (#4266)

BestJuly
and
claude
authored
1fc9200

Commits on Apr 14, 2026

[Dev] Add diagnostic warnings to TEGroupedMLP fused impl checks (#4269)

Dennis(Zhenhuan) Liu
and
claude
authored
7ff046b
Merge remote-tracking branch 'origin/main' into main2dev/14_04_2026
svcnvidia-nemo-ci
committed
66e55f9
chore: nightly sync main into dev (14_04_2026)

svcnvidia-nemo-ci
and
claude
committed
11d78a1
fix: take main's pyproject.toml and uv.lock for lock-file consistency

svcnvidia-nemo-ci
and
claude
committed
1d7f3a9
fix: re-run black formatting on 4 files missed in initial commit

svcnvidia-nemo-ci
and
claude
committed
74ec0f9
fix: add missing docstrings to MegatronGradScaler abstract methods

svcnvidia-nemo-ci
and
claude
committed
913ca3d
fix: revert to dev's pyproject.toml, uv.lock, and Dockerfile.ci.dev

svcnvidia-nemo-ci
and
claude
committed
861c840
fix: use main's pyproject.toml, uv.lock, and Dockerfile.ci.dev

svcnvidia-nemo-ci
and
claude
committed
4012662
fix: restore RMSNorm import order in legacy model __init__

svcnvidia-nemo-ci
and
claude
committed
e763da8
fix: remove stale sequence_packing parametrize and use dev's TE revision

svcnvidia-nemo-ci
and
claude
committed
9f69681
fix: restore missing CudaGraphScope import, take dev's gated_delta_net.py and test
Phlip79
committed
2be925c
fix: add fast-hadamard-transform dependency from dev for DSA test
Phlip79
committed
c203444
fix: remove fast-hadamard-transform from no-build-isolation-package to fix linting
Phlip79
committed
e7e7a3e
fix: splice fast-hadamard-transform into pyproject.toml and uv.lock
Phlip79
committed
6b8f089
fix: disambiguate packaging version in uv.lock for fast-hadamard-transform
Phlip79
committed
bd24b5c
fix: take dev's uv.lock — main's lockfile is missing dev-only dependencies
Phlip79
committed
98b2076
fix: take dev's pyproject.toml and uv.lock together — they must be consistent
Phlip79
committed
0ba88b5
revert: restore pyproject.toml, uv.lock, Dockerfile.ci.dev to last known good state
Phlip79
committed
7c27108
fix: add fast-hadamard-transform and regenerate uv.lock in CUDA container
Phlip79
committed
22e70ca
fix: take dev's Dockerfile.ci.dev — it must match dev's pyproject.toml and uv.lock
Phlip79
committed
09a76f8
fix: take dev's pyproject.toml, uv.lock, and Dockerfile.ci.dev
Phlip79
committed
e9f1020
fix: add nvidia-resiliency-ext git source from main to fix ImportError

Phlip79
and
claude
committed
9e33263

Commits on Apr 15, 2026

fix mfsdp unwrap stuck at MegatronFSDP [dev] (#4273)

wplf
and
claude
authored
9a7c5dd
Fix UT timeout (#4310)

kunlunl
and
Kunlun Li
authored
76371d4
fix: restore dev's GroupedQuantizedTensor handling in distrib_optimizer

Phlip79
and
claude
committed
817b2c4
fix: remove double-remove in fine_grained_activation_offload bulk_offload

Phlip79
and
claude
committed
e56a6c0
fix: resolve fine-grained offloading API mismatches from merge

Phlip79
and
claude
committed
2a68a9c

Commits on Apr 16, 2026

[dev] fix(ssm): handle alignment padding in GDN packed seq + CP (#4230)

authored and
wyzthu
committed
01b70a1
chore: nightly sync main into dev (14_04_2026) (#4291)

authored
a2e673f
ci(action): improve GitHub Actions output UX (#4336)

ko3n1g
and
claude
authored
6f0795c

Commits on Apr 17, 2026

fix(dev): correct params->parameters typo in ChainedOptimizer.step() (#4357)

Phlip79
and
claude
authored
c2c7f0f
[Dev] Revert code owner changes from pull main (#4354)

yaox12
and
ko3n1g
authored
f145c98
build: bump TransformerEngine to release_v2.14 (dev) (#4332)

ko3n1g
and
claude
authored
ac6ca5b
[Dev] Add permute/unpermute fusion with dispatch/combine in Hybrid-EP (#4073)

Autumn1998
and
root
authored
f2a40ef
Fix fused grouped MLP wgrad hooks for DDP reduce-scatter (#4311)

authored
bd698d1
Fix activation_func check and MLP sharded_state_dict (#4325)

authored
73cc2ce
Allow fine-grained offloading with MC impl of full-CG. (#4253)

authored
6efb083

Commits on Apr 20, 2026

Add TEFusedDenseMLP for Dense+Grouped GEMM fusion on SM100+ (#4318)

authored
be3b874
[Dev] Fix docs build from main sync (#4356)

Dennis(Zhenhuan) Liu
and
claude
authored
1b47bc0
[Dev] overload factor logging (#4110)
nanz-nv
authored
55df4e5

Commits on Apr 21, 2026

Revert "fix mfsdp unwrap stuck at MegatronFSDP [dev] (#4273)" (#4393)
wplf
authored
13557a2
M4 leftover for TE cuda graph (dev) (#4369)

authored
546a448

Commits on Apr 22, 2026

[Dev] Add high-priority a2a comm stream option and hybridep preprocessing SMs (#4401)

gdengk
and
Gao Deng
authored
85bced0
Merge remote-tracking branch 'origin/main' into main2dev/22_04_2026
github-actions[bot]
committed
57005c8
chore: post-merge fixes for nightly sync main into dev (22_04_2026)

github-actions[bot]
and
claude
committed
8add4e4
fix: revert nvidia-resiliency-ext revision to match uv.lock

github-actions[bot]
and
claude
committed
baa3df4
fix: reformat 4 files with correct black==24.4.2 and isort==5.13.2

github-actions[bot]
and
claude
committed
bbb06e2
fix: restore missing ArgumentGroupFactory import in arguments.py

svcnvidia-nemo-ci
and
claude
committed
89798f3

Commits on Apr 23, 2026

Reorder mtp_post_process after attn backward in 1F1B schedule plan (#4430)

gdengk
and
Gao Deng
authored
db5ade5
chore: keep CODEOWNERS unchanged in main→dev sync

Phlip79
and
claude
committed
8cf5458
chore: update gpt3_mcore_te_tp2_pp2_mhc golden values for main→dev sync

Phlip79
and
claude
committed
fba3a80

Commits on Apr 25, 2026

[Dev] Fix mis-set decoupled gradient for Megatron-FSDP. (#4426)
cspades
authored
78858b2

Commits on Apr 28, 2026

chore: nightly sync main into dev (22_04_2026) (#4436)

authored
64d2e0a

Commits on Apr 29, 2026

Commits on Apr 30, 2026

Commits on May 3, 2026

feat(attention): Add rotary_base_per_layer for Step-3.5-Flash (#4473)

authored
a2d7153

Commits on May 6, 2026

test(transformer): add DSv4 hybrid hash MoE integration coverage (#4596)
Glitchfix
authored
7b570e6

Commits on May 7, 2026

Commits on May 8, 2026

Allow optimizer CG to share the same pool as full-iter CG (#4521)
nanz-nv
authored
829a7b7

Commits on May 10, 2026

Merge remote-tracking branch 'origin/main' into main2dev/10_05_2026
github-actions[bot]
committed
bfad45c

Commits on May 11, 2026

fix: post-CI corrections
github-actions[bot]
committed
9453c6e
fix: restore dev-only changes lost by main-priority merge
Phlip79
committed
bfce39d
fix
FDecaYed
committed
fb297f0
fix: rename args.hybrid_context_parallel to args.dynamic_context_parallel
Phlip79
committed
c3dbea7
fix: port wrap_data_iterator pattern from PR #4659 to fix DCP test
Phlip79
committed
e04e1c2
chore: nightly sync main into dev (10_05_2026) (#4716)

authored
d338cc5

Commits on May 12, 2026

Commits on May 13, 2026

Commits on May 14, 2026

Commits on May 15, 2026

[dev] Fix GDN DTensor splitting for FSDP checkpointing (#4799)
conver334
authored
df12802
[dev] [4/5] Qwen3.5 support: Interleaved MRoPE layout (#4750)

wplf
and
BestJuly
authored
cfbd9df
[DEV] fix(megatron-fsdp): preserve non-meta tensors during meta device materialization (#4155)

xuwchen
and
yaox12
authored
2672ff5
[dev] [3/5] Qwen3.5 support: SharedExpertMLP meta init (#4749)

wplf
and
BestJuly
authored
8195337
[Dev][feat] Support A2A Overlap for Megatron-FSDP (#3796)

Wohox
and
claude[bot]
authored
77c0f8c

Commits on May 18, 2026

[Dev][opt] Optimize e_proj and h_proj TP communication for MTP with mHC (#4817)

Baibaifan
and
yangfan.bai
authored
6139c51
[Dev][fix] FSDP EP-overlap CUDA-graph guard uses post-refactor API (#4834)

buptzyb
and
claude
authored
bac68ec

Commits on May 19, 2026

[Dev] Fix full CUDA graph capture reverted by pull main (#4792)
Dennis(Zhenhuan) Liu
authored
fad6c1e
[dev] [2/5] Qwen3.5 support: FSDP DTensor Bridge checkpoint compatibility (#4748)

authored
92ab682
[MXFP8] Mirror fixes in Mbridge for mxfp8 param gather (#4818)

zhongbozhu
and
WanZzzzzz
authored
b8ffa5e
Add opt-in nonuniform tensor parallelism (#4585)

daiyaanarfeen
and
FDecaYed
authored
e3ef5d1
Paged stashing updae (#4778)
nanz-nv
authored
bb525f1
Fix mxfp8 param gather numerical issue when DP overlap is off (#4769)
WanZzzzzz
authored
ee3f1ff
[dev] feat(attention): add use_head_wise_attn_gate for Step-3.5-Flash (#4841)

shifangx
and
claude
authored
bd5c98f

Commits on May 21, 2026

[Dev] add support for deepep/hybridep dispatcher under thd format training (#4816)

HaochenYuan
and
claude[bot]
authored
0afbb98
[dev] Fix FSDP TP metadata for LinearCrossEntropyModule (#4888)
conver334
authored
98b595c
[Dev] Add MoE example recipes (#4890)
Dennis(Zhenhuan) Liu
authored
6b6fb95

Commits on May 22, 2026

[Dev] Remove duplicated `bias_act_func` from main to dev sync (#4927)
yaox12
authored
56481b0

Commits on May 27, 2026

Commits on May 28, 2026

[Dev] Add Qwen3 30B MoE recipes (#5012)
Dennis(Zhenhuan) Liu
authored
7b9593b
[DEV] fix(megatron-fsdp): reduce padding for grouped expert weights (#5013)

xuwchen
and
yaox12
authored
473145c
[Dev] fix(combined-1f1b): release loss-node input storage after combined backward (#4908)

Wohox
and
yaox12
authored
3e8ce1f
[dev] [fix] [DeepSeek-v4] fix dense loss and rope type in DSv4 Hybrid Attention (#5018)

hxbai
and
yuzhongw-nvidia
authored
35f36c7

Commits on May 29, 2026

Commits on Jun 2, 2026

[Dev] Skip identity alltoall chunk sort (#5102)
lhb8125
authored
630956b

Commits on Jun 4, 2026

fix: correct dsv4_hybrid Q-up FLOPs by using args.v_head_dim (#5142)
dingqingy-nv
authored
60ef5e2
[dev] [follow-up] Qwen3.5 support: MoE aux loss padding_mask (#4776)

wplf
and
BestJuly
authored
05a93e0
[Dev] Support isolated MTP loss (#5080)

Dennis(Zhenhuan) Liu
and
liuzhenhai93
authored
d23ca85

Commits on Jun 5, 2026

[dev] moe(fix): Avoid TE cuda graph dummy attention masks (#5131)

yuzhongw-nvidia
and
Yuzhong Wang
authored
cefc252

Commits on Jun 6, 2026

chore: nightly sync main into dev (06_06_2026)

github-actions[bot]
and
claude
committed
ae1fc65
fix: post-CI corrections (docs dup fields, nvrx version gate, utils re-export)

github-actions[bot]
and
claude
committed
06f659b

Commits on Jun 7, 2026

[dev] [DeepSeek-v4] Add ClampedSwiGLU to MoE mlp_op_fuser and add force balance to hash routing (#5130)
hxbai
authored
ca1253c

Commits on Jun 8, 2026

Merge current dev into nightly sync
Phlip79
committed
12fc125
Minor improvements for Dynamic-cp (#4226)
xiaoyao0115
authored
959a542
fix: post-CI corrections
Phlip79
committed
fb1e6d7
Merge remote-tracking branch 'origin/dev' into main2dev/06_06_2026
Phlip79
committed
0af5708
varlendataset for thd e2e and benchmark (#4832)
xiaoyao0115
authored
39f8d55
fix: wrap mtp logging comment
Phlip79
committed
179df43

Commits on Jun 9, 2026

Commits on Jun 11, 2026

[Dev] DeepSeek-V4-Flash recipe 20260610 (#5266)
hxbai
authored
89ad096
[Dev] Generalized fix for mxfp8 param gather (#4994)

zhongbozhu
and
yaox12
authored
694cb0d
[Dev] Cherry-pick MTP detach heads (#5223)
Dennis(Zhenhuan) Liu
authored
7eedac5
[TE] Restore original CP group after dynamic CP forward in TEDotProductAttention (#5215)

rui23
and
rionawang
authored
0e0814a
[examples] Add dynamic context parallel benchmark example (#5123)
ilml
authored
192cbc3
[dev] Add experimental Megatron Lite as agentic exploration (#4885)

ISEEKYAN
and
FDecaYed
authored
08518ad

Commits on Jun 12, 2026

Commits on Jun 15, 2026

[Dev] Add MoE recipe performance summary (#5289)

Dennis(Zhenhuan) Liu
and
Dennis Liu
authored
e5ad098
[Dev] Add DeepEP v2 flex dispatcher backend (#4793)

Autumn1998
and
Dennis(Zhenhuan) Liu
authored
c049020
fix tflops calculation when sequence_packing_scheduler is not none (#5342)
xiaoyao0115
authored
15a4ffc
[dev] bump emerging optimizers to v0.3.0 (#5320)
FDecaYed
authored
3d8b95c
Merge branch 'dev' into main2dev/12_06_2026
Phlip79
authored
0302fac
fix: preserve seqlen stats in train_step
Phlip79
committed
95654c9

Commits on Jun 16, 2026

Enable Deepseek-v4 hybrid_model in dev branch Part (1/N) (#5042)

authored
377b2e0
chore: nightly sync main into dev (12_06_2026) (#5314)

authored
b6bea7e
feat(ssm): whole-module 'gdn' selective recompute for GatedDeltaNet (#5296)

authored
8dc6e66
[Fix] Fix optimizer parameter override bugs. (#5213)

authored
232c478
[Dev] Add Megatron-FSDP weight prefetch for full recompute (#5175)
lhb8125
authored
2047dec
[Dev] add cuda graph support for thd format training. (#4359)

HaochenYuan
and
Haochen Yuan
authored
7f91752
[Dev] restore DSv4 tflops calc in training and fix the packed seq case (#5358)
hxbai
authored
9af7c79

Commits on Jun 17, 2026

[dev] moe(perf): Restore fused GDN THD all-to-all on dev (#5389)
yuzhongw-nvidia
authored
3346fa8

Commits on Jun 22, 2026

Merge remote-tracking branch 'origin/main' into main2dev/22_06_2026

github-actions[bot]
and
claude
committed
c2a9a60

Commits on Jun 23, 2026

[Fix] Fix MoE router z-loss compatibility with TE CUDA Graph capture. (#5401)

Baibaifan
and
yangfan.bai
authored
3fae48f
test: mark ep_a2a_overlap activation-offloading test flaky_in_dev (#5450)

ko3n1g
and
claude
authored
94f70b6

Commits on Jun 24, 2026

test: mark TestParallelTransformerBlockCudagraphs::test_gpu_cudagraph flaky_in_dev (#5475)

ko3n1g
and
claude
authored
8c0aa6c
test: mark gated_delta_net selective-recompute test flaky_in_dev (#5476)

ko3n1g
and
claude
authored
6c6ce4e
fix: post-CI corrections for sync test/impl reconciliation

authored and
svcnvidia-nemo-ci
committed
cc083e1

Commits on Jun 25, 2026

fix sequence packing wrapper for eval (#5483)
xiaoyao0115
authored
15486f7
[dev] Megatron Lite (4/4) shared attention (#5427)
ISEEKYAN
authored
4463f78
fix: restore fused group MLP offload in main2dev sync (#5493)

lhb8125
authored and
svcnvidia-nemo-ci
committed
f6c77e7

Commits on Jun 26, 2026

Commits on Jun 27, 2026

[dev] moe(perf): Pre-GDR kernel fusion (#5361)
yuzhongw-nvidia
authored
d963266

Commits on Jun 29, 2026

Preserve DSA output across fused inverse RoPE (#5526)

kunlunl
and
shyoshyo
authored
04bbfe6
Enable Deepseek-v4 hybrid_model in dev branch Part (2/N) (#5485)
guihong-nv
authored
3e89f3c

Commits on Jun 30, 2026

[dev] Add experimental decoupled compact LayerWise DDP layout for Muon (#5388)

Wohox
and
claude
authored
3b9c248

Commits on Jul 1, 2026

[dev] Sync Megatron Lite with the latest implementation (#5577)
ISEEKYAN
authored
5e846ab

Commits on Jul 2, 2026

[Dev] fix padding mask docstring (#5598)
HaochenYuan
authored
66dc96f
Add GDN context parallel communication modes
yuzhongw-nvidia
committed
6031238
Make GDN conv padding configurable
yuzhongw-nvidia
committed
d749b6c
Refactor and rename GDN CP layout handling
yuzhongw-nvidia
committed
d8b0afa
Address GDN CP review guards
yuzhongw-nvidia
committed
df90299
Fix THD contiguous CP rank partition
yuzhongw-nvidia
committed
098d7d9
Require flash-linear-attention CP API
yuzhongw-nvidia
committed
3d39e26
Document THD CP routing future work
yuzhongw-nvidia
committed
a9fad1c
Update Mamba MoE config golden
yuzhongw-nvidia
committed
50a0a89
fix(thd): use padded boundaries for TE attention
yuzhongw-nvidia
committed
0beec47