[DEV] fix(megatron-fsdp): compute SWiGLU/GDN split in item coordinates for non-DTensor optimizer states - #4424

[DEV] fix(megatron-fsdp): compute SWiGLU/GDN split in item coordinates for non-DTensor optimizer states#4424

xuwchen wants to merge 2 commits into

NVIDIA:devNVIDIA/Megatron-LM:devfrom

xuwchen:fix_mfsdp_ckpt_save_devxuwchen/Megatron-LM:fix_mfsdp_ckpt_save_devCopy head branch name to clipboard