fix(megatron-fsdp): preserve non-meta tensors during meta device materialization - #4154

fix(megatron-fsdp): preserve non-meta tensors during meta device materialization#4154

xuwchen wants to merge 4 commits into

NVIDIA:mainNVIDIA/Megatron-LM:mainfrom

xuwchen:fix_mfsdp_meta_device_initxuwchen/Megatron-LM:fix_mfsdp_meta_device_initCopy head branch name to clipboard