[main] fix(moe): Fix several bugs for DSA rope and spec. - #3026

[main] fix(moe): Fix several bugs for DSA rope and spec.#3026

yuzhongw-nvidia merged 18 commits into

NVIDIA:mainNVIDIA/Megatron-LM:mainfrom

yuzhongw-nvidia:yuzhongw/fix_exp_specyuzhongw-nvidia/Megatron-LM:yuzhongw/fix_exp_specCopy head branch name to clipboard