feat: add moe kernel support for non-glu#3558

NanoCode012 wants to merge 5 commits into

axolotl-ai-cloud:mainaxolotl-ai-cloud/axolotl:mainfrom

NanoCode012:feat/non-glu-kernelNanoCode012/axolotl:feat/non-glu-kernelCopy head branch name to clipboard