Skip to content

[main] perf(moe): Refine gated delta net implementation - #3042

Draft
yuzhongw-nvidia wants to merge 263 commits into
NVIDIA:mainfrom
yuzhongw-nvidia:yuzhongw/refine_gdn_impl
Draft

[main] perf(moe): Refine gated delta net implementation#3042
yuzhongw-nvidia wants to merge 263 commits into
NVIDIA:mainfrom
yuzhongw-nvidia:yuzhongw/refine_gdn_impl

refine gdn impl: (1) use fla conv1d; (2) combine l2norm to query and key

4123fb6
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar