Changes to support latent MoEs#2296

Merged

pablo-garay merged 9 commits intoNVIDIA:mainfrom

deepakn94:dnarayanan/latent_moe

Dec 8, 2025

Commits on Nov 22, 2025

Changes to support latent MoEs

venmugil
authored and
deepakn94
committed
Modify throughput equations for hybrid models to account for --moe-latent-size
deepakn94
committed
Move fc2_latent_proj into combine method to make sure tensors are the right shape
deepakn94
committed

Commits on Dec 5, 2025

Move fc1_latent_proj into router_and_preprocess method to make sure the MoE routing happens in hidden dimension and correct tensor shape is captured in token dispatcher
venmugil
committed
Merge branch 'main' into dnarayanan/latent_moe
deepakn94
authored

Commits on Dec 6, 2025

Commits on Dec 7, 2025

Move attribute to right place in config
deepakn94
committed

Commits on Dec 8, 2025

Merge branch 'main' into dnarayanan/latent_moe
ericharper
authored