[models] fix: accept Gemma4 router input ids - #3835
Conversation
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Light Code ReviewMCore dev bump + Gemma4 router compatibility fix. The change is minimal and correct: Gemma4TopKRouter.routing() now accepts and forwards the new input_ids kwarg that MCore TopKRouter.routing() started passing through. No issues found. The code change is straightforward:
Test coverage gap: Gemma4TopKRouter.routing() has no unit test. There are no tests in tests/unit_tests/models/gemma/ that instantiate Gemma4TopKRouter or call its routing() method. While this specific change is safe (just passing through a kwarg), the renormalization + per_expert_scale logic in routing() is also untested. Consider adding a unit test that verifies: (1) routing() accepts input_ids without error, and (2) the renormalization and per_expert_scale application produce correct outputs. Lock file: The uv.lock diff is large but consists entirely of transitive dependency version bumps from the MCore submodule update. Notable changes: torch 2.11 to 2.12, wandb 0.26 to 0.27, langchain-openai removed (replaced by langchain-nvidia-ai-endpoints via nvidia-resiliency-ext), ssm and te extras folded into dev/lts. No pyproject.toml changes in this PR, so the lock changes are expected to come from the submodule. Suggested test cases: No perf tests impacted. |
|
/ok to test c36259c |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test e4d4802 |
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: dimapihtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Vasudevan Rengasamy <vrengasamy@nvidia.com>
Mirrors #3820 for MB-365 and adds the Gemma4 router compatibility fix needed for the MCore dev bump. MCore dev now forwards
input_idsintoTopKRouter.routing(), soGemma4TopKRouterneeds to accept and pass that argument through to the parent routing implementation.Original bump PR: #3820
Linear: MB-365
Validation:
python3 -m py_compile src/megatron/bridge/models/gemma/gemma4_provider.pyUV_CACHE_DIR=/tmp/uv-cache-yuya uv run --no-sync ruff check src/megatron/bridge/models/gemma/gemma4_provider.py11767705: Gemma/Gemma-VL unit subset passed,128 passed, 34 warnings in 2.38s11767727:tests/functional_tests/launch_scripts/h100/active/L0_Launch_recipes_gemma4_vl.shpassed,1 passed, 929 warnings in 119.55s; Slurm stateCOMPLETED, exit0:0Note: cw did not expose a GB200 partition in
sinfo, so the Gemma4-VL functional rerun used the H100 active launch script for the same failing recipe group.