Revert "[AMD][Quantization] Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" - #28213
Conversation
…FP4 requ…" This reverts commit 3f4a338.
github-actions
Bot
added
documentation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request removes the online requantization feature from FP8 to MXFP4, simplifying the codebase by deleting unused dequantization utilities, removing requantization configuration options, and refactoring the weight creation methods in FP8 and Quark quantization schemes. A critical issue was identified in quark_w4a4_mxfp4_moe.py where calling scale_param.weight_loader inside the online weight loader causes infinite recursion and crashes model loading, which can be resolved by calling original_weight_loader directly.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| scale_param.weight_loader( | ||
| scale_param, weight_scale, scale_weight_name, shard_id, expert_id | ||
| ) |
There was a problem hiding this comment.
In online_mxfp4_moe_weight_loader, calling scale_param.weight_loader results in infinite recursion and a crash during model loading. This is because extra_weight_attrs["weight_loader"] was overwritten with the online weight loader (online_mxfp4_moe_weight_loader) before the scale parameters were registered, meaning scale_param.weight_loader points back to online_mxfp4_moe_weight_loader itself.\n\nTo fix this, call original_weight_loader directly to load the scale parameter, bypassing the online quantization wrapper.
original_weight_loader(\n scale_param, weight_scale, scale_weight_name, shard_id, expert_id\n )
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
/tag-and-rerun-ci |
…FP4 requantization on AMD GPUs" (sgl-project#28213)
…FP4 requantization on AMD GPUs" (sgl-project#28213)
1 participant
Reverts #18182
it breaks the ci
https://github.com/sgl-project/sglang/actions/runs/27494298180/job/81265388307#step:6:260
See a correct fix here #28191
CI States
Latest PR Test (Base): 🚫 Run #27511029332
Latest PR Test (Extra): ❌ Run #27511029236