tests: xfail moe quantization classes mxfp8_bf16 UTs on sm103 - #1754
Conversation
Summary of ChangesHello @Quackens, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a temporary measure to mark certain Mixture-of-Experts (MoE) quantization unit tests as expected to fail ( Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request temporarily disables a test case for mxfp8_bf16 on sm103 to prevent regressions on B200. The change is straightforward and correct. I've suggested a minor refactoring to improve code style and robustness by using isinstance for type checking and comparing the compute capability tuple directly.
| if ( | ||
| type(moe_impl) is FP4Moe | ||
| and moe_impl.quant_mode == QuantMode.FP4_MXFP4_Bf16 | ||
| and compute_capability[0] == 10 | ||
| and compute_capability[1] == 3 | ||
| ): |
There was a problem hiding this comment.
For better code style and robustness, it's recommended to use isinstance() for type checking instead of type() is. isinstance() correctly handles inheritance. Additionally, comparing the compute_capability tuple directly with (10, 3) is more concise and readable.
if (
isinstance(moe_impl, FP4Moe)
and moe_impl.quant_mode == QuantMode.FP4_MXFP4_Bf16
and compute_capability == (10, 3)
):Expose the existing TRTLLM-gen `MxFP4xMxFP8` (W4A8) and `MxFP4xBf16` (W4A16) kernels through the unified MoE API. ## 📌 Description This is PR 3 in the unified MoE quantization series for FP8 support: 1. #4026 — unified block-scale FP8 (merged) 2. #4091 — unified per-tensor FP8 (merged) 3. This PR — unified TRTLLM MXFP4×MXFP8/W4A8 and MXFP4×BF16/W4A16 ### Changes - Generalize `TrtllmFp4RoutedRunner` beyond NVFP4: - `QuantVariant.MXFP4`: `MxE2m1` weights × `MxE4m3` activations - `QuantVariant.W4A16`: `MxE2m1` weights × BF16 activations - Add variant-aware TRTLLM FP4 preparation: - MXFP4 weights with 32-element UE8M0 scales - MXFP8 activation preparation for W4A8 - BF16 activation preparation for W4A16 - Add shape, dtype, and scale-layout validation. - Add unified conformance and fuzzer coverage for packed and `FromLogits` routing. ### Support matrix - NVFP4 and MXFP4/W4A8: SM100, SM103 - W4A16: SM100 only (remains disabled on SM103, matching upstream xfail #1754) - SM120/121: separate b12x backends where available - SM107 is unsupported after #4171 ### Scope - No CUDA/C++ kernel changes; both modes already exist in the TRTLLM flat API. - CUTLASS W4A8 is out of scope. ## 🚀 Pull Request Checklist ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. ## 🧪 Tests SM100, CUDA 13 CI container: - `tests/moe/test_unified_moe_mxfp4.py` - 21 passed - Unified fuzzer, packed routing: - seeds `900017,900018` - 2 passed - Unified fuzzer, `FromLogits`: - seeds `900019,900020` - 2 passed
📌 Description
Temporarily marking test_trtllm_gen_fused_moe mxfp8_bf16 cases as xfail until we converge on fix without causing regression on B200.
🔍 Related Issues
🚀 Pull Request Checklist
Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete.
✅ Pre-commit Checks
pre-commitby runningpip install pre-commit(or used your preferred method).pre-commit install.pre-commit run --all-filesand fixed any reported issues.🧪 Tests
unittest, etc.).Reviewer Notes