Conversation
No NVFP4-KV (E2m1) decode kernels exist for SM107; the library already
raises ValueError("KV Cache NVFP4 is not supported on SM107") for this
configuration, so exercise the tests only where the kernels exist. The
skip mirrors that guard and covers all nvfp4 entry points in the file.
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
<!-- .github/pull_request_template.md --> ## 📌 Description This PR relands SM 107 support to main branch (reverted in #4171) as well as some other release fixes. #### Cherry Picks - #4191 - #4189 - #4200 - #4215 - #4225 - #4230 - #4235 - #4226 - #4257 - #4258 - #4261 #### Other Changes - Rubin guards from #4252's conflict resolution (`TLLM_RUBIN_FEATURES`: SiTuGlu static_asserts + tile-192 advertisement, compiled out for the Rubin BMM pin) - Test-contract update: `test_unified_moe.py` arch assertions written post-revert (#4159) flipped to the restored contract (FP4/BF16 claim 107; FP8 stays 100/103) <!-- What does this PR do? Briefly describe the changes and why they’re needed. --> ## 🔍 Related Issues <!-- Link any related issues here --> #4107, #4164, reverts #4171 ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [ ] Tests have been added or updated as needed. - [ ] All tests are passing (`unittest`, etc.). ## Reviewer Notes <!-- Optional: anything you'd like reviewers to focus on, concerns, etc. --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for Rubin/SM107 GPUs across GEMM, MoE, attention, quantization, sampling, and DeepGEMM workflows. * Added architecture-aware kernel selection, memory sizing, compilation, and artifact handling. * **Bug Fixes** * Improved validation and error messages for incompatible GPU architectures and invalid kernel configurations. * Clearly rejects unsupported NVFP4 KV-cache operations on SM107. * **Documentation** * Updated installation guidance with the SM107 architecture target. * **Tests** * Expanded architecture coverage and compatibility checks across GPU test suites. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Vinnie6167 <Vinnie6167@users.noreply.github.com> Co-authored-by: Ka-Hyun Nam <knam@nvidia.com> Co-authored-by: Alex Yang <aleyang@nvidia.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Jimmy Zhou <79552142+jimmyzho@users.noreply.github.com>
What
Adds a
skip_if_nvfp4_kv_unsupported()helper totests/attention/test_batch_decode_kernels.pyand calls it at the top oftest_batch_decode_with_paged_kv_cache_nvfp4, skipping the NVFP4-KV decode tests on SM107 (compute capability(10, 7)).Why
NVFP4 KV-cache decode is deliberately unsupported on SM107: #4122 (Adds SM107 support) added guards in
flashinfer/decode.pyandflashinfer/prefill.pythat raiseValueError("KV Cache NVFP4 is not supported on SM107")for any packed-FP4 (uint8) KV cache on that arch. These tests exercise the FA2 tensor-cores decode path (use_tensor_cores=True) with NVFP4 KV, so on Rubin hardware all 289 nvfp4 ids in this file currently fail with that exact ValueError instead of skipping:(full release-v0.6.16 sweep on VR200, 2026-07-28)
Notes
== (10, 7)to mirror the library guard exactly — it skips precisely where the library declares unsupported, nowhere else.test_batch_decode_with_paged_kv_cache_nvfp4(including the_large_headvariant and the torch-compile smoke test), so the single call site covers all 289 ids.🤖 Generated with Claude Code