Skip to content

test: skip NVFP4-KV decode tests on SM107 - #4257

Merged
jimmyzho merged 1 commit into
flashinfer-ai:release-v0.6.16from
Vinnie6167:skip-nvfp4-kv-decode-tests-sm107
Jul 30, 2026
Merged

jimmyzho merged 1 commit into
flashinfer-ai:release-v0.6.16from
Vinnie6167:skip-nvfp4-kv-decode-tests-sm107

Conversation

@Vinnie6167

@Vinnie6167 Vinnie6167 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What

Adds a skip_if_nvfp4_kv_unsupported() helper to tests/attention/test_batch_decode_kernels.py and calls it at the top of test_batch_decode_with_paged_kv_cache_nvfp4, skipping the NVFP4-KV decode tests on SM107 (compute capability (10, 7)).

Why

NVFP4 KV-cache decode is deliberately unsupported on SM107: #4122 (Adds SM107 support) added guards in flashinfer/decode.py and flashinfer/prefill.py that raise ValueError("KV Cache NVFP4 is not supported on SM107") for any packed-FP4 (uint8) KV cache on that arch. These tests exercise the FA2 tensor-cores decode path (use_tensor_cores=True) with NVFP4 KV, so on Rubin hardware all 289 nvfp4 ids in this file currently fail with that exact ValueError instead of skipping:

[289 failed, 2970 passed] ValueError: KV Cache NVFP4 is not supported on SM107 (x289)

(full release-v0.6.16 sweep on VR200, 2026-07-28)

Notes

  • The skip keys on == (10, 7) to mirror the library guard exactly — it skips precisely where the library declares unsupported, nowhere else.
  • All nvfp4 entry points route through test_batch_decode_with_paged_kv_cache_nvfp4 (including the _large_head variant and the torch-compile smoke test), so the single call site covers all 289 ids.
  • Related but distinct: the trtllm-gen backend has its own NVFP4-KV gap on SM107 (published E2m1-KV FMHA kernels exist only for Sm100a/103a). That affects the trtllm-gen decode test files, not this one.
  • When NVFP4-KV support lands for SM107, lifting this skip belongs to the same change that lifts the library guard.

🤖 Generated with Claude Code

No NVFP4-KV (E2m1) decode kernels exist for SM107; the library already
raises ValueError("KV Cache NVFP4 is not supported on SM107") for this
configuration, so exercise the tests only where the kernels exist. The
skip mirrors that guard and covers all nvfp4 entry points in the file.
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 070e7d14-54af-41ff-bcfa-24d42cc4af61

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jimmyzho
jimmyzho merged commit efd289a into flashinfer-ai:release-v0.6.16 Jul 30, 2026
4 checks passed
aleozlx added a commit that referenced this pull request Jul 30, 2026
<!-- .github/pull_request_template.md -->

## 📌 Description

This PR relands SM 107 support to main branch (reverted in #4171) as
well as some other release fixes.

#### Cherry Picks
- #4191
- #4189
- #4200
- #4215
- #4225
- #4230
- #4235
- #4226
- #4257
- #4258 
- #4261

#### Other Changes
- Rubin guards from #4252's conflict resolution (`TLLM_RUBIN_FEATURES`:
SiTuGlu
static_asserts + tile-192 advertisement, compiled out for the Rubin BMM
pin)
- Test-contract update: `test_unified_moe.py` arch assertions written
post-revert
(#4159) flipped to the restored contract (FP4/BF16 claim 107; FP8 stays
100/103)

<!-- What does this PR do? Briefly describe the changes and why they’re
needed. -->

## 🔍 Related Issues

<!-- Link any related issues here -->

#4107, #4164, reverts #4171

## 🚀 Pull Request Checklist

Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.

### ✅ Pre-commit Checks

- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.

> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).

## 🧪 Tests

- [ ] Tests have been added or updated as needed.
- [ ] All tests are passing (`unittest`, etc.).

## Reviewer Notes

<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for Rubin/SM107 GPUs across GEMM, MoE, attention,
quantization, sampling, and DeepGEMM workflows.
* Added architecture-aware kernel selection, memory sizing, compilation,
and artifact handling.
* **Bug Fixes**
* Improved validation and error messages for incompatible GPU
architectures and invalid kernel configurations.
  * Clearly rejects unsupported NVFP4 KV-cache operations on SM107.
* **Documentation**
  * Updated installation guidance with the SM107 architecture target.
* **Tests**
* Expanded architecture coverage and compatibility checks across GPU
test suites.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Vinnie6167 <Vinnie6167@users.noreply.github.com>
Co-authored-by: Ka-Hyun Nam <knam@nvidia.com>
Co-authored-by: Alex Yang <aleyang@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Jimmy Zhou <79552142+jimmyzho@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants