Conversation
Select the Rubin (sm_107 / cc 10.7) deep-gemm artifact directory at runtime, mirroring the enable_rubin split added for the trtllm-gen GEMM/BMM/MoE modules in PR flashinfer-ai#4191. - Add is_rubin_arch() / get_deepgemm_artifact_path() helpers and use them in load_all(), load(), and KernelMap.init_indices() so cubins and kernel_map.json are fetched from DEEPGEMM_RUBIN on Rubin. - Add KernelMap.KERNEL_MAP_HASH_RUBIN for the Rubin kernel_map.json. - Register DEEPGEMM_RUBIN in get_subdir_file_list()'s download list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jimmyzho
requested review from
aleozlx,
bkryu,
cyx-6,
kahyunnam,
saltyminty,
sricketts,
yongwww,
yyihuang and
yzh119
as code owners
July 30, 2026 03:35
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
jimmyzho
requested review from
Aneureka,
dhiraj113,
feih-nv,
jiahanc,
nv-yunzheq,
qiching,
qsang-nv and
samuellees
as code owners
July 30, 2026 03:35
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
5 tasks
aleozlx
added a commit
that referenced
this pull request
Jul 30, 2026
<!-- .github/pull_request_template.md --> ## 📌 Description This PR relands SM 107 support to main branch (reverted in #4171) as well as some other release fixes. #### Cherry Picks - #4191 - #4189 - #4200 - #4215 - #4225 - #4230 - #4235 - #4226 - #4257 - #4258 - #4261 #### Other Changes - Rubin guards from #4252's conflict resolution (`TLLM_RUBIN_FEATURES`: SiTuGlu static_asserts + tile-192 advertisement, compiled out for the Rubin BMM pin) - Test-contract update: `test_unified_moe.py` arch assertions written post-revert (#4159) flipped to the restored contract (FP4/BF16 claim 107; FP8 stays 100/103) <!-- What does this PR do? Briefly describe the changes and why they’re needed. --> ## 🔍 Related Issues <!-- Link any related issues here --> #4107, #4164, reverts #4171 ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [ ] Tests have been added or updated as needed. - [ ] All tests are passing (`unittest`, etc.). ## Reviewer Notes <!-- Optional: anything you'd like reviewers to focus on, concerns, etc. --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for Rubin/SM107 GPUs across GEMM, MoE, attention, quantization, sampling, and DeepGEMM workflows. * Added architecture-aware kernel selection, memory sizing, compilation, and artifact handling. * **Bug Fixes** * Improved validation and error messages for incompatible GPU architectures and invalid kernel configurations. * Clearly rejects unsupported NVFP4 KV-cache operations on SM107. * **Documentation** * Updated installation guidance with the SM107 architecture target. * **Tests** * Expanded architecture coverage and compatibility checks across GPU test suites. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Vinnie6167 <Vinnie6167@users.noreply.github.com> Co-authored-by: Ka-Hyun Nam <knam@nvidia.com> Co-authored-by: Alex Yang <aleyang@nvidia.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Jimmy Zhou <79552142+jimmyzho@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #4258 (which added
ArtifactPath.DEEPGEMM_RUBINand its checksum). That PR merged the path + checksum only; this PR adds the arch-based split that actually routes deep-gemm cubin loading to the Rubin directory on sm_107, mirroring theenable_rubinpattern from #4191.flashinfer/deep_gemm.pyis_rubin_arch()/get_deepgemm_artifact_path()helpers selectingDEEPGEMM_RUBINon arch107aandDEEPGEMMotherwise.load_all(),load(), andKernelMap.init_indices()so both the kernel cubins andkernel_map.jsonare fetched from the correct per-arch directory.KernelMap.KERNEL_MAP_HASH_RUBIN = f8bf2b1b…a36e27for the Rubinkernel_map.json(a separate manifest from the default one), selected by arch.flashinfer/artifacts.pyDEEPGEMM_RUBINtoget_subdir_file_list()'s download list soflashinfer artifacts downloadfetches it, and updated the DEEPGEMM comment to reference the Rubin map hash.Both Rubin hashes were verified by downloading the artifacts directly from the cubin repository.
Targets the
release-v0.6.16release branch.🤖 Generated with Claude Code