Cherry-pick unit, wheel, infra, and Rubin follow-ups onto release-v0.6.18 - #4764
Merged
Merged
Conversation
Alternate the math/IO barrier ids by tile parity so a leading math warp cannot double-arrive one phase (#3700).
…107)" This reverts commit c3b96d0.
(cherry picked from commit 9a0e83b)
## 📌 Description CUDA 13.4 changes `nvcc --dryrun` output in a way that sccache v0.17.0 parses incorrectly. The missing compile steps later surface as `fatbinary` failures because the expected cubins were never produced. - Bypass sccache for cu134 NVCC invocations while keeping host C++ compilation cached. - Accept both CUDA version forms used by the release/nightly (`13.4`) and PR (`134`) build paths. - Trigger the Release dry-run matrix when the shared JIT-cache helper changes, so both cu134 architecture jobs exercise this workaround before merge. - Remove the guard once the pinned sccache release includes the upstream CUDA 13.3+ fix. ## 🔍 Related Issues - Upstream fix: [mozilla/sccache#2722](mozilla/sccache#2722) - Failing nightly: [run 32544126473](https://github.com/flashinfer-ai/flashinfer/actions/runs/32544126473) ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [x] Tests have been added or updated as needed. - [x] All tests are passing (`unittest`, etc.). Focused validation passed: - `bash -n scripts/jit_cache_build_common.sh` - `shellcheck scripts/jit_cache_build_common.sh` - Mocked launcher checks for CUDA `13.4`, `134`, and `13.0` - Release workflow YAML parse - `pre-commit run --all-files` - `git diff --check` ## Reviewer Notes This intentionally disables only the affected NVCC launcher for cu134. The sccache server and host C++ launcher remain enabled so safe cache hits are preserved. The Release workflow should provide the end-to-end cu134 x86_64 and aarch64 validation. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved CUDA 13.4 build compatibility by avoiding an incompatible compiler-cache path. * Preserved compiler caching for other supported CUDA versions. * **Chores** * Updated release automation to recognize changes affecting shared build tooling. * Added clearer build logs showing which compiler-cache launchers are enabled. <!-- end of auto-generated comment: release notes by coderabbit.ai --> (cherry picked from commit cf9a04d)
(cherry picked from commit ac1c275)
## 📌 Description Keep `test_supported_jit_cache_versions_match_cuda_config` active in source-tree presubmit runs, but skip it when `ci/cuda-versions.json` is intentionally absent from the isolated installed-package layout used by Nightly Release. This fixes the identical cu129/cu130 shard 5 failures in Nightly Release #360 without copying repository-only CI metadata into the installed-package test directory. ## 🔍 Related Issues - Nightly Release #360: https://github.com/flashinfer-ai/flashinfer/actions/runs/32920335499 ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [x] Tests have been added or updated as needed. - [x] All tests are passing (`unittest`, etc.). Validation performed: - `python3 -m compileall -q tests/cli/test_cli_cmds.py` - `pre-commit run --files tests/cli/test_cli_cmds.py` - `git diff --check` Targeted pytest was not run locally because the existing repository virtualenv does not contain a usable pytest installation. PR CI should confirm both the source-tree pass and isolated-nightly skip behavior. ## Reviewer Notes The test still protects the duplicated CLI/config CUDA-version invariant during presubmit. The skip applies only when the repository-only `ci/cuda-versions.json` file is unavailable, as it is in the Nightly Release installed-wheel test directory. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved CUDA version consistency testing. * Tests now skip gracefully when the required configuration file is unavailable. * Added support for running tests with `pytest`. <!-- end of auto-generated comment: release notes by coderabbit.ai --> (cherry picked from commit e4b7fa4)
Drop SM75 from all cache wheels and SM121a from the cache arch lists so published jit-cache artifacts stay under GitHub's release-asset limit.
…uilds (#4760) Skip the single-request attention modules in AOT jit-cache so those kernels stay JIT-only and the cache wheel stays smaller.
attention-ts context and decode reject compute capability 10.7 outright:
NotImplementedError: attention-ts context requires an SM100a/B200 or
SM103a/B300 GPU; device cuda:0 has compute capability (10, 7)
The exclusion carries no rationale. Unlike the tinygemm2 capability tuple it
has no explaining comment, the module does not declare native_only=True (the
mechanism for kernels that need arch-specific tcgen05 instructions), and its
tcgen05 usage is limited to tcgen05_alloc/dealloc, which are family-portable.
Widen both tuples to include (10, 7) and gate on the CuTe DSL arch, so a DSL
older than 4.8 without CUTE_DSL_ARCH=sm_100f gets the actionable message from
require_cute_dsl_arch instead of KeyError: sm_107a during compilation.
Measured on SM107 hardware, tests/attention/test_attention_ts_context.py:
test_attention_ts_context_plan_rejects_critical_public_contracts
before: 3 failed, 2 passed
after : 5 passed
That test is a negative test -- it feeds deliberately invalid inputs and
asserts plan() rejects each with a specific message. plan() only validates and
builds scheduling metadata, so it never compiles a kernel; with the gate closed
the device check short-circuits the contract checks mid-sequence, which is why
two cases passed and three reported the arch error instead.
KNOWN LIMITATION: this fixes the gate, not attention-ts on Rubin. The decode
suite is unchanged at 4 failed / 45 passed / 76 skipped before and after,
because those four fail earlier at kernel construction in the exhaustive
deadlock/race checker -- a failure that also reproduces on B200 (cc 10.0) and
on SM107 with stock code, so it is not Rubin-specific. Landing this alone
trades a clear NotImplementedError for an opaque checker ValueError on the
decode path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit dd0646f)
Decline Rubin MoE tactics when the CuTe DSL lacks rubin_helpers, and pin the multi-arch Rubin batched-GEMM cubin package.
…ass (#4753) Move availability probes to a cutlass-free module so import flashinfer does not hard-require nvidia-cutlass-dsl. Paths that do not exist on 0.6.18 (kda_prefill_cute, SM120 KDA prefill runtime, attn_scores) are omitted.
kahyunnam
requested review from
Aneureka,
IwakuraRein,
StudyingShao,
aleozlx,
bkryu,
cyx-6,
dhiraj113,
feih-nv,
jiahanc,
jimmyzho,
nv-yunzheq,
qsang-nv,
saltyminty,
samuellees,
sricketts,
yongwww,
yyihuang and
yzh119
as code owners
August 26, 2026 23:47
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kahyunnam
added
op: linear attention
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-picks onto
release-v0.6.18(after #4762), one commit per source PR (plus a revert of the old #4577 yaml pin so #4469 can land).Unit tests
is_cute_dsl_experimental_availableonly. 0.6.18 has nokda_prefill_cute/ feat(kda): add CuTe DSL recurrent prefill backend #4605 auto CuTe prefill path (recurrent_kdaalready uses Cake), so the main-line kda.py/tests hunks do not apply.Wheel size
Infra
release.yml/nightly-release.ymlcu134 matrix (conflicts with ci: consolidate CUDA coverage and validate candidate images #4469).ci/cuda-versions.json(includes cu134 + 10.7a, so Rubin jit-cache coverage is kept after the revert). Nightly dual-upload to the legacy + wheel repos is preserved. DSL image pin stays 4.6.2 with the Rubin skip from ci: keep Rubin cute-dsl pins after the 4.6.2 CI pin #4762.>=4.6.2a0extras,ci_image_specifier ==4.6.2) instead of main’s 4.7 floors.Rubin
flashinfer.cute_dsl.availability. Modules that do not exist on 0.6.18 (kda_prefill_cute.py,kda_kernels/sm120_prefill/runtime.py,attn_scores) are not resurrected.Test plan
python3 ci/validate_cuda_versions.py(already run locally: 2 runtime images, 3 JIT-cache targets)nvidia-cutlass-dsl 4.6.2; RubinPREPARE_RUBIN_TEST_IMAGE=1left in place10.7a; SM75/SM121a gone