fix(gemm): report a failed cuDNN tactic as unsupported during profiling (not silent tactic=-1)

[YangXu1990uiuc]

📌 Description

The cuDNN GEMM runners (FP8 / BF16 / FP4 / MXFP8 / BF16×FP4) silently execute tactic=-1 when a requested tactic can't be used — both when resolving an engine/knob tactic to a plan index (no match / out-of-range) and when a tactic errors at execute time. That is correct for serving robustness, but during autotuner profiling it attributes the fallback's timing to the requested tactic. Consequences:

• a tactic that never actually ran can win the autotune (it's timed as the fast default);
• a genuinely faster tactic can be masked by the default's time;
• the tuner then persists a "winner" that always falls back at serving — violating serve what was tuned, and validate_tactic-style checks can't catch it because the tactic "works" (via fallback).

Fix

Report a failed tactic as unsupported during profiling (re-raise), so the autotuner marks it inf and disqualifies it; keep the warn-and-fall-back path for serving. The profiling scope is already detectable via is_in_profile_measurement(), and choose_one already turns a raised tactic into inf + skip — so this is a small per-site change, not an autotuner rework.

8 sites: 2 in the tactic→plan_index resolver (_get_cudnn_plan_index_for_tactic: no-match, out-of-range) + 6 runner-forward except clauses (fp8/bf16/mxfp8×2/fp4 in gemm_base.py, bf16×fp4 in gemm_bf16_fp4_cudnn.py).

🔍 Related Issues

Raised in the #3707 review ("Do not treat a failed tactic's fallback execution as a successful tactic during profiling") and deferred there — the concern at the time was that it "would require a relatively broad autotuner change." Re-examined: is_in_profile_measurement() has existed since #3252 (before #3707), and the autotuner already marks a raised tactic inf, so the change is local to the runner fallback sites. Independently re-surfaced during a CUDA-graph-compatibility audit of the autotuner-v2 work (#3861).

🧪 Tests

tests/autotuner/test_cudnn_profiling_fallback.py (GPU, SM90+): an out-of-range tactic on the real cuDNN FP8 runner warns + falls back + produces valid output outside profiling, and raises (so it's disqualified) inside a _profile_measurement_scope(). Verified on SM100. Serving behavior is unchanged.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved cuDNN GEMM autotuning by correctly marking failed or unsupported tactics during profiling.
  • Preserved serving behavior by warning and falling back to a default tactic when execution fails.

• Tests

  • Added coverage validating distinct tactic-failure behavior during profiling and serving.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[coderabbitai]

📝 Walkthrough

Walkthrough

cuDNN GEMM tactic resolution and runner failures now raise during AutoTuner profiling, while serving continues warning and falling back to tactic=-1. A CUDA FP8 regression test verifies both behaviors.

Changes

cuDNN profiling failure handling

Layer / File(s)	Summary
Tactic plan resolution flashinfer/gemm/gemm_base.py	Plan mismatches and out-of-range tactic indexes raise during profiling and retain warning-based default-tactic fallback otherwise.
Runner exception propagation flashinfer/gemm/gemm_base.py, flashinfer/gemm/gemm_bf16_fp4_cudnn.py	Supported cuDNN GEMM runners re-raise execution failures during profiling while preserving serving fallback behavior.
Profiling and serving regression coverage tests/autotuner/test_cudnn_profiling_fallback.py	CUDA FP8 GEMM tests verify invalid tactics fall back during serving and raise during profiling.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AutoTuner
  participant CudnnFp8GemmRunner
  participant cuDNN
  AutoTuner->>CudnnFp8GemmRunner: execute requested tactic
  CudnnFp8GemmRunner->>cuDNN: resolve and run tactic plan
  cuDNN-->>CudnnFp8GemmRunner: tactic failure
  alt Profiling
    CudnnFp8GemmRunner-->>AutoTuner: re-raise RuntimeError
  else Serving
    CudnnFp8GemmRunner-->>AutoTuner: warn and use tactic=-1
  end

Loading

Possibly related PRs

• flashinfer-ai/flashinfer#3260: Both modify cuDNN tactic mismatch and fallback handling in GEMM runner paths.
• flashinfer-ai/flashinfer#3437: Both adjust cuDNN FP8/FP4 tactic-failure behavior.
• flashinfer-ai/flashinfer#3707: Both modify cuDNN tactic-to-plan failure handling and runner fallback paths.

Suggested labels: run-ci

Suggested reviewers: aleozlx, yzh119, dhiraj113, saltyminty, jiahanc

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the core change: failed cuDNN tactics are treated as unsupported during profiling while serving fallback remains.
Description check	✅ Passed	The description covers the change, motivation, related issues, and testing; only the checklist and reviewer notes are not fully filled out.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/autotuner/test_cudnn_profiling_fallback.py`:
- Around line 26-27: Update the output allocation in the test setup around the
existing out tensor to initialize every element to NaN instead of leaving it
uninitialized, while preserving its CUDA device and bfloat16 dtype. Apply the
same initialization to the additional output allocation noted in the comment so
the subsequent isfinite assertions reliably detect whether fallback populated
the results.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: efcbbcee-d85e-49c9-8a1b-23076b329c88

📥 Commits

Reviewing files that changed from the base of the PR and between 817e4bd and 8a3ffac.

📒 Files selected for processing (3)

• flashinfer/gemm/gemm_base.py
• flashinfer/gemm/gemm_bf16_fp4_cudnn.py
• tests/autotuner/test_cudnn_profiling_fallback.py

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Initialize the output with NaNs before asserting fallback output.

torch.empty may already contain finite values, allowing this assertion to pass if fallback never writes out. Initialize out to NaNs so isfinite() proves the fallback populated every result element.

Suggested fix

-    out = torch.empty([1, m, n], device="cuda", dtype=torch.bfloat16)
+    out = torch.full(
+        [1, m, n], float("nan"), device="cuda", dtype=torch.bfloat16
+    )

Also applies to: 54-55

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/autotuner/test_cudnn_profiling_fallback.py` around lines 26 - 27,
Update the output allocation in the test setup around the existing out tensor to
initialize every element to NaN instead of leaving it uninitialized, while
preserving its CUDA device and bfloat16 dtype. Apply the same initialization to
the additional output allocation noted in the comment so the subsequent isfinite
assertions reliably detect whether fallback populated the results.