frost(sdpa): retire Capabilities.bottom_right_padded_seq_q

[Aneureka]

Before submitting

• I agree to license this contribution under the terms of LICENSE.txt.
• I ran pre-commit run and committed any formatting changes.
• I added GitHub labels: one cat-*, one or more mod-*, and one orig-* (see label list).

Affected area

• FE OSS kernels or CuTeDSL

Summary

1. Anchor the dense bottom-right diagonal per-batch: with a padding mask carrying per-batch seq_len_q, the BR diagonal now correctly anchors at (seq_len_q[b], seq_len_kv[b]) per cuDNN semantics instead of the global S_q for SM100-family SDPA engines; with the gap closed, we can retire Capabilities.bottom_right_padded_seq_q
2. Zero dead-tile O explicitly: the epilogues relied on 0 * garbage = 0 to zero rows whose KV loop was empty or collapsed, but the O TMEM is unwritten for such tiles and 0 * inf = NaN (pre-existing bug)

Why

To close kernel gap and retire bottom_right_padded_seq_q for cleaner capability declaration.

Related issues

Related to #379.

API and compatibility impact

• No public API change. Graphs with bottom-right causal + padding mask + per-batch seq_len_q now route to FROST on SM100 instead of native cuDNN.
• No performance impact: both new kernel branches are const_expr-folded — specializations without SEQ_Q_LENS_PRESENT+BOTTOM_RIGHT (diagonal) or without MAY_BE_EMPTY (zero-store) compile identically; the zero-store guard is one warp-uniform branch per tile otherwise. No compile-key or ABI change.
• Fixes wrong results on released code: NaN in dead O rows of padded d512 graphs with seq_len_q[b] < S_q.

Testing

On B200-class SM100 (cc 10.0), cuDNN backend 9.20, CUDA 13.0, torch 2.13, from test/python:

• pytest sdpa/frost/test_sdpa_graph_analyzer.py — 84 passed (includes the inverted probe test, now test_probe_accepts_bottom_right_with_padded_seq_len_q).
• pytest sdpa/frost/test_sdpa_fwd_dsl_sm100.py sdpa/frost/test_sdpa_fwd_split_kv_sm100.py — 533 passed, 3 failed: thd_cu_seq_len*, pre-existing on this 9.20 backend (cu_seq_len ports need 9.24+; identical failures on pristine develop).
• New coverage: test_dsl_sm100_graph_api_padded_bottom_right_gqa (formerly-declined graph shape end to end), test_dsl_sm100_bottom_right_padded_seq_q (4 flavors, BR+SWA+GQA+LSE, mid-tile trim / zero-length / collapsed tiles, dead rows asserted exactly zero); test_split_kv_padded_q_trim extended to d128+d512 with S_q=1024 so collapsed tiles exist (regression for the zero-store fix, top-left).
• NaN repro (4-flavor in-process kernel sequence, then d512 with short seq_len_q; BR, top-left, and top-left+SWA forms): 3/3 NaN before — also 3/3 on pristine develop, proving pre-existence — 9/9 clean after.
• Not tested: SM80/SM120 kernels (untouched; three constant spec lines removed; SM100-only box), mxfp8 suite (fails wholesale on this 9.20 backend, identically on pristine develop).

Summary by CodeRabbit

• New Features

  • Added support for dense bottom-right causal attention with per-batch query and key/value lengths on SM100.
  • Correctly handles trimmed query rows, empty ranges, and zero-filled outputs.
  • Expanded support across multiple head dimensions and masking configurations.

• Bug Fixes

  • Prevented invalid reads and incorrect results for empty or padded query rows.
  • Updated capability reporting and validation to allow supported configurations.

• Documentation

  • Clarified the capability flag describing per-batch LSE trimming requirements.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 792c674c-22e2-4746-8bb1-8e915baa82fc

📥 Commits

Reviewing files that changed from the base of the PR and between 55b2773 and 11ff5d7.

📒 Files selected for processing (11)

• python/cudnn/frost/README.md
• python/cudnn/sdpa/fwd/api_dsl.py
• python/cudnn/sdpa/fwd/engines.py
• python/cudnn/sdpa/fwd/kernels/_common_sm100.py
• python/cudnn/sdpa/fwd/kernels/prefill_d128_f16_sm100.py
• python/cudnn/sdpa/fwd/kernels/prefill_d192_d128_f16_sm100.py
• python/cudnn/sdpa/fwd/kernels/prefill_d256_f16_sm100.py
• python/cudnn/sdpa/fwd/kernels/prefill_d512_f16_sm100.py
• test/python/sdpa/frost/test_sdpa_fwd_dsl_sm100.py
• test/python/sdpa/frost/test_sdpa_fwd_split_kv_sm100.py
• test/python/sdpa/frost/test_sdpa_graph_analyzer.py

💤 Files with no reviewable changes (2)

• python/cudnn/sdpa/fwd/engines.py
• python/cudnn/sdpa/fwd/api_dsl.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Walkthrough

SM100 SDPA now supports dense bottom-right causal execution with per-batch query lengths. Kernel bounds, correction paths, references, graph eligibility, and tests now use per-batch lengths and handle dead query rows.

Changes

SM100 padded-query causal execution

Layer / File(s)	Summary
Eligibility and capability contract python/cudnn/sdpa/fwd/api_dsl.py, python/cudnn/sdpa/fwd/engines.py, python/cudnn/frost/README.md	SM100 validation and capability matching no longer reject bottom-right causal graphs with per-batch query lengths. The documented interaction flag is renamed to padded_stats.
Kernel length propagation and empty-tile handling python/cudnn/sdpa/fwd/kernels/_common_sm100.py, python/cudnn/sdpa/fwd/kernels/prefill_d*_f16_sm100.py	_resolve_seqlen_q reads and clamps per-batch query lengths for dense bottom-right causal execution. Prefill kernels pass the tensor through TMA, MMA, softmax, and correction paths. Correction epilogues produce zero output for empty KV ranges.
Reference behavior and execution coverage test/python/sdpa/frost/test_sdpa_fwd_dsl_sm100.py, test/python/sdpa/frost/test_sdpa_fwd_split_kv_sm100.py, test/python/sdpa/frost/test_sdpa_graph_analyzer.py	References and tests cover per-batch alignment, dead rows, LSE trimming, GQA, sliding windows, zero-length queries, direct API execution, graph eligibility, and d128/d512 split-KV execution.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 11ff5

The PR provides a localized SM100 SDPA correctness fix with targeted regression coverage; no actionable merge-blocking risk remains after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant SDPA_Graph
  participant SM100_Validation
  participant Prefill_Kernel
  participant _resolve_seqlen_q
  participant Output_Correction
  SDPA_Graph->>SM100_Validation: validate bottom-right causal graph
  SM100_Validation-->>SDPA_Graph: accept per-batch query lengths
  SDPA_Graph->>Prefill_Kernel: launch with seq_q_lens_tensor
  Prefill_Kernel->>_resolve_seqlen_q: resolve effective Q bounds
  _resolve_seqlen_q-->>Prefill_Kernel: return clamped per-batch Q length
  Prefill_Kernel->>Output_Correction: process tile output
  Output_Correction-->>Prefill_Kernel: zero empty-KV tiles

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title identifies the main change: removing and replacing the bottom-right padded-query capability for SDPA.
Description check	✅ Passed	The description completes all required sections and provides clear scope, compatibility impact, related issue, and detailed test results.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[Aneureka]

@cudnn-ci-bot run frost

[cudnn-ci-bot]

🚀 Running mirror pipeline

Branch: cudnn-gh/pr-683-11ff5d7
Pipeline: 63708860
Targets: frost