Skip to content

feat(rocke): port the SDPA backward resource refusals to C++ (AICK-3015) - #15046

Draft
afagaj wants to merge 17 commits into
users/jafagani/rocke-bwd-cpp-stackfrom
users/jafagani/rocke-bwd-cpp-refusal-tables
Draft

afagaj wants to merge 17 commits into
users/jafagani/rocke-bwd-cpp-stackfrom
users/jafagani/rocke-bwd-cpp-refusal-tables

Conversation

@afagaj

@afagaj afagaj commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

JIRA ID : AICK-3015

Summary: The C++ SDPA backward builders now refuse the same resource-gate specs Python refuses, using a C++ port of spec_digest and of both refusal tables.

Stacked on #14365; retarget to develop after it merges.

Motivation

Review comment on #14365: #14365 (comment)

Python keeps two tables of specs that compile but fail the resource gate (spills, AGPR copy traffic) or crash the code-object compiler: RESOURCE_REFUSED (2,283 main-kernel rows) and DQ_RESOURCE_REFUSED (144 dQ rows), both keyed by (arch, spec_digest(spec, arch)). In #14365 the C++ main builder skipped the main table (it mirrored _emit_main(..., main_kernel_refusals=False)) and the C++ dQ validator skipped the dQ table. Only comments said so, so a future C++ caller would build and launch those kernels without noticing.

Change

  • platform/cpp/instances/common/_attention_bwd_refusals.cpp (new): sha256_hex, spec_digest / dq_spec_digest (the existing non_default_fields plus head_size, sorted, Python repr, SHA-256, first 16 hex digits), and binary-search lookups resource_refusal / dq_resource_refusal.
  • _attention_bwd_refusals_table.inc (new, generated): both tables, written by library/tests/parity/attention_bwd_refusals_emit.py --print-c-table.
  • Main validator and builder take main_kernel_refusals, with no default, so every caller has to choose:
    • rocke_attn_bwd_validate_spec(spec, arch, main_kernel_refusals, out, err) and rocke_build_attn_bwd_main_new(b, spec, arch, main_kernel_refusals). With true (Python's default) a refused spec is ROCKE_ERR_VALUE with Python's exact message (resource gate: this resolved spec fails on <arch> (<record>; RESOURCE_REFUSED)).
    • The dense and unified shells pass false, as their Python twins do. The IR parity emitters (attention_bwd_main/body/spec_emit.c) pass false, matching their Python side.
  • The dQ validator always applies DQ_RESOURCE_REFUSED between DQ_SPILLING_COMBOS and DQ_SCHED_SPILLING, in Python's order. The dense dQ validator inherits this, as in Python.
  • New C entry points: rocke_attn_bwd_spec_digest, rocke_attn_bwd_resource_refusal, rocke_attn_bwd_dq_spec_digest, rocke_attn_bwd_dq_resource_refusal.
  • Headers, attention_bwd.md section 0.3 and the parity-emitter comments now describe the port instead of the gap.

Testing

  • New parity pair attention_bwd_refusals_emit.{py,c}: 68 configs (11 main/dQ defaults, knob specs and early rejections, plus 50 main and 7 dQ recorded refusals covering every target, every failure kind and every spec field that appears in the recorded rows). Each config prints the spec digest, the resolved digest, the table record and the validator verdicts. C and Python agree on all 68. Enrolled in NativeGoldenTests.cmake, with goldens recorded at llvm20/22/23/24.
  • New library/tests/test_attention_bwd_refusals_cpp_table.py: the committed .inc must equal the generator output (so all 2,283 + 144 rows match).
  • check_byte_identity.py --only attention_bwd,attention_dense_bwd (llvm22): GREEN, 21 families, including attention_bwd_refusals.
  • ctest: 50/50 passed (includes rocke_native_golden_ll).
  • run_diff.py --python-only --check-golden at llvm20/22/23/24: GOLDEN OK, 0 unblessed changes.
  • pytest test_attention_bwd_refusals_cpp_table.py, test_attention_bwd_refusals.py, test_attention_bwd_layout_spill_rows.py: all passed.
  • pre-commit run --from-ref <stack head> --to-ref HEAD: clean.
  • Negative checks:
    • Making the C++ main validator ignore the table made rocke_native_golden_ll fail on attention_bwd_refusals at all four flavors. Restoring it brought it back to pass.
    • Deleting one row from the .inc failed both table tests. Restoring it brought them back to pass.

🤖 Generated with Claude Code

afagaj and others added 14 commits November 4, 2026 01:43
…ward (AICK-2762)

Python builds each attention backward kernel (general, dense, unified
atomic/direct; gfx942 and gfx950; shipped defaults and every tuned-table
entry), both engines lower it, and the .ll must match byte for byte at
every golden LLVM flavor.

ws_bounds='buffer' on the general kernel is also pinned (one table key per
arch): it is the only knob value that emits tile.buffer_load.

The test needs the rocke_engine binding and skips without it. It is in no
CTest list, so it is a local gate and never runs in CI. CI's check of the
C++ side is the ctest rocke_native_golden_ll (the CI-check commit).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The attention backward kernels emit tile.buffer_load with
ws_bounds='buffer'. Its C++ HIP lowering had no check that runs in CI:
test_buffer_load_scalar.py compares both engines, but skips its C++ side
without the rocke_engine binding, which the CI test tree does not build.

- New ctest rocke_buffer_load_hip lowers the op through the C++ HIP route
  for f16, bf16, f32 and i32 on gfx942 and gfx950. It requires the two
  full lines of Python's _op_tile_buffer_load template, spelled with the
  kernel's own value names: the raw_buffer_load_b16 / _b32 call with its
  operands in order (rsrc, voffset, soffset, 0), and the memcpy into the
  result. A swapped or dropped operand fails it. It is in the quick and
  ffm-quick tiers of ROCKE_ENGINE_test_categories_external.
- The JSON importer and recipe VM comments still said tile.buffer_load is
  aliased to a dtype-suffixed opcode. It has its own dtype-generic opcode,
  so the exact name resolves it. Comment-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…caps helpers (AICK-2762)

First layer of the C++ port of the attention backward builders. Adds
C++ twins of attention_bwd_aux (the prep, convert, reduce, page zero and
page convert kernels) and the helpers they use: _attention_bwd_addr,
_attention_bwd_worklist and the caps header. addr and worklist are
partial here; later layers fill them in.

Each twin has a parity pair under library/tests/parity: a Python emit
script and a C driver build the same configs, and the .ll must match
byte for byte. The Python modules' docstrings now point at their twins.

attention_fwd_ext's LOG2E is defined locally in attention_bwd_aux.cpp:
that forward helper has no C twin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the shared platform band helper (rocke_band_*,
helper_rocke.helpers.attention_band.h) and the backward band twin
(_attention_bwd_band), with a parity pair.

All backward twins use one rocke_attn_bwd_ / ROCKE_ATTN_BWD_ prefix,
and the addr twin takes I32_FAR from the band helper, as Python does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…er (AICK-2762)

Ports the main backward kernel spec (instance_attention_bwd.h,
attention_bwd.cpp), the caps checks, the tile geometry and the LDS
planner. Parity pairs: layout, lds, spec and spec_extra.

The spec pair has a config for every LAYOUT_SPILLING_COMBOS row
(configs 255-264 cover the 10 rows the others miss), so the parity
checks prove C++ refuses each row. ROCKE_ATTN_BWD_NONE (INT32_MIN), the
None of an Optional field, is documented as reserved: it always reads
as None, never as a value.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s (AICK-2762)

Ports _attention_bwd_frag and _attention_bwd_gemm, with a parity pair.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… helpers (AICK-2762)

Ports four small backward helpers, each with a parity pair, and the
addr pieces they need. The Python docstrings point at the twins.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…CK-2762)

The parallel ports left private copies of shared helpers. frag now
calls valu's cast_c_frag and lds's DmaSlabLayout, and attention_bwd.cpp
builds COUNTED_ISSUES, COUNTED_VMCNT_MAX, BF16_CASTS, BWD_EXP2S and
STATS_VECS from the counted and valu exports.

The shared DmaSlabLayout now computes in int64_t, as the removed private
copy did, so a large slab offset no longer wraps; its row_bytes, size
and offset C entry points return int64_t. Every narrowing to int (the
lane-map coordinates, the xlane index, align_up, the LDS plan's nbytes
and access offsets) is range-checked and raises ROCKE_ERR_VALUE instead
of wrapping.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ports _attention_bwd_sched (the stage-scheduler tables) and the first
part of _attention_bwd_ring. Parity pairs: sched, sched_ir and
sched_params.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…g (AICK-2762)

Ports _attention_bwd_body and the rest of _attention_bwd_ring, plus the
addr, counted and gemm pieces they need. Unknown arch raises, as in
Python. Parity pair: body.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ports the general attention backward main kernel builder
(attention_bwd_main.cpp) and the rest of the addr and worklist
helpers. The main parity pair builds every phase-1 general case.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ports the split dQ kernel (attention_bwd_dq.cpp) and its body
(_attention_bwd_dqbody). Parity pair: dq.

Refusals: validate refuses the plain-tuple tables DQ_SPILLING_COMBOS and
DQ_SCHED_SPILLING as Python does, with Python's messages and in Python's
order. DQ_RESOURCE_REFUSED is keyed by spec_digest and stays Python, so
a caller that launches the kernel must consult Python's
validate_attn_bwd_dq_spec first. The dq pair has one refused config per
table row, and test_attention_bwd_dq_refusal_rows.py (in
rocke_library_attention_cpu_pytest) fails if Python adds a row without
one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ports the dense backward spec, main and dQ kernels and the dense shell
(_attention_dense_bwd_shell). The main and dQ builders gain _kv_tile
hooks so the dense shell can reuse them. Dense body_knobs names are
refused as Python does. Parity pairs: dense_main and dense_dq.

A C spec is a plain struct, so a caller can edit it after init or skip
init. The dense and dense-dQ validators therefore re-run init before
anything else, so such a spec is refused with init's own message.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…-2762)

Ports the unified backward main kernel (attention_bwd_unified.cpp). It
shares the _kv_tile hooks with the dense kernels through
instance_attention_bwd_main_internal.h. Parity pair: unified_main.

Its validate() re-runs post_init on a copy first, as main and dQ do
through resolved(). New ctest rocke_attention_bwd_spec_reinit checks
that the unified, dense and dense-dQ validators refuse a field edited
after init, and a spec never initialised, with init's message.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
afagaj and others added 2 commits November 4, 2026 09:01
… golden (AICK-2762)

CI never compared the backward C++ twins with Python. check_byte_identity.py
builds the engine from source, so it cannot run in the installed test tree;
test_attention_bwd_ir_cpp_parity.py is in no CTest list; and the
differential golden had no entries for these families.

- Bless the 20 attention_bwd / attention_dense_bwd parity families into
  golden/llvm_gfx_all.json: ll at llvm20, llvm22, llvm23 and llvm24
  (llvm24 is its own section, with the same hashes as llvm23), 2,646
  configs each, and canonical IR (15 families, 1,846 configs; the other 5
  emit no IR). rocke_differential_golden_ll now fails when a backward
  Python builder changes.
- tests/cmake/NativeGoldenTests.cmake builds those 20 C++ emitters with the
  engine and installs them as test binaries. A listed family with no
  _emit.c is a FATAL_ERROR; a platform-only copy (no library tree) skips
  with a STATUS message. Warnings do not fail these test drivers, as in
  run_diff.py, which builds them with no warning flags.
- New check_native_golden.py runs each emitter over its configs and
  compares the .ll with the same golden hashes. It enforces the emitters'
  exit codes: 0 must be a blessed config with the blessed .ll; 1 an
  unblessed one with no output; 2 + "unknown config" the end of the
  family; anything else fails. When every index up to the family's cap
  answers, it asks for index cap and fails if that config exists. A missing emitter, a family with
  no golden or a crash fails, never skips. test_check_native_golden.py
  covers each case with a scripted emitter.
- run_diff.py: attention_bwd_body, attention_bwd_spec and
  attention_bwd_unified_main get a 512-config cap; they have 256 or more
  configs, and the default cap is 256.
- New ctest rocke_native_golden_ll (build tree and installed tree, quick
  and ffm-quick tiers) runs it at llvm20, llvm22, llvm23 and llvm24.
- New test_attention_bwd_layout_spill_rows.py (in
  rocke_library_attention_cpu_pytest): every LAYOUT_SPILLING_COMBOS row
  has a spec parity config refused at that row, present in the C table.
  attention_bwd_spec_emit.c is installed with the library tests (the
  install skips *.c otherwise) so this check also runs in CI.
- New ctest rocke_attention_bwd_dma_slab_int64: the DMA slab layout keeps
  64-bit offsets (expected values from Python).
- golden/README.md: for these families rocke_differential_golden_ll and
  rocke_native_golden_ll together are the parity verdict in CI.

Python == golden and C++ == golden, so C++ == Python in CI. A tampered
hash, a removed family, an extra config and a missing binary each fail
the new ctest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The design doc, the dispatcher notes and the barrier-fence parity files
still said the C++ engine has no backward builder (the "C++-engine
waiver"). The twins now exist and CI checks them, so:

- attention_bwd.md: 0.3 replaces the waiver with what is ported (every
  kernel builder and its helpers), the three checks that hold the
  dual-engine rule (check_byte_identity.py, the ctest
  rocke_native_golden_ll, test_attention_bwd_ir_cpp_parity.py) and what
  is not ported (routing, plans, tuning, and the RESOURCE_REFUSED /
  DQ_RESOURCE_REFUSED tables, which a future C++ caller must consult in
  Python first). The file map lists the twins and their checks; 0.4,
  8.4 and the section 11 gate table follow.
- library/dispatch/AGENTS.md: the dispatcher is Python-only; the C++
  engine has the kernel builders, not this routing.
- barrier_fence_emit.py, test_barrier_fence_parity_emit.py: there is
  still no barrier_fence_emit.c, but because the twins' own parity
  pairs compare every fenced barrier, not because no twin exists.

Docs and comments only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@afagaj
afagaj force-pushed the users/jafagani/rocke-bwd-cpp-stack branch from 3c32658 to d83c2d6 Compare November 4, 2026 15:01
The C++ main and dQ builders built the specs that Python's
RESOURCE_REFUSED and DQ_RESOURCE_REFUSED refuse, and only comments
said so. Port spec_digest (SHA-256 over the non-default fields) and
both tables, generated from the Python dicts into
_attention_bwd_refusals_table.inc.

The C++ main validator and builder now take main_kernel_refusals with
no default; the public C entry points refuse on true with Python's
message, and only the dense and unified shells and the IR parity
emitters pass false, as Python does. The dQ validator always applies
DQ_RESOURCE_REFUSED, as Python does.

Checks: test_attention_bwd_refusals_cpp_table.py (the committed table
equals the dicts) and the parity pair attention_bwd_refusals_emit
(digests and verdicts of 68 main and dQ specs, 57 of them recorded
refusals), enrolled in rocke_native_golden_ll with goldens at
llvm20/22/23/24.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@afagaj
afagaj force-pushed the users/jafagani/rocke-bwd-cpp-refusal-tables branch from 4a0f856 to 2b481bf Compare November 4, 2026 15:28
@afagaj
afagaj force-pushed the users/jafagani/rocke-bwd-cpp-stack branch 2 times, most recently from bd91264 to 94ea1c1 Compare November 7, 2026 03:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant