Skip to content

[rocjitsu] Translate gfx1250 lane-banked and kernarg-supplied indirect calls - #10444

Merged
atgutier merged 27 commits into
release/bkc/therock-10.1-20260811from
users/agutierr/release-bkc-10.1-hotswap-update-20260819
Aug 20, 2026
Merged

atgutier merged 27 commits into
release/bkc/therock-10.1-20260811from
users/agutierr/release-bkc-10.1-hotswap-update-20260819

Conversation

@atgutier

@atgutier atgutier commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

hipTensor and rocThrust gfx1250 objects are refused at code-object load time with "indirect branch or call target recovery is not implemented for relocated kernel text" on s_swap_pc_i64. A refused object leaves HIP holding an invalid kernel
handle, so the failure surfaces as a segfault rather than a diagnostic. The fix is upstream as #10376; this backports it to release/bkc/therock-10.1-20260811.

Commits Cherry-Picked

# Upstream PR Backport Subject
1 b6d838028e #10197 846fb16d12 [rocjitsu][docs] Refresh architecture support documentation
2 5c8e33e391 #10200 db5c58b1be [rocjitsu] Define standalone decoder ownership
3 be1f91998f #10199 ed102baf1c [rocjitsu] Apply async IOFFSET to LDS addresses
4 fda635e5b4 — ad32ff7c18 [rocjitsu] Extract per-scope rollback and object-level checks from translate()
5 8f75eb781e — 4c0cb05207 [rocjitsu] Emit leading V_NOPs ahead of a gfx1250 MODE write
6 00aa1e260f #9941 ded43b1787 fix(rocjitsu): disable zero-count TDM descriptors
7 c810647609 #10192 50ca7f1732 [rocjitsu] Apply MUBUF LDS-load instruction offset to the LDS address
8 1d7866269d #9560 2e6bf26943 [rocjitsu] code rewrite condition validator
9 95b6c410bb #10175 8418901076 [rocjitsu] Fix ds_read_b64_tr_b16 lane layout in gfx950 simulator
10 bfaebcfb5c #10240 c4eac1e1f7 [rocjitsu][util] Add result and diagnostic utilities
11 dbb7221fa7 #9775 cdd0993a5f fix(rocjitsu): route tensor DMA through wave VMID
12 2779247222 #9779 7bfeba6f5d feat(rocjitsu): Lazy vgpr allocation system
13 a659172a31 #9936 180cf00325 fix(rocjitsu): honor TDM tensor bounds across iterations
14 60c29efb39 #10204 370bfb9796 [rocjitsu] Fix gfx950 decoder extension handling
15 c1dba564e7 #10345 536110a083 [rocjitsu] Model gfx1250 wave ABI state
16 7f71be9f32 #10346 3f922ad5f2 [rocjitsu] Model gfx1250 barrier synchronization
17 bea7ac8937 #10368 7b5ecf6c7b [rocjitsu] Replace decoder exceptions with results
18 f2e878694e #10371 8cf750d23f [rocjitsu] Dispatch operand validation by type
19 dd619bed77 #9735 1a612c47c0 fix(rocjitsu): respect same-wave LDS ordering
20 c5fae8d2b2 #10137 4360c4b978 [rocjitsu] Rename gfx1250 to cdna5 for enums, configs, amdisa codegen helpers and generated symbols. (NFC)
21 b05a587f89 #10274 bfb5f9fab0 [rocjitsu] Align CDNA4 MI350X configs with captured topology
22 f81f9d40db #10362 8177b5725c feat(rocjitsu): cdna5 OOB behavior implementation
23 53a939bd00 #10283 141b109432 [rocjitsu] add per-XCD workgroup dispatch observability
24 a0fb11bc17 #10414 6bb4c5a959 [rocjitsu] Refactor test encoding handling in execution tests (NFC)
25 e5109e315e #9844 322ac26b44 feat(emulation): debug emulated GPU kernels with real ROCgdb
26 efa24f23c9 — c3c9641073 rocjitsu: Point the XCD distribution test at the config that exists
27 8b7763ef5e #10376 c3f419555a [rocjitsu] Translate gfx1250 lane-banked and kernarg-supplied indirect calls

Issue Tracking

ROCM-29436

Submission Checklist

@atgutier
atgutier requested review from a team August 19, 2026 23:07
@atgutier
atgutier requested a review from a team as a code owner August 19, 2026 23:07

@ScottTodd ScottTodd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you mark this as a cherry-pick of #10376 / 8b7763e in the PR description?

kuhar and others added 24 commits August 19, 2026 16:32
Align the public documentation with the simulator and translation
coverage currently represented in the tree.

Mark CDNA3, CDNA4, CDNA5, RDNA3, and RDNA4 simulation support as Beta;
keep CDNA5 scoped to gfx1250; use family target names where appropriate;
remove the DBT and DBI table columns; and document the experimental
CDNA4-to-CDNA3 translation path.

Refresh the emulation overview without a brittle ISA count, and expand
the configuration guide to list every checked-in simulator and DBT guest
configuration.

(cherry picked from commit b6d8380)
Contain decoder failures inside the C API and clear writable outputs on
every failure path.

Document successful standalone decode results as caller-owned, provide a
matching destroy API, and cover invalid encodings plus leak-free
lifetime behavior.

Fixes #8810

Fixes #8814

(cherry picked from commit 5c8e33e)
Apply the signed 24-bit IOFFSET to both the global and LDS addresses for
ordinary and cluster async loads and async stores, matching CDNA5
section 10.8.1 and expressions 95-102 and 106-109.

Reject LDS destinations outside the issuing workgroup allocation, keep
cluster remapping widened through validation, and filter invalid lanes
from writeback and race detection.

Add generator, execution, multicast, and plugin coverage for signed
offsets, allocation bounds, and wraparound addresses.

Fixes #8829

(cherry picked from commit be1f919)
…anslate()

BinaryTranslator::translate() had grown to roughly 2280 lines because the class holds almost no
state: the arch pair, the options, three engine handles. Everything the translation actually works
on lived in function locals, which is why fifteen lambdas had accumulated -- each existed to reach
a local a free function could not. Nothing could be extracted while the state stayed anonymous.

Name the state first. The five per-scope rollback locals become one ScopeTransaction, so the
twenty-six failure sites drop from seven arguments to three; output_begin turned out to be consumed
entirely by those calls, which is the evidence the bundling is complete rather than partial. This
is deliberately not an RAII guard: rollback is conditional on skip_failed_kernels and the failing
scope still contributes a stub body, so unwinding has to be an explicit call. The five local types
move to file scope, which is what makes any further promotion possible, and three lambdas that
captured nothing follow them. The object-level refusals become translation_refusal(), returning a
diagnostic rather than a bool because one of them -- an object with no executable text whose target
already matches -- is a successful no-op whose warning severity must survive; the image-size and
decoder checks stay behind because they gate constructing the patcher and decoder the rest of
translation needs, which leaves that function reading an ELF header the caller has already sized,
so the precondition is asserted rather than left to the reader. resolve_pending_code_relocations()
comes out whole, being the one tail block that sits outside the scope loop and so needs no
skip-cascade threading.

Comments travel with the code they describe: both hoisted checkpoint structs keep the text that
documented them in situ, and the remaining local types gain a brief, which matters more now that
they sit hundreds of lines from their only use. Nothing in the rollback body aliases the
transaction back into the five names it replaced, so a sixth participant can only be added in one
place.

No behavior changes. Each step was verified against a binary built before any of them by
translating all 905 gfx1250 objects in the corpus with idempotence checking: byte-identical output
every time, cumulatively against the original rather than against the previous step. The remaining
extractions all sit inside the scope loop, where four separate skip_scope break cascades mean a
naive bool return would silently change which of translated_text, text_relocations, and
canonical_placement roll back -- a wrong rollback still produces self-consistent bytes, so the
corpus would not catch it. Those need a tri-state status and are left for a follow-on.

Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
(cherry picked from commit fda635e)
On this target an s_setreg* naming the MODE register requires two V_NOPs
immediately before it to be ordered against the instructions that precede it,
and compiler output does not supply them. Register a semantic expansion that
emits the missing V_NOPs followed by the original instruction. What identifies
such a write is the hardware-register selector in SIMM16 rather than the
mnemonic, so the rule discriminates on that and declines every other register,
which leaves it on the verbatim copy path. The scope is the setreg instruction
rather than MODE as a location, so the other writers of MODE state --
s_set_vgpr_msb and the SOPP round- and denorm-mode forms -- are excluded. V_NOPs
already present immediately before the write are counted and only the missing
slots emitted, so a repeated translation is a fixed point.

The integer WMMA spacing rule needed the same block-local scan, so both now
share one walk, and both record that V_NOP is the best available filler rather
than a guarantee: ordinary VALU is skipped when EXEC==0, while V_NOP issues but
occupies no slot there. Rule tables are binary-searched, so each now
static_asserts that it is strictly ordered by encoding id and opcode; a
misplaced or duplicated entry otherwise misses its own rule with no diagnostic.
Rejecting a duplicate holds only while the rule comparison orders by key alone,
so a same-key pair differing in everything else is asserted to be rejected too.
The diff report gains a label for a semantic lowering that carries an
instruction with no legalization entry, which is how this rule is registered.

Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
(cherry picked from commit 8f75eb7)
## Summary

This small change makes tensor-DMA load and store operations honor a
zero
descriptor count as a disabled transfer.

The gap was found while reducing the stock gfx1250 TensileLite TDM
workload
tracked by #9935. Generated TensileLite code explicitly writes zero to
the
descriptor Group 0 count field when disabling TDM loads in the final
prefetch
iterations, but rocJITsu previously ignored that field and executed the
transfer.

## Related contract evidence

- ROCm/rocm-libraries#7453 introduced the HalfPLR TDM disable convention
by
conditionally clearing descriptor Group 0 and restoring it before the
tail
  loop.
- #7361 established the shared TDM wait-counter and
atomic
  barrier sequencing that this PR preserves inside the common load/store
  executor.

## Failure before this fix

A descriptor that generated code had disabled could still copy global
data
into LDS or store LDS data to global memory. If the descriptor requested
an
atomic-barrier arrival, that side effect was also performed.

## Fix

- Parse the two-bit count field from descriptor Group 0.
- Represent descriptor activity as part of the parsed descriptor
contract.
- Route load and store through one common tensor-DMA executor, which
performs
  no copy or barrier side effect for an inactive descriptor.

## Test coverage

The focused regression initializes global memory, LDS, and an atomic
barrier
with distinct values, executes decoded load and store instructions, and
verifies that both operations leave all three unchanged. Existing
tensor-DMA
execution tests continue to pass.

## Workload status

This PR fixes one independent descriptor contract. It does not claim to
resolve
the complete TensileLite reproducer or the remaining tail-loop semantic
question.

## Issue tracking

Progress toward #9935.

(cherry picked from commit 00aa1e2)
…#10192)

## Motivation

`buffer_load_* ... lds` ignores the instruction's immediate `offset`
when computing the LDS destination address in the simulator. Per the
CDNA3/CDNA4 ISA reference guides, §9.1.9 "Memory Buffer Load to LDS":

```
LDS_ADDR = LDSbase + LDS_offset (M0) + inst_offset + (TIDinWave * 4)
MEM_ADDR = Base + mem_offset (soffset) + inst_offset + off_vgpr + stride * (idx_vgpr + TIDinWave)
```

`inst_offset` feeds both addresses (LLVM `IntrinsicsAMDGPU.td`,
`int_amdgcn_load_to_lds`, which also covers buffer fat pointers: "imm
offset (applied to both input and LDS address)"). Kernels that set M0
once and issue several `buffer_load_dword ... offen offset:0/256/512
lds` to fill consecutive LDS tiles therefore kept overwriting the first
tile under the simulator.

## Technical Details

- `lib/python/amdisa/codegen/_generator.py`: the MUBUF LDS-load template
now sets `d->lds_base = wf.m0() + inst_.offset + wf.lds_base()`.
`soffset` and the VGPR offset stay memory-side only, matching the
formula above.
- Regenerated
`generated/{cdna1,cdna2,cdna3,cdna4,rdna1,rdna2}/mubuf_exec.cpp`
(documented `python -m amdisa --multi ... --gen-isas` flow +
clang-format). With the in-tree ISA XMLs the unmodified generator
reproduces every checked-in `mubuf*` file byte-for-byte; the only
resulting change is that one line in the 18 `buffer_load_*` LDS branches
per file. gfx1250's `*_load_async_to_lds_*` (per-lane VGPR LDS address,
separate template) and rdna3_5's `buffer_load_lds_*` stubs
(`UnimplementedInst`) are untouched; rdna3/rdna4 generate no MUBUF
LDS-load path.
- `tests/amdgpu_vm_test.cpp`:
`MubufLdsTest.LoadDwordLdsAppliesInstOffset` (CPU-only, cdna1-4): M0 =
0x200, two encoded `buffer_load_dword v1, s[4:7], 0 offen [offset:256]
lds` from two distinct source rows, all 64 lanes of both LDS rows
checked.

## Issue Tracking

No existing issue; related: #5747 (workgroup `lds_base` term on the same
path).

## Test Plan

- `rocjitsu_tests --gtest_filter='MubufLdsTest.*'` before and after the
generator change.
- `python -m pytest lib/python/amdisa/tests` and full `ctest` for
regressions.

## Test Result

- `MubufLdsTest.LoadDwordLdsAppliesInstOffset`: on unmodified `develop`
it fails for all four CDNA targets (lane 0: LDS+0x200 reads 0xB0000000
instead of 0xA0000000 because the `offset:256` load overwrote row 0;
LDS+0x300 still holds the 0xDEADBEEF fill); passes with this change.
- Cross-checked on gfx950 hardware (ROCm 7.2.0) with an equivalent
inline-asm kernel: the two rows land at LDS+0 and LDS+256 on hardware;
the simulator matches only with this change.
- Full `ctest`: no status changes versus `develop` other than the added
`MubufLdsTest.LoadDwordLdsAppliesInstOffset`.
- `python -m pytest lib/python/amdisa/tests`: 1319 passed, 46 skipped.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit c810647)
## Motivation

Add another rewrite validator similar to the idempotence check. This
confirms that no registered rewrite condition remains after rewrites are
applied. (Idempotence would not, for example, detect a rewrite that
happened to be the identity rewrite).

## Technical Details

- refactor rewrites into central registry shared by both the existing
rewrite system and the new validator.
-Include operand-driven rewrites in the refactored registry, since they
were not representable by the existing opcode-keyed table.
- Implement read only pass over translated binary to check for remaining
rewrite sites.

## Issue Tracking

N/A

## Test Plan

Enable this checking pass over the existing rewrite tests + add tests
for remaining added interface.

## Test Result

Tests pass

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit 1d78662)
…0175)

## Motivation

`ds_read_b64_tr_b16` returns incorrect results under the gfx950 (CDNA4)
simulator. The instruction performs a 4x16-lane halfword transpose:
within each 16-lane group, destination lane `l` halfword `n` comes from
source lane `((l & ~0xf) | ((l >> 2) & 3)) + 4 * n`, halfword `l & 3` —
the layout already encoded by the DBT reference in
`tests/dbt/cdna4_to_cdna3_lds_hip_test.cpp`
(`LargeStaticLdsReadB64TrB16`). The pure-simulation path's
`transpose_b16()` instead performed a 4x4 transpose within groups of 4
consecutive lanes (lane 0 received source lanes 0,1,2,3 halfword 0
instead of lanes 0,4,8,12), so every gfx950 kernel that loads bf16/f16
MFMA operands with `ds_read_b64_tr_b16` computed on wrong data under the
simulator.

## Technical Details

- `lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/shared/ds_transpose.h`:
`transpose_b16()` now handles the wave64 B64 case (4 halfwords per lane,
i.e. `ds_read_b64_tr_b16`) with the 4x16-lane stride-4 layout above.
This changes only that path; the B128 path used by gfx1250
`ds_load_tr16_b128` / `global_load_tr16_b128` and RDNA4
`global_load_tr_b128` is left byte-for-byte unchanged. `tr_b4` / `tr_b6`
/ `tr_b8` are untouched and were not hardware-verified here.
- `tests/amdgpu_vm_test.cpp`: new
`DsTransposeTest.ReadB64TrB16_LaneLayout` (CPU-only). Fills LDS with a
per-lane halfword pattern, dispatches an encoded `ds_read_b64_tr_b16` on
the `cdna4` VM fixture, and checks all 64 lanes x 4 halfwords against
the same reference formula the DBT test uses.
- `TransposeKind` is split into `TR16_B128` (gfx1250/RDNA4 B128 loads,
unchanged) and `B64_TR_B16` (CDNA4 `ds_read_b64_tr_b16`), emitted from
`InstructionSemantics`; `transpose_response()` dispatches on the kind.

## Issue Tracking

No existing GitHub issue. Related: #6829 (AccVGPR `acc` bit handling for
the same DS transpose reads).

## Test Plan

- `ctest` / `rocjitsu_tests --gtest_filter='DsTransposeTest.*'` for the
new CPU-side layout test, before and after the `transpose_b16()` change.
- Existing test suite for regressions.

## Test Result

- `DsTransposeTest.ReadB64TrB16_LaneLayout`: built and run locally (CPU
only, GCC 13, Release): fails on the pre-change `transpose_b16()` with
128 mismatches (64 lanes x 2 dwords; e.g. lane 0 low dword 0x12111200 vs
expected 0x12441200) and passes after.
- Cross-checked on gfx950 hardware (ROCm 7.2.0) with an equivalent
inline-asm `ds_write_b64`/`ds_read_b64_tr_b16` kernel: hardware matches
the expected layout; the simulator matched only after this change (all
64 lanes differed before).
- Full `ctest` shows no status changes versus `develop` other than the
added `DsTransposeTest.ReadB64TrB16_LaneLayout`.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit 95b6c41)
Add compact Result and FailureOr types for expected failures that should
not propagate exceptions. Their operations establish an explicit
termination boundary if a value operation throws.

Factor FailureOr storage behind a type traits-style customization point.
We provide one for pointers -- `nullptr` is considered the failure
state, which makes the representation more space efficient and avoids
the usual confusion about double-nullable types.

Add non-owning diagnostic sinks and stream-style in-flight diagnostics
-- this decouples diagnostics from error handling, as an alternative to
strongly typed error code enums or custom error types.

Issue: #10239
(cherry picked from commit bfaebcf)
## Summary

This change fixes the first rocJITsu tensor-DMA memory-translation
failure reached by a gfx1250 TensileLite BF16 TDM GEMM.

The workload comes from:

```text
Tensile/Tests/common/gemm/gfx12/tdm_gfx1250.yaml
problem identifier: Contraction_l_Alik_Bljk_Cijk_Dijk
generated group: Cijk_Alik_Bljk_BBS_BH_UserArgs_00
representative problem: (64, 64, 1, 512)
```

## Failure before this fix

`tensilelite-client` segfaulted while executing a gfx1250 tensor
load-to-LDS operation:

```text
tensor_dma_detail::copy_bytes
  -> tensor_dma_detail::copy_dense_tensor
  -> execute_tensor_load_to_lds<TensorLoadToLdsVimage>
```

The fault was a direct host dereference of a GPU virtual address such as
`0x7fff2cc00000`.

## Root cause

Tensor DMA accessed `GpuMemory` without binding the issuing wavefront's
process ID. The access therefore used VMID zero. In local functional
mode, VMID-zero passthrough treated the user-space-looking GPU VA as a
host pointer instead of resolving it through the dispatched process page
table.

## Fix

- Add wave-scoped block read and write methods that always bind
`Wavefront::process_id()`.
- Route tensor-DMA global-memory accesses through those methods.
- Remove the raw `InstructionComputeUnitView::memory()` accessor so
instruction code cannot accidentally omit the wave VMID.
- Use one `GpuMemory::read_block` or `write_block` operation per 1-, 2-,
4-, or 8-byte tensor element instead of acquiring translation locks once
per byte.
- Keep VMID zero as the intentional default for host, driver, and test
callers.

An equivalent VMID propagation exists in the broader unmerged
[`users/kuhar/dbt-sim`](https://github.com/ROCm/rocm-systems/tree/users/kuhar/dbt-sim)
work at commit
[`08a5c461a5`](08a5c46);
this PR isolates the tensor-DMA correction, strengthens the
instruction-facing memory contract, and adds direct regression coverage.

## Test coverage

The new unit test:

- registers a nonzero process page table;
- maps load and store GPU VAs to separate host arrays;
- proves VMID zero resolves to different storage;
- verifies tensor load-to-LDS uses the wave process page table; and
- verifies tensor store-from-LDS uses the same page table.

Local validation includes the complete `Gfx1250ExecutionTest.TensorDma*`
test set and changed-file pre-commit checks.

## Workload status

With this fix, the generated TensileLite client advances past the
original VMID-zero direct-VA fault. The complete stock workload still
has separate tensor-DMA tail and descriptor-contract questions tracked
by #9935. This PR removes the narrow translation failure and does not
claim to resolve the full workload.

## Issue Tracking

Closes #9774

This is the first independently reviewable rocJITsu correction tracked
by #9935.

(cherry picked from commit dbb7221)
## Motivation

Rocjitsu currently eagerly allocates the entire VGPR file for every
simulated CU. This completely dominates RSS for most kernels: the gfx942
topology alone commits approximately 1,280 MiB (1.25 GiB) of fixed VGPR
storage. This PR replaces eager allocation with demand-paged backing for
VGPRs. RSS is significantly reduced on current benchmarks and workloads,
while performance is also maintained or improved.

## Technical Details
- Implement lazy VGPR allocation using a portable software page table
with 4KiB pages, allocated and zero-initialized on first non-const
access.
- Contiguous VGPR range guarantee is lost: This was only ever used for
checkpointing save and restore: A replacement mechanism to copy pages
into the checkpoint buffer is implemented.
- Add a saturation benchmark with full-output checksum validation and
compiled resource-profile verification.

## Issue Tracking

N/A

## Test Plan

Perform benchmarking on workloads available in
rocjitsu-test-coprpus/IREE.

## Test Result
### IREE e2e Softmax Test
The `softmax_static_10x256x256xf32` IREE corpus workload processes
655,360 FP32 elements and writes a 2.5 MiB output. It was compiled
independently for gfx942 and gfx950, executed through IREE's HIP runtime
under the corresponding rocjitsu simulator, and compared bit-for-bit
with the checked-in NumPy golden output.

Each exact-output total combines five baseline and five lazy-storage
executions. The process measurements retained for the RSS and runtime
summary were:

| Target | Build          | Mean peak RSS (KiB) | Mean runtime |
|--------|----------------|--------------------:|-------------:|
| gfx942 | Baseline       |         1,627,477 |       2.09 s |
| gfx942 | Lazy registers |           316,328 |       0.70 s |
| gfx950 | Baseline       |         1,387,861 |       1.84 s |
| gfx950 | Lazy registers |           338,881 |       0.72 s |

### Corpus Kernel Results

| Workload | Register pressure | Baseline median | Lazy median |
Baseline peak RSS | Lazy peak RSS | Result |

|---------------------------|--------------------------|-----------:|------------:|------------------:|--------------:|-------------------------------|
| Startup, gfx942 | Fixed topology only | 0.86 s | 0.13 s | 1,511,424
KiB | 198,656 KiB | 84.9% faster, 86.9% less RSS |
| Corpus matvec set | 6–88 VGPR | 3.26 s | 1.86 s | 1,616,700 KiB |
305,864 KiB | 42.9% faster, 81.1% less RSS |
| Corpus HipKittens 256^3 | 248 VGPR | 2.03 s | 0.60 s | 1,626,112 KiB |
314,368 KiB | 70.4% faster, 80.7% less RSS |
| Corpus hip-matmul 128^3 | Up to 133 VGPR plus AGPR | 6.34 s | 4.91 s |
1,615,000 KiB | 302,940 KiB | 22.6% faster, 81% less RSS |

All three completed corpus workloads produced their expected correctness
results. The fixed-RSS reduction is 1,312,768 KiB. That closely matches
the original estimate of approximately 1,281 MiB to eagerly committed
VGPR files.

### Adversarial Kernel Analysis
The storage cost has three distinct sections:
1. Untouched VGPR pages consume no RSS.
2. The first access during a wave dispatch commits pages, with cost (RSS
and time) growing approximately linearly with the number of touched
pages.
3. Subsequent operations use the same contiguous pointer path as the
preexisting eager implementation and show no pressure-dependent penalty.
Physical pages are returned when the wave retires.

The worst case is therefore an extremely short kernel that declares and
touches nearly every VGPR across many CUs but performs very little work
with them. This is not a common real-world workload pattern. The
observed runtime penalty is about 8–9% for that deliberately adversarial
initialization-only case. Amortization from eight rounds of computation
(applying one XOR-multiply-add sequence) over every live VGPR reduces
the penalty below 0.7%, while retaining an 81.2% peak-RSS reduction.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit 2779247)
## Summary

This change fixes tensor-DMA bounds contracts exposed while reducing the
stock gfx1250 TensileLite TDM tail failure tracked by #9935.

It is independent of the VMID correction in #9775 at the source level.
Both changes are needed to advance the original workload through the
currently known rocJITsu model gaps.

## Related producer-side evidence

- ROCm/rocm-libraries#7976 introduced TDM iterate mode and the
`tdm_gfx1250.yaml` configuration lineage that exposed this gap.
- ROCm/rocm-libraries#8406 contains the same affected configuration
after iterate mode was enabled and reports passing FFM and B0
validation. This is file-level evidence rather than a published
per-solution trace.
- ROCm/rocm-libraries#9410 added iterate-mode edge cases with waves that
have zero valid rows specifically to exercise descriptor bounds
clamping, and reports passing FFM validation.

## Failure before this fix

Iterate mode advanced the starting global address for each repeated
tile, but rocJITsu continued to apply bounds using tile-local
coordinates. A final row outside the tensor was therefore copied into
LDS instead of being zero-filled.

An active tensor extent of zero was treated as unbounded. The complete
dense or gather tile could therefore be copied instead of being masked.

## Fixes

### Include iterate-mode address advancement in tensor bounds

The model decomposes each advancing iteration's linear global-address
increment into an absolute tensor-coordinate origin, then applies one
bounds predicate to that origin plus the tile-local coordinate.

This handles sub-row advances, row advances, and rank-3 plane advances
through the same path. Overlapping-stride layouts are rejected only when
multiple iterations actually advance the global origin; repeated-origin
descriptors do not require inverse coordinate decomposition.

### Treat zero active tensor extents as empty

For an active dense or gather tile dimension, tensor extent zero masks
the complete dimension. Loads zero-fill the affected LDS elements and
stores suppress the corresponding global writes.

Optional null descriptor groups retain their architectural zero-filled
value. A rank-3 tile with null D2 therefore has a zero third tensor
extent: it masks the transfer without aborting execution, still
completes an enabled barrier, and releases TENSORCNT normally.

### Avoid unnecessary origin decomposition

Non-iterating descriptors use an all-zero origin directly. This avoids
inverse stride decomposition, including accepted rank-5 layouts whose
large default strides can wrap.

## Test coverage

Focused decoded-instruction regressions verify:

- partial final-row load zero-fill and corresponding store suppression;
- active zero-length dense and gather extents;
- null-D2 rank-3 load masking and store suppression;
- completion-barrier arrival and TENSORCNT release for null D2;
- sub-row, row, and rank-3 plane iteration origins;
- rejection of advancing overlapping-stride layouts;
- acceptance of repeated-origin overlapping-stride layouts; and
- safe non-iterating rank-5 execution.

The complete `Gfx1250ExecutionTest.TensorDma*` test set and changed-file
pre-commit checks pass locally.

## Workload status

This PR intentionally does not claim to fix the complete TensileLite
reproducer. It removes independently testable descriptor-model
discrepancies and gets the workload closer to the unresolved tail-loop
semantic question.

## Issue tracking

Progress toward #9935.

(cherry picked from commit a659172)
These were found by the fuzzer added in
#9912.

- validate CDNA4 VOP3PX2 suffix pairs before constructing combined
instructions
- size literal and compact modifier extensions from the selected opcode
and operand capabilities
- render SDWA source and destination modifiers from extension words and
reject invalid selector combinations

(cherry picked from commit 60c29ef)
Model the architectural wave state that gfx1250 compiler-generated
kernels and debugger paths consume instead of relying on legacy scalar
register conventions.

Treat scalar selectors 102 and 103 as ordinary SGPRs on GFX10+ targets,
preserve Wave32 VCC_HI scratch across vector predicate and carry writes,
and expose explicit masked VCC access for VCCZ. Keep the flat-scratch
encoding policy architecture-explicit so unknown future targets do not
inherit the legacy alias.

Populate the gfx1250 TTMP launch payload from the AQL dispatch: encode
the queue packet index, wave-in-workgroup position, and Y/Z validity in
TTMP8, while retaining the architected workgroup and cluster coordinates
in TTMP6, TTMP7, and TTMP9. Extend execution and dispatch tests across
every scalar resolver direction, both VCC execution paths, one- and
two-dimensional TTMP layouts, and nonzero queue slots.

(cherry picked from commit c1dba56)
Implement the CDNA5 split-barrier model used by compiler-generated
gfx1250 kernels. Derive the signal, wait, init, join, leave, and state
operations for the gfx1250 profile while retaining the RDNA4 behavior
inherited by other targets.

Track independent completion state for named, workgroup, trap, cluster,
and cluster-trap barriers. Carry workgroup counters in the compute unit
and coordinate per-workgroup cluster arrivals across compute units under
a dedicated execution-state lock. Shrink membership when waves or
clustered workgroups terminate, honor privilege and single-wave rules,
and notify execution plugins outside internal locks with the complete
synchronized domain.

Decode named-barrier allocation from the AQL kernel descriptor, use the
generated M0 operand selector, and align LDS tensor-barrier phase
expectations with the architectural layout. Add instruction and dispatch
coverage for named membership, negative barrier IDs, independent
domains, early termination, cross-CU cluster synchronization, packed
state, and barrier transitions.

(cherry picked from commit 7f71be9)
Instruction rejection is normal decoder control flow, so represent it
explicitly instead of throwing. Generated decoders now return
DecodeResult, diagnostics travel through the failure path, and decoder,
operand, translation, patching, fuzz, test, and tool callers handle the
result directly. The generator remains the source of truth and all
architecture outputs are regenerated.

Build on the prerequisite Result and diagnostic utilities. Keep
validation call sites uniform without emitting empty per-format methods:
IsaInstruction supplies a static constexpr success fallback, while
formats hide it only when they have real checks. Generated validation
temporaries spell out Result, with no virtual dispatch.

Add a valid-dominated large-object decode benchmark. Against the
exception-based baseline, the final niche-backed implementation is 4.1%
faster on successful decodes and 34.0x faster when rejecting invalid
encodings, based on medians from nine interleaved CPU-pinned runs with
identical checksums and no unexpected accepts or rejects.

(cherry picked from commit bea7ac8)
Generated operand constructors validated each encoding by walking every
operand-type predicate. Operand construction is the largest decoder
hot-path cost, and common operand types appeared late in that linear
chain.

Emit a switch over OperandType so construction evaluates only the
matching selector ranges while preserving deferred diagnostics. Delegate
the literal16 constructor to the primary constructor so each generated
translation unit contains one validation switch, regenerate all ten
AMDGPU models, and extend the generator profile gates for the shared
structure.

On the pinned large-object benchmark, median valid-decode throughput
improves 25.1%, cycles fall 19.7%, and instructions fall 35.6% across
nine alternating CPU-pinned pairs with identical corpus results and
checksums.

(cherry picked from commit f2e8786)
## Summary

This PR fixes a general misunderstanding I had about same-wave LDS
synchronization on our GPUs..

I previously assumed that sequences such as:

```asm
ds_write_b8 v2, v0 # write to LDS at address in v2
ds_read_u8  v14, v2
```

and the reverse:

```asm
ds_read_u8  v14, v2
ds_write_b8 v2, v0
```

were racy unless an `s_waitcnt lgkmcnt(0)` appeared between the two
instructions. That is true for consuming the result of an LDS read. For
example, this is racy:

```asm
ds_read_u8 v14, v2
v_and_b32_e32 v5, 1, v14
```

The read may not have returned its result to `v14` yet. This requires a
wait:

```asm
ds_read_u8 v14, v2
s_waitcnt lgkmcnt(0)
v_and_b32_e32 v5, 1, v14
```

However, ordinary DS operations issued by the same wave are ordered with
respect to each other. They may be in flight at the same time, but a
later DS instruction **does not overtake an earlier one** in the LDS
pipeline. Consequently:

- A same-wave DS write followed by a DS read observes the write.
- A same-wave DS read followed by a DS write observes the value from
before the write.
- The destination VGPR of an LDS read still requires the appropriate
wait before it can be consumed or overwritten.

An agent confirmed this experimentally on a gfx1151 GPU using both
same-lane and cross-lane byte accesses. Repeated no-wait write/read and
read/write sequences completed without a single value mismatch.

The issue originally surfaced as an LDS race report in hipBLASLt's HPL
initialization kernel. LLVM had promoted a private Boolean temporary to
per-workitem LDS storage and generated:

```asm
ds_write_b8 ...
ds_read_u8 ...
s_waitcnt lgkmcnt(0)
```

The plugin incorrectly reported the read as racing with the earlier
same-wave write.

## Changes

The race detector now:

- Treats ordinary same-wave DS write-to-read and read-to-write accesses
as ordered.
- Continues tracking an LDS read's destination VGPR until `lgkmcnt`
permits its use.
- Continues reporting overlapping cross-wave LDS accesses without the
required barrier.
- Continues requiring `vmcnt` before the owning wave consumes data
written through a direct-to-LDS VMEM operation.

The tests now include focused same-lane and cross-lane RAW/WAR unit
coverage, plus one gfx950 end-to-end ordering test.

## Testing

New ones added, incorrect ones fixed.

## Issue Tracking

Related: #9577
(cherry picked from commit dd619be)
… helpers and generated symbols. (NFC) (#10137)

## Motivation

Migrate architecture-level GFX1250 identifiers to the logical CDNA5
architecture identity. This separates the reusable architecture identity
from concrete GFX1250 hardware and target-specific concepts.

## Technical Details

* Replaces ROCJITSU_CODE_ARCH_GFX1250 with ROCJITSU_CODE_ARCH_CDNA5
across ISA dispatch, builders, code processing, configuration, VM,
analysis, DBT, hooks, tools, and tests.
* Makes cdna5 the canonical ISA provider and amdisa profile key.
* Renames the simulator configuration to gfx1250_cdna5.json and updates
its consumers.
* Renames the Python ISA profile and CDNA5-specific code-generation
helpers.
* Renames generated matrix/WMMA helper symbols through
cdna5_scaled_wmma_src1_size_bits.
* Regenerates affected amdisa output.
* Adapts tests split by the intervening CDNA5 simulator-test
reorganization.

This PR is limited to the logical CDNA5 architecture migration,
including enum usage, provider/profile/config names, amdisa
code-generation helpers, and generated matrix/WMMA symbols.

## Issue Tracking

N/A

## Test Plan

* Build the rocjitsu_tests target from build-nosan-debug.
* Run focused CTest coverage for:
  * OperandSelectorDecodeTest
  * BinaryTranslatorE2E
  * Gfx1250ExecutionTest
* Verify no tracked source references remain to the removed
ROCJITSU_CODE_ARCH_GFX1250 enum.

## Test Result

All good.

## Submission Checklist

- [X] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit c5fae8d)
Take the updated topology and device properties from the checked-in
MI350X KFD captures:

```
  projects/rocprofiler-sdk/source/lib/rocprofiler-sdk/tests/data/topology/nodes/5/properties:
    simd_count 1024
    array_count 32
    simd_arrays_per_engine 1
    cu_per_simd_array 9
    simd_per_cu 4
    device_id 30112
    num_sdma_engines 2
    num_sdma_xgmi_engines 14
    num_cp_queues 24
    max_engine_clk_fcompute 2200
    capability 2889327232
    capability2 1
    debug_prop 1494
```

```
  projects/rocprofiler-sdk/source/lib/rocprofiler-sdk/tests/data/topology/nodes/5/mem_banks/0/properties:
    size_in_bytes 309220868096
    width 8192
    mem_clk_max 1900
```

The topology values describe 288 physical CU slots but only 256 active
CUs:

```
  physical CUs = 32 arrays * 9 CUs/array = 288
  active CUs   = 1024 SIMDs / 4 SIMDs/CU = 256
```

With eight XCDs, the physical geometry becomes four shader engines and
36 CU slots per XCD:

```
  shader engines/XCD = (32 arrays / 1 array/engine) / 8 XCDs = 4
  physical CUs/XCD   = 4 engines * 1 array/engine * 9 CUs/array = 36
```

The config's local_mem_size field represents advertised VRAM capacity in
bytes, so set it from the memory-bank size_in_bytes value. rocminfo
displays the same capacity in KB:

```
  309220868096 bytes / 1024 = 301973504 KB
```

Adjust topology stress workloads to cover the 288 instantiated CU slots
and bound matrix workgroups beyond their 256 useful output tiles. Add
sysfs and rocminfo regressions to prevent the physical topology and
externally reported active-CU count from drifting apart.

rocJITsu does not currently model a harvested-CU mask, so all 288
physical slots remain schedulable even though rocminfo reports 256
active CUs.

## Issue Tracking

N/A

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

---------

Signed-off-by: hanhanW <hanhan0912@gmail.com>
(cherry picked from commit b05a587)
## Motivation

Implement OOB memory access behavior for gfx1250/cdna5. Lay groundwork
for other architectures.

## Technical Details

- Implemented missing gfx1250’s per-component VBUFFER OOB behavior.
- Model CDNA5 descriptor bounds, scaled stride, OOB_SELECT, and 45-bit
offset rules.
- Implement ordinary aligned, unswizzled TYPE=0 semantics: each dword is
checked independently, OOB load components return zero, OOB store
components are dropped, atomics remain all-or-nothing, and valid lanes
retain coalesced cache traffic. Swizzle, alignment modes/MEMVIOL, and
resource-TYPE validation remain follow-ups.

## Issue Tracking

N/A

## Test Plan

Added new tests. Ran tests.

## Test Result

Tests pass.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit f81f9d4)
A HW queue is bound to a single XCD by SoC::assign_queue_cp(), so every
workgroup of every dispatch on that queue is placed by that one XCD's
command processor on that one XCD's compute units. On the 8-XCD CDNA4
configs a single-queue application reaches 32 of the 256 CUs the
topology advertises, but nothing in the tree measured this.

Count workgroups placed per command processor and expose the per-XCD
histogram from the SoC, then pin current behavior in
xcd_distribution_test: one queue confines the whole grid to one XCD, and
only opening multiple queues reaches multiple XCDs. The follow-on
fan-out work flips the first expectation.

(cherry picked from commit 53a939b)
…10414)

## Motivation

Better code quality in tests.

## Technical Details

Share mnemonic prefix helpers and use typed templates for generated ISA
encoding tables, eliminating duplicate structs and unsafe casts.

## Issue Tracking

N/A

## Test Plan

Test on CI

## Test Result

Test pass

## Submission Checklist

- [X] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

(cherry picked from commit a0fb11b)
amd-arosa and others added 3 commits August 19, 2026 16:33
## Motivation

Debugging a GPU kernel has required a physical AMD GPU. rocjitsu already
emulates
the AMD KFD closely enough to run real ROCr/HIP workloads, but it had no
debug
surface: `AMDKFD_IOC_DBG_TRAP` was unimplemented, waves could not be
stopped, and
there was no CWSR state for a debugger to read or write.

This PR closes that gap. **Real ROCgdb and real `librocm-dbgapi`,
unmodified, now
attach to a workload running on the emulated GPU and debug its kernels
end to end**,
with no AMD hardware present: breakpoints, single-step, watchpoints,
multi-wave
`info threads`, source-level locals out of private/scratch, SIGILL and
SIGSEGV on GPU
faults, and clean detach. One command drives it:

```bash
mirage run --profile mi350x --gdb -- ./my_hip_app
```

The bar this is measured against is the official ROCgdb `gdb.rocm` test
suite, which
is written for hardware and knows nothing about the emulator. All 89 of
its files
pass.

## Technical Details

All KFD debug behaviour is mirrored from the real driver source
(`amd/amdkfd/{kfd_chardev.c,kfd_debug.c,kfd_topology.c}`,
amdgpu-6.16.13),
cross-checked against a physical MI300X and against the in-tree
`projects/rocdbgapi` client. Changes are confined to `emulation/`.

**KFD debug protocol**
(`kmd/linux/{simulated_kfd,remote_driver,rj_daemon,interposer}`)

- `DBG_TRAP` sub-operation dispatcher with the driver's validation
ladder, authorized
by a *real* ptrace relationship and marshalled over the daemon. The
debugger and the
inferior are separate processes, so in-process mode cannot host this and
daemon mode
  is mandatory.
- Debug sessions keyed by inferior pid, so ROCgdb can attach before the
inferior ever
  opens `/dev/kfd`; full attach/detach lifecycle, `GET_DEVICE_SNAPSHOT`,
  `GET_QUEUE_SNAPSHOT`, `SET_EXCEPTIONS_ENABLED` / `SET_FLAGS`, and
  `SET_WAVE_LAUNCH_OVERRIDE` validated against the gfx9.4 trap mask.
- Debuggee memory access authorized through the daemon (debugger-opened
proc memory
and pinned process descriptors travel with the enable request),
preserving remote
`DONTFORK` semantics and naming the synthetic KFD marker as `/dev/kfd`
in proc-map
  snapshots so fork and `gcore` behave as they do on hardware.
- Runtime enable/disable handshake reported only to an attached
debugger, `-ESRCH`
(not `-EPERM`) once the target's KFD state is gone, and suspend/resume
of
already-destroyed queues answered so rocdbgapi can retire them during
teardown.

**Engine wave-stop and CWSR**
(`vm/amdgpu/{compute_unit,wavefront,command_processor}`,
`kmd/linux/cwsr.{h,cpp}`)

- Waves stop on `s_trap` for the KFD debugger; stopped-wave state is
serialized into
  the real CWSR area layout and written back on resume.
- The runtime's own gfx9.4 trap handler is executed rather than shimmed.
A wave frozen
inside it needs the *interrupted* architectural state published, not the
handler's,
so EXEC and STATUS are shadowed in both directions: a debugger's edit
lands on the
state `s_rfe` restores instead of the register the handler is still
running under.
- Deterministic single-step via `TRAPSTS.trap_after_inst`; reserved
debugger memory for
displaced stepping; GPU address watchpoints; illegal instruction
reported as SIGILL;
memory violation reported as SIGSEGV; private/scratch reads so `print`
and
  `info args` resolve.
- Multi-wave: atomic queue resume, wave reclaim on queue destroy,
scheduler-locked
residency, 3D workgroup coordinates in CWSR records, and gfx950 LDS
preservation.
- Bounded functional-mode scheduling quantum while debugging so active
CUs make fair
  progress without starving peer waves.

**ISA and memory fixes surfaced by the suite** (`isa/arch/amdgpu/`,
`vm/amdgpu/`)

Running real debugger workloads exercised paths nothing else had: linked
helper calls
resolved against the `GETPC` base, ALU exception classification in
generated execute
paths, scalar memory access faults, signed scratch offsets, unaligned
swizzled scratch,
cross-XCD atomic coherence, the `v_bcnt` accumulator, pageable host
copies through SDMA,
and `s_sleep`'s architected delay. Generator changes live in
`lib/python/amdisa/` with
the generated output regenerated, not hand-edited.

**Debugger UX, docs, demos.** `mirage run --gdb` / `--gdb-ex`;
`docs/rocgdb-debugging.md`; four recorded walkthroughs under `demos/`
(quickstart,
watchpoint, fault, multi-wave), each with a `.md` explainer, a runnable
`.sh`, and an
asciinema `.cast`.

**Test tooling.** `tools/run_rocgdb_official.py` runs each `gdb.rocm`
file in a fresh,
bounded, verified Mirage daemon session and records a per-test summary
plus backend
hashes; `tools/repeat_rocgdb_test.sh` repeats a subset serially under an
`flock` (two
concurrent Mirage sessions collide and manufacture failures) for chasing
intermittents.
Both archive the daemon's `host.log` alongside `gdb.log`. The emulated
KFD's
diagnostics land there and nowhere else, and three debugger defects
fixed in this PR
were invisible until it was collected.

## Issue Tracking

Epic: #8276

Closes #8364, closes #8365, closes #8366, closes #8367, closes #8368,
closes #8369,
closes #8370, closes #8371, closes #8372, closes #8373, closes #8374,
closes #8375,
closes #8376, closes #8377, closes #8378, closes #8379, closes #8380,
closes #8381.

| Item | Work |
|---|---|
| #8364 | Authorize `DBG_TRAP` by real ptrace + daemon transport |
| #8365 | Key debug sessions by inferior pid so rocgdb can attach |
| #8366 | Complete the `DBG_TRAP` attach/detach lifecycle |
| #8367 | Enumerate compute queues in `GET_QUEUE_SNAPSHOT` |
| #8368 | Stop a GPU wave on `s_trap` for the KFD debugger |
| #8369 | Serialize stopped-wave state into the CWSR area layout |
| #8370 | Raise a debug event and serialize the CWSR when a wave traps |
| #8371 | Drive GPU wave debugging end to end with ROCgdb |
| #8372 | `mirage run --gdb` one-command debugging |
| #8373 | Deterministic single-step via `TRAPSTS.trap_after_inst` |
| #8374 | Reserve debugger memory for step-over |
| #8375 | Validate `SET_WAVE_LAUNCH_OVERRIDE` against the gfx9.4 trap
mask |
| #8376 | GPU address watchpoints |
| #8377 | Illegal instruction reported as SIGILL |
| #8378 | Memory-access violation reported as SIGSEGV |
| #8379 | Private/scratch reads for `print` / `info args` |
| #8380 | Multi-wave GPU kernel debugging |
| #8381 | ROCgdb debugging guide and demos index |

Built on the already-merged foundations #8280, #8281, #8353 and #8363.

## Test Plan

1. **Official ROCgdb suite.** All 89 `gdb.rocm` test files, each in its
own fresh,
   bounded Mirage daemon session, run serially via
`tools/run_rocgdb_official.py --profile mi350x`. The suite is upstream
ROCgdb's own,
   written for hardware and unmodified.
2. **rocjitsu unit/integration suite.** Full non-benchmark `ctest` run,
including the 66
test cases added here (19 in the new `wave_debug_test.cpp`, 34 in
`kfd_ioctl_test.cpp`,
   the rest across shared-infra, VM, interposer and decode tests).
3. **End-to-end smoke test in CI.** New `RocgdbDebug.KernelBreakpoint`
ctest target drives
real ROCgdb through `mirage run` to set a kernel breakpoint, stop, read
wave registers,
and continue to a correct result. Gated on `rocgdb` and `hipcc` being
present; skips
   cleanly otherwise.

## Test Result

Both suites green at `c998c5cd` (branch tip):

| Suite | Result |
|---|---|
| Official ROCgdb `gdb.rocm` | **89/89 files pass, 0 FAIL**. 2376 PASS
over 2407 executed assertions (9 UNSUPPORTED, 7 KFAIL, 3 XFAIL, 6
WARNING, 6 UNTESTED) |
| rocjitsu `ctest` (non-benchmark) | **2492 tests, 0 failures** (4
skipped by design) |

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit e5109e3)
The test named a gfx950 config that had been renamed out from under it, and
described a part with 32 CUs per XCD. Both facts came from the same commit:
aligning the MI350X configs with the captured KFD topology renamed
gfx950_cdna4.json to gfx950_mi355x.json and widened a shader array from eight
CUs to nine. Neither branch touched the other's files, so the merge was clean
and the breakage appeared only once both were on develop -- every test in the
suite threw before reaching an assertion, because the fixture could not open
its config at all.

Name the config that ships and derive the geometry from it: four shader
engines of nine CUs is 36 per XCD and 288 across the eight, which is what the
topology instantiates. The active-CU count the driver reports is still 256,
since 1024 SIMDs at four per CU excludes the harvested slots, but nothing here
models a harvest mask and every instantiated slot is schedulable -- so the
count these tests must agree with is the physical one.

Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com>
(cherry picked from commit efa24f2)
…t calls (#10376)

## Motivation

hipTensor and rocThrust gfx1250 objects were refused at code-object load
time with indirect branch or call target recovery is not implemented for
relocated kernel text on s_swap_pc_i64. A refused object leaves HIP
holding an invalid kernel handle, so the failure surfaced as a segfault
rather than a diagnostic: the hipTensor smoke suite lost
bilinear_contraction,
scale_contraction_with_unary_ops and trinary_scale_contraction, and
rocThrust lost the whole binary_search set. Separately,
translating large device libraries was slow enough to matter at load
time — an 8.3 MB hipTensor object took 510 s and 2.4 GB on unmodified
develop, against a 30 s and 4096 MiB per-object budget.

## Technical details

Three independent causes. First, patch_recovered_builder_fixups
regenerated a recovered builder's delta into the pair the consumer
reads; since append_pc_delta_builder emits only the add and leaves the
original getpc in place, a lane-banked dispatcher — which builds the
address in a scratch pair, stashes it through v_writelane_b32 and
restores it into a different pair — got an add against whatever the
consumer's register happened to hold, and a getpc/add pairing the next
translation read as a code address naming no body (117 of 4640 fixups on
one object). PcValue and IndirectCallFixup now carry the producer's pair
alongside the consumer's, and that pair is part of the rewritten-region
reuse key, so two consumers sharing a range while naming different pairs
can no longer silently inherit one replacement. Second, the kernarg
admission was fenced off from objects defining device functions;
rocPRIM's trampoline_kernel calls a device function pointer the host
wrote into kernarg, and admitting it promises every externally
resolvable .text symbol still names its body. That obligation is now
discharged by adopting the exported bodies, triggered on the admission's
own fact rather than a proxy — a kernarg-supplied target cannot be
recovered, so translation cannot fold the transfer and both passes see
the identical set, which keeps the adopted set a fixed point. Third,
run_block_dataflow popped a FIFO seeded in block-index order; recovered
indirect-call edges give a shared callee block one predecessor per call
site (5,646 on the largest object), so that O(predecessors × pairs) join
was recomputed ~151 times per block — 84.8 M visits per round.
Reverse-postorder
scheduling reduces that to 558 k and the object from 556.8 s to 29.4 s,
with output byte-identical because the least fixed point of a monotone
join does not depend on visit order. The lane-recovery retained-fact
bound also rises to 1<<26, which the 4.65/4.70 MB contraction objects
need to avoid abandoning recovery wholesale.

(cherry picked from commit 8b7763e)
@atgutier
atgutier force-pushed the users/agutierr/release-bkc-10.1-hotswap-update-20260819 branch from 4c879df to c3f4195 Compare August 19, 2026 23:38
@atgutier
atgutier requested a review from amd-arosa as a code owner August 19, 2026 23:38
@github-actions github-actions Bot added documentation Improvements or additions to documentation github actions Pull requests that update GitHub Actions code labels Aug 19, 2026
@ScottTodd

Copy link
Copy Markdown
Member

Did you mean to push 27 commits here? Can you update the PR description?

@kuhar

kuhar commented Aug 19, 2026

Copy link
Copy Markdown
Member

Did you mean to push 27 commits here? Can you update the PR description?

probably safer than doing a one-off rebase and separate validation, even if not strictly necessary for hotswap

@ScottTodd

Copy link
Copy Markdown
Member

Ok, as long as it's intentional :)

@atgutier

Copy link
Copy Markdown
Contributor Author

Did you mean to push 27 commits here? Can you update the PR description?

yes. it's the safest option as i don't want to manually apply fixes for conflicts or selectively pull in changes.

most of the code is well isolated from anything in rocm or even hotswap so it is safe.

@atgutier
atgutier merged commit 44405e9 into release/bkc/therock-10.1-20260811 Aug 20, 2026
26 checks passed
@atgutier
atgutier deleted the users/agutierr/release-bkc-10.1-hotswap-update-20260819 branch August 20, 2026 00:04
ScottTodd added a commit to ROCm/TheRock that referenced this pull request Aug 20, 2026
## Motivation

This includes ROCm/rocm-systems#10444 which
itself is 27 cherry-picks. See ROCm/rockrel#88
for background on the BKC release channel.

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/TheRock/blob/main/CONTRIBUTING.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation github actions Pull requests that update GitHub Actions code organization: ROCm project: rocjitsu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants