Repository navigation
[rocjitsu] code rewrite condition validator - #9560
Conversation
❌ PR Check — Action Required
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🚫 Please fix the failed policies before requesting reviews. The following policy checks failed:
The |
34a913c to
c8ece44
Compare
There was a problem hiding this comment.
BLOCKER
[BLOCKER] emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/code/dbt/binary_translator.cpp:1570 —
External entries are split as executable roots but are not guaranteed to be emitted or relocated
Reasoning: text_external_block_leaders() adds global/weak STT_FUNC and STT_NOTYPE symbols and
address-taken text entries to the CFG leader set. However, kernel_translation_scopes() at lines 791-
833 still roots emitted translation scopes only at kernel descriptors. An externally visible
function that is not descriptor-reachable may therefore receive no output placement.
The patcher’s existing contract confirms the mismatch: code_object_patcher.cpp:449-525 only requires
relocation-referenced text symbols to have mappings. An unreferenced symbol without a mapping
retains its old value. The final verifier rediscovers that stale symbol at lines 1381-1390 and
rejects it at lines 1392-1398 because it is not a decoded output block start. Without verification,
the symbol can instead silently point into unrelated translated bytes.
Evidence: Release corpus CI rejects two previously expected-success code objects with:
error: residual-rewrite: rewrite-discharge verification found an invalid final executable entry
Translation and materialization succeeded; the failure occurs only when reconstructing entries from
the produced ELF, demonstrating that the source and final entry models disagree.
Suggested fix: Establish one authoritative external-entry model across reachability, emission, text-
offset relocation, and verification. Either:
-
emit supported externally addressable functions as standalone translation roots and require an
output mapping for each; or -
restrict text_external_block_leaders() to entry forms the translator can actually preserve,
instead of treating symbol visibility alone as sufficient.
Add a regression with an unreferenced global/weak text function and verify that its body and symbol
either survive at a relocated address or cause translation to fail before materialization.
Confidence: high.
MAJOR
[MAJOR] emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/code/dbt/binary_translator.cpp:1570-1589 —
Adding every symbol entry to full indirect-CFG discovery makes ordinary translation time out
Reasoning: The new external-leader set is passed to BasicBlock::build() even when
verify_rewrite_discharge is disabled. That routine runs the complete indirect-branch recovery/
dataflow pipeline. In indirect_branch_discovery.cpp:1251-1260, every analysis block linearly
searches every external leader; this is O(blocks × external_entries) and is repeated during fixed-
point discovery. Large packaged objects can contain many blocks and many function symbols.
The optional final verifier repeats another complete BasicBlock::build() at
binary_translator.cpp:1392, and idempotence can add another full translation, further multiplying
the cost.
Evidence:
-
ASan/UBSan uses the ordinary translator without the new verification option and times out two
expected-success objects after 90 seconds. -
GCC sanitizer times out the same large object after 180 seconds.
-
Release verification times out two objects after 60 seconds.
-
These are deterministic corpus regressions; unit tests, TSan, formatting, and TheRock checks pass.
This also affects runtime HSA-tool translation because the expensive external-leader analysis is
unconditional, not limited to the offline audit.
Suggested fix:
-
Replace the repeated linear external-entry lookup with an offset set/bitmap or block-index map.
-
Separate CFG split boundaries from indirect-analysis roots so symbol boundaries do not
unnecessarily expand heavyweight dataflow. -
Reuse a compact final boundary/entry map for discharge verification instead of rerunning indirect-
target discovery over the materialized ELF. -
Add large-object performance coverage using the corpus objects that currently exceed 60/90/180
seconds.
Confidence: high.
7e78462 to
a70cfa1
Compare
kuhar
left a comment
There was a problem hiding this comment.
The external-entry model still needs one shared policy across entry recovery and text-symbol relocation (binary_translator.cpp:348, code_object_patcher.cpp:531). Entry recovery only collects R_AMDGPU_ABS64 references to STT_FUNC/STT_NOTYPE, while the patcher requires a mapping for any allocated zero-addend SHT_RELA reference to a non-STT_SECTION .text symbol. A targeted probe at a non-emitted .text offset was caught early only for STT_FUNC + ABS64; STT_OBJECT + ABS64 and STT_FUNC + types 1/2/12 completed translation before the patcher rejected them, and STT_OBJECT at a mapped offset was accepted where STT_FUNC was rejected. We should define the supported relocation/symbol predicate once and use it in both paths, with regressions for the non-ABS64 and symbol-type cases.
3e374d6 to
1e1099b
Compare
4a28a48 to
8894211
Compare
|
@lua1235 can you rebase and address #9560 (comment) ? |
81ba440 to
68e0a06
Compare
kuhar
left a comment
There was a problem hiding this comment.
Left an inline reply and 2 new comments
d07d796 to
431046c
Compare
|
Gfx1250F32K128WmmaAcceptsOperandsThePackedF16PathRejects now fails rewrite-discharge verification due to a (small) upstream inconsistency. The translated scaled-WMMA form preserves an SGPR SRC2, but the final-stream decoder currently rejects that. Will investigate. |
kuhar
left a comment
There was a problem hiding this comment.
LGTM % the failure you noted above
6931896 to
7cf32ab
Compare
38a5a91 to
979e170
Compare
979e170 to
59e47c4
Compare
## Motivation Add another rewrite validator similar to the idempotence check. This confirms that no registered rewrite condition remains after rewrites are applied. (Idempotence would not, for example, detect a rewrite that happened to be the identity rewrite). ## Technical Details - refactor rewrites into central registry shared by both the existing rewrite system and the new validator. -Include operand-driven rewrites in the refactored registry, since they were not representable by the existing opcode-keyed table. - Implement read only pass over translated binary to check for remaining rewrite sites. ## Issue Tracking N/A ## Test Plan Enable this checking pass over the existing rewrite tests + add tests for remaining added interface. ## Test Result Tests pass ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. (cherry picked from commit 1d78662)
…t calls (#10444) ## Motivation hipTensor and rocThrust gfx1250 objects are refused at code-object load time with "indirect branch or call target recovery is not implemented for relocated kernel text" on `s_swap_pc_i64`. A refused object leaves HIP holding an invalid kernel handle, so the failure surfaces as a segfault rather than a diagnostic. The fix is upstream as #10376; this backports it to `release/bkc/therock-10.1-20260811`. ## Commits Cherry-Picked | # | Upstream | PR | Backport | Subject | |---|----------|-----|----------|---------| | 1 | `b6d838028e` | #10197 | `846fb16d12` | [rocjitsu][docs] Refresh architecture support documentation | | 2 | `5c8e33e391` | #10200 | `db5c58b1be` | [rocjitsu] Define standalone decoder ownership | | 3 | `be1f91998f` | #10199 | `ed102baf1c` | [rocjitsu] Apply async IOFFSET to LDS addresses | | 4 | `fda635e5b4` | — | `ad32ff7c18` | [rocjitsu] Extract per-scope rollback and object-level checks from translate() | | 5 | `8f75eb781e` | — | `4c0cb05207` | [rocjitsu] Emit leading V_NOPs ahead of a gfx1250 MODE write | | 6 | `00aa1e260f` | #9941 | `ded43b1787` | fix(rocjitsu): disable zero-count TDM descriptors | | 7 | `c810647609` | #10192 | `50ca7f1732` | [rocjitsu] Apply MUBUF LDS-load instruction offset to the LDS address | | 8 | `1d7866269d` | #9560 | `2e6bf26943` | [rocjitsu] code rewrite condition validator | | 9 | `95b6c410bb` | #10175 | `8418901076` | [rocjitsu] Fix ds_read_b64_tr_b16 lane layout in gfx950 simulator | | 10 | `bfaebcfb5c` | #10240 | `c4eac1e1f7` | [rocjitsu][util] Add result and diagnostic utilities | | 11 | `dbb7221fa7` | #9775 | `cdd0993a5f` | fix(rocjitsu): route tensor DMA through wave VMID | | 12 | `2779247222` | #9779 | `7bfeba6f5d` | feat(rocjitsu): Lazy vgpr allocation system | | 13 | `a659172a31` | #9936 | `180cf00325` | fix(rocjitsu): honor TDM tensor bounds across iterations | | 14 | `60c29efb39` | #10204 | `370bfb9796` | [rocjitsu] Fix gfx950 decoder extension handling | | 15 | `c1dba564e7` | #10345 | `536110a083` | [rocjitsu] Model gfx1250 wave ABI state | | 16 | `7f71be9f32` | #10346 | `3f922ad5f2` | [rocjitsu] Model gfx1250 barrier synchronization | | 17 | `bea7ac8937` | #10368 | `7b5ecf6c7b` | [rocjitsu] Replace decoder exceptions with results | | 18 | `f2e878694e` | #10371 | `8cf750d23f` | [rocjitsu] Dispatch operand validation by type | | 19 | `dd619bed77` | #9735 | `1a612c47c0` | fix(rocjitsu): respect same-wave LDS ordering | | 20 | `c5fae8d2b2` | #10137 | `4360c4b978` | [rocjitsu] Rename gfx1250 to cdna5 for enums, configs, amdisa codegen helpers and generated symbols. (NFC) | | 21 | `b05a587f89` | #10274 | `bfb5f9fab0` | [rocjitsu] Align CDNA4 MI350X configs with captured topology | | 22 | `f81f9d40db` | #10362 | `8177b5725c` | feat(rocjitsu): cdna5 OOB behavior implementation | | 23 | `53a939bd00` | #10283 | `141b109432` | [rocjitsu] add per-XCD workgroup dispatch observability | | 24 | `a0fb11bc17` | #10414 | `6bb4c5a959` | [rocjitsu] Refactor test encoding handling in execution tests (NFC) | | 25 | `e5109e315e` | #9844 | `322ac26b44` | feat(emulation): debug emulated GPU kernels with real ROCgdb | | 26 | `efa24f23c9` | — | `c3c9641073` | rocjitsu: Point the XCD distribution test at the config that exists | | 27 | `8b7763ef5e` | #10376 | `c3f419555a` | [rocjitsu] Translate gfx1250 lane-banked and kernarg-supplied indirect calls | ## Issue Tracking [ROCM-29436] ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. [ROCM-29346]: https://amd-hub.atlassian.net/browse/ROCM-29346?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [ROCM-29436]: https://amd-hub.atlassian.net/browse/ROCM-29436?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Signed-off-by: Tony Gutierrez <anthony.gutierrez@amd.com> Signed-off-by: hanhanW <hanhan0912@gmail.com> Co-authored-by: Jakub Kuderski <jakub@nod-labs.com> Co-authored-by: James Newling <james.newling@gmail.com> Co-authored-by: Fei Kou <feikou@meta.com> Co-authored-by: Austin Lu <austin.lu@amd.com> Co-authored-by: Fei Kou <feikou@anthropic.com> Co-authored-by: Erick Ochoa Lopez <eochoalo@amd.com> Co-authored-by: Han-Chung Wang <hanhan0912@gmail.com> Co-authored-by: Rosa, Ada <arosa@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Motivation
Add another rewrite validator similar to the idempotence check. This confirms that no registered rewrite condition remains after rewrites are applied. (Idempotence would not, for example, detect a rewrite that happened to be the identity rewrite).
Technical Details
-Include operand-driven rewrites in the refactored registry, since they were not representable by the existing opcode-keyed table.
Issue Tracking
N/A
Test Plan
Enable this checking pass over the existing rewrite tests + add tests for remaining added interface.
Test Result
Tests pass
Submission Checklist