Repository navigation
[rocjitsu] Support tinygrad native KFD on RDNA and CDNA - #12526
Merged
Merged
Conversation
kuhar
force-pushed
the
users/kuhar/tinygrad-rdna3-pm4
branch
from
September 30, 2026 13:54
cfcce34 to
647b41f
Compare
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
kuhar
force-pushed
the
users/kuhar/tinygrad-rdna3-pm4
branch
from
September 30, 2026 14:22
647b41f to
4d3b1d8
Compare
This was referenced Sep 30, 2026
kuhar
added a commit
that referenced
this pull request
Oct 1, 2026
## Motivation Represent AQL and PM4 as the initial packet format of a compute queue, with one owner for registration, VM lifetime, CP register state, dispatches, and teardown. ## Technical Details - Replace separate AQL, native PM4, and DRM queue ownership with `ComputeQueueConfig` and `ComputeQueueRecord`. - Use a common compute queue binding factory while retaining existing PM4 entry points as adapters. - Keep root packet format and cursor units explicit: AQL uses packet slots; PM4 uses dwords. - Route native KFD PM4 queues through the common owner and preserve target-XCC selection. - Retain VM snapshots and durable cursor publication across retries, and share queue suspension and cancellation with shader dispatches. - Keep existing AQL vendor IB behavior for the follow-up PR. KFD reference: [`set_queue_properties_from_user`](https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c#L338) maps COMPUTE and COMPUTE_AQL to the same compute queue type, with format configured separately. Existing exception handling is retained; this change adds no explicit throw statements or try/catch blocks. ## Issue Tracking Related discussion: #12526. This PR establishes the shared queue model used by the follow-up IB fix. ## Test Plan - Build `rocjitsu_tests` and `interposer_dup_test` with AMD Clang in Release mode, including the existing `-Werror` checks. - Run selected AQL, PM4, VM, queue registration, doorbell, completion, KFD, XCD, MES, and PCI unit tests. - Run all `InterposerPm4Test` cases with the gfx1100_w7900 and gfx1201_r9700 emulator configurations. - Run the command processor/SDMA, SDMA frontend, MES frontend, and GPU memory architecture checks. ## Test Result - Both build targets passed for this commit independently of the follow-up. - 280 selected unit tests passed. - 64 PM4 interposer tests and four architecture checks passed. - Commit hooks and `git diff --check` passed. - No physical GPU execution was used for these results. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Jakub Kuderski <jakub@nod-labs.com>
atgutier
reviewed
Oct 5, 2026
atgutier
left a comment
atgutier
left a comment
Contributor
There was a problem hiding this comment.
[MAJOR] emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/command_processor.cpp:2645 — CU idle
can complete an AQL PM4 buffer without executing it
Reasoning: If a barrier-bit PM4 buffer follows a running kernel, on_cu_idle() can reach it when that
kernel finishes. The callback marks every non-kernel entry complete, publishes its completion
signal, and advances the queue. It never executes the buffer, so a dependent packet can proceed
while the buffer’s memory writes are missing.
Evidence: The PR admits vendor buffers as executable Pm4Ib entries in aql/
aql_packet_processor.cpp:239 and adds execution through execute_aql_pm4() in
command_processor.cpp:4744. The idle path at command_processor.cpp:2633–2658 bypasses that function.
The new test starts with a PM4 buffer, so it does not cover a preceding kernel’s idle transition.
Suggested fix: Gate non-kernel retirement in on_cu_idle() on the same PM4 execution and retry logic
used by the other queue paths. Add a kernel → barrier-bit PM4 memory write → dependent successor
regression.
Confidence: High. Verified against the immutable PR head; not dynamically reproduced.
kuhar
force-pushed
the
users/kuhar/tinygrad-rdna3-pm4
branch
from
October 6, 2026 02:40
4d3b1d8 to
d5aa2b4
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Low-level queue and cross-XCD execution changes need corrections and final human validation.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Extends rocjitsu’s existing PM4 support so tinygrad’s native KFD backend can initialize and execute workloads on simulated RDNA and CDNA devices.
Changes:
- Publishes configured hardware-block versions and page-aligns KFD allocation backing.
- Executes AQL vendor PM4 buffers across XCDs, adding ordering, predication, and validation.
- Corrects gfx1151 register capacity and adds discovery, queue, and instruction regressions.
| File | Description |
|---|---|
| emulation/rocjitsu/tests/sysfs_test.cpp | Tests discovery versions and validation. |
| emulation/rocjitsu/tests/readlane_selector_test.cpp | Tests CDNA VCC spill reloads. |
| emulation/rocjitsu/tests/kfd_ioctl_test.cpp | Tests native queues, multi-XCD execution, and allocations. |
| emulation/rocjitsu/tests/config_test.cpp | Tests gfx1151 register isolation. |
| emulation/rocjitsu/tests/aql_queue.h | Builds valid vendor PM4 packets. |
| emulation/rocjitsu/tests/aql_packet_processor_test.cpp | Tests vendor-packet decoding and rejection. |
| emulation/rocjitsu/tests/amdgpu_vm_test.cpp | Tests PM4 ordering, retries, and faults. |
| emulation/rocjitsu/schemas/simulation_config.fbs | Defines configurable IP versions. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/pm4/pm4.h | Adds predication opcode and dispatch permission. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/pm4/pm4_packet_processor.h | Adds execution-context XCD identity. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/pm4/pm4_packet_processor.cpp | Handles predication, HDP flushes, and packet lengths. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/gpu_queue_registry.h | Clarifies initial-cursor documentation. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/dispatch_entry.h | Tracks vendor-buffer locations. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/command_processor.h | Declares vendor-buffer execution helper. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/command_processor.cpp | Integrates vendor execution and cross-XCD completion. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/aql/aql_packet_processor.h | Defines decoded vendor-buffer state. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/vm/amdgpu/aql/aql_packet_processor.cpp | Validates and admits vendor buffers. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/kmd/linux/sysfs.h | Stores discovery versions. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/kmd/linux/sysfs.cpp | Publishes discovery files. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/kmd/linux/simulated_kfd.cpp | Updates allocation backing and queue initialization. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/config/kfd_device_config.h | Adds owned IP-version configuration. |
| emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/config/config_common.h | Copies and validates IP versions. |
| emulation/rocjitsu/configs/guest_gfx950_on_simulated_gfx942.json | Adds guest gfx950 discovery versions. |
| emulation/rocjitsu/configs/guest_gfx950_on_gfx942.json | Adds guest gfx950 discovery versions. |
| emulation/rocjitsu/configs/guest_gfx950_on_gfx1201.json | Adds guest gfx950 discovery versions. |
| emulation/rocjitsu/configs/gfx950_mi355x.json | Adds MI355X discovery versions. |
| emulation/rocjitsu/configs/gfx950_mi355x_kmd.json | Adds KFD MI355X discovery versions. |
| emulation/rocjitsu/configs/gfx950_mi355x_kmd_2gpu.json | Adds dual-GPU discovery versions. |
| emulation/rocjitsu/configs/gfx942_cdna3.json | Adds CDNA3 discovery versions. |
| emulation/rocjitsu/configs/gfx942_cdna3_kmd.json | Adds KFD CDNA3 discovery versions. |
| emulation/rocjitsu/configs/gfx1201_r9700.json | Adds R9700 discovery versions. |
| emulation/rocjitsu/configs/gfx1151.json | Adds discovery versions and 106 ordinary SGPRs. |
| emulation/rocjitsu/configs/gfx1100_w7900.json | Adds W7900 discovery versions. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Tinygrad's native AMD backend submits PM4 directly through KFD. Publish configured device IP versions, handle HDP flushes, and back KFD allocations with whole pages for compiler-widened loads. Reuse the unified compute queue lifecycle and initialize native PM4 cursors independently of their writeback storage. Execute CDNA AQL vendor PM4 buffers on every XCD through the existing packet processor, retaining retry progress and completion ordering. Handle per-XCD predication and GFX9 ACQUIRE_MEM packet lengths. Reject malformed buffers and shader dispatches inside vendor buffers. Require PM4 execution before retiring non-kernel entries on CU idle, as on the other queue paths. Cover a kernel followed by a barrier-bit vendor buffer and a dependent successor, including blocked waits, no replay of atomic effects, and terminal faults without completion. Stop vendor PM4 execution when a peer has faulted the shared grid, even before the local fault inbox is drained. Cancel blocked vendor streams and release their VM snapshots when the inbox removes a faulted dispatch. Retire PRED_EXEC and COND_EXEC skips to the updated root-ring cursor, including writeback retries and wraps. Provide gfx1151's 106 ordinary SGPRs and cover VCC spill reloads. Validate with 715 focused tests, native-launch configuration handoff, four architecture-boundary checks, and pre-commit. Tinygrad matmul, gradient, and JIT checks pass on simulated gfx1100, gfx1151, gfx1201, gfx942, and gfx950; CDNA uses eight-XCD AQL+PM4 profiles. Outputs match retained W7900 reference arrays within tolerance.
kuhar
force-pushed
the
users/kuhar/tinygrad-rdna3-pm4
branch
from
October 6, 2026 03:17
d5aa2b4 to
dac88af
Compare
atgutier
approved these changes
Oct 6, 2026
Contributor
TheRock Submodule Bump ActivityNewest first
|
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
- AQL/PM4 multi-XCD test (PR #12526, ec0ae87): the step-count loop can finish before the host doorbell poller retries a blocked IB. Use a five-second deadline with brief sleeps, and observe another command-processor pass on every XCD before checking blocked-IB state. - SDMA indirect-buffer test (PR #11396, bdc4aea): draining the doorbell inbox can return between asynchronous service turns, before fence, read-pointer and interrupt results are ready. Wait for queue completion before checking them; includes the existing SDMA fix. - Satisfied SDMA poll test (PR #11396, bdc4aea): the same inbox drain can expose an unpublished read pointer even when both polls are satisfied. Wait for queue completion before checking the pointer. - Idle-engine CPU test (PR #13631, 4ed8951): the process CPU clock charges sanitizer/helper work against idle workers. Sample each worker's CPU time across the idle window, retaining the 50 ms budget. - ROCr HSA initialization reports (tests added in PR #5053, ca4b97e): the prebuilt runtime's uninstrumented async-event handoff can produce TSan allocation/reclamation reports after assertions pass. Add HsaTest.InitSucceeded and HsaTest.GpuAgentFound to the TSan CI skip list, retaining coverage in the other lanes. - SDMA test hangs (explicit helper introduced in PR #11396, bdc4aea): completion waits never expire when a packet cannot retire. Bound both explicit and host-polled helpers with a five-second default so broken polls fail their assertions. Preserve the queue for later completion and cover timeout/resume behavior with regression tests. Validation: focused Release, UBSan, ASan/UBSan and TSan selections (existing local TSan runtime CHECK excluded); two Peanut review rounds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tinygrad's native AMD backend submits PM4 directly through KFD. Publish
configured GC, SDMA and NBIF versions and handle HDP flushes so it can
initialize simulated RDNA and CDNA devices. Route native RDNA compute
queues through the existing PM4 executor, preserving ring writeback,
pause/resume and teardown. Keep queue support independent of discovery
metadata. Back KFD allocations with whole pages for compiler-widened
loads.
Execute CDNA's AQL vendor PM4 buffers on every XCD, retaining retry
progress and completion ordering. Handle per-XCD predication and GFX9
ACQUIRE_MEM packet lengths. Reject malformed packets and unsupported
shader dispatches inside vendor buffers.
Configure gfx1151's 106 ordinary SGPRs, with VCC stored separately, and
fix CDNA3/4 V_READLANE destinations exposed by LLVM's VCC spill reloads.
Regenerate the affected executors and cover discovery, queue lifetimes,
multi-XCD ordering and the failing instruction encodings.
Validate unmodified tinygrad at 7bfaa18daaec on simulated gfx1100,
gfx1151, gfx1201, gfx942 and gfx950. The selected core, ops, JIT and
tensor-core suites pass 492 tests with 20 skips on each RDNA target and
491 with 21 skips on each CDNA target. CDNA uses full eight-XCD profiles
and tinygrad's normal AQL+PM4 path. Gfx950 FP8 requires the LLVM
scale-operand fix, tested with LLVM 22.1.8.
Matmul, gradient, JIT and small MNIST samples match retained W7900
results within tolerance. The full build and 471 focused simulator and
configuration checks pass; the generator tests also pass. Broader CTest
still has 14 DBI/DBT failures under separate investigation and six
hardware-test fixture failures before kernel loading or dispatch.
Stacked on #12296.