Skip to content

[rocjitsu] Reuse VM snapshots across dispatch execution - #12147

Merged
newling merged 5 commits into
ROCm:developfrom
newling:users/jnewling/rocjitsu-dispatch-vm-snapshot
Oct 7, 2026
Merged

newling merged 5 commits into
ROCm:developfrom
newling:users/jnewling/rocjitsu-dispatch-vm-snapshot

Conversation

@newling

@newling newling commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

#12217 already caches instruction-fetch VM snapshots per CU. This PR extends reuse to data operations, kernarg preloads, and scratch validation, avoiding their repeated global-registry acquisitions. Waves share the snapshot captured during AQL admission; the per-CU instruction cache remains the fallback.

Waves retain the admission root across VM-root replacement; changes within that root remain visible. Invalidation, unregister, and reset revoke access. Fetch and uncommitted wave setup retry with fresh access when available. Revoked in-flight data operations abort the dispatch and report a runtime memory-violation exception without successful completion, since replay could repeat partial effects. The deferred-load regression also reproduces develop’s missing completion/exception and verifies the PR’s failure notification.

A hipBLASLt comparison used 20 measured process runs per revision, plus two excluded warmups each, executing the same first 100 solutions of the 513 × 513 × 513 sk_sgemm_quick group Cijk_Ailk_Bjlk_S_B_UserArgs_01. The PR used less wall time in all 20 pairs: 9.7% lower paired geometric mean wall time, with a 95% paired-bootstrap interval of 7.2–12.6%. All 4,000 measured solution validations passed. This is one workload on a shared host, with substantial timing variation.

All 20 paired process wall times

Measured develop 9219e7f966 against PR ef5b637212, before the setup-recovery follow-up. Identical artifacts/configuration, Clang 23 Release builds, six-CPU emulator budget, and OMP_NUM_THREADS=1. Odd pairs ran develop first; even pairs ran the PR first. Every measured run is included. The bootstrap resamples these 20 pairs and describes variability within this experiment.

Pair Develop (s) PR (s)
1 54.46 50.58
2 52.44 41.62
3 53.45 50.83
4 54.54 50.70
5 53.73 50.91
6 54.34 50.85
7 53.66 50.78
8 53.08 51.21
9 53.84 50.51
10 54.42 50.85
11 53.18 50.78
12 53.58 50.95
13 50.03 40.16
14 51.49 38.91
15 54.58 50.92
16 53.77 50.78
17 45.97 39.80
18 52.28 45.05
19 43.56 39.43
20 54.56 46.67

Focused Release/ThreadSanitizer tests, GCC UBSan compilation, and changed-file pre-commit checks pass, including controls for successful setup recovery and runtime-visible data-access failure.

Motivated by hipBLASLt emulation timeouts in ROCm/rocm-libraries#10534.

JIRA ID: AIHPBLAS-3988

@therock-pr-bot

therock-pr-bot Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@newling
newling force-pushed the users/jnewling/rocjitsu-dispatch-vm-snapshot branch from 5f3f9ab to 982e364 Compare September 23, 2026 22:00
@newling
newling requested review from IanWood1 and kuhar September 23, 2026 22:19
@newling
newling marked this pull request as ready for review September 23, 2026 22:26
@newling
newling requested review from a team and atgutier as code owners September 23, 2026 22:26

@kuhar kuhar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one note to consider

@newling
newling force-pushed the users/jnewling/rocjitsu-dispatch-vm-snapshot branch from 982e364 to 58fca88 Compare September 24, 2026 18:46
@newling
newling requested a review from kuhar September 24, 2026 19:36
@newling
newling force-pushed the users/jnewling/rocjitsu-dispatch-vm-snapshot branch 4 times, most recently from 4565e3b to 3551782 Compare September 30, 2026 18:38

@kuhar kuhar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@newling
newling force-pushed the users/jnewling/rocjitsu-dispatch-vm-snapshot branch 2 times, most recently from 0ed97a5 to 68579f5 Compare October 5, 2026 18:14
Discard uncommitted AQL wave setup and schedule a retry when kernarg preload or scratch validation observes revocation. No guest code has executed and scratch provisioning is idempotent, so setup can recover without publishing a terminal queue fault. Cover both paths with controls, successful completion, and a two-wave workgroup that must reclaim its reservations.
@newling
newling force-pushed the users/jnewling/rocjitsu-dispatch-vm-snapshot branch from 68579f5 to 8a11a3d Compare October 6, 2026 21:01
@newling
newling merged commit 5894390 into ROCm:develop Oct 7, 2026
87 checks passed
@systems-assistant

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants