Repository navigation
[rocjitsu] Preserve contiguous memory transaction atomicity - #14170
Merged
Merged
Conversation
❌ PR Check — Action Required
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🚫 Please fix the failed policies before requesting reviews. The following policy checks failed:
The |
kuhar
force-pushed
the
users/kuhar/rocjitsu-communication-transactions
branch
from
October 23, 2026 17:35
b6b980f to
7c8e801
Compare
atgutier
reviewed
Oct 23, 2026
Lane-order coalescing splits contiguous records when active lane addresses are permuted. Independent XCD/L2 instances also copy shared host backing without coordinating access. Both can expose torn payloads to communication protocols that accept a flag and payload from the same load. Group contiguous, non-overlapping active operands by address in L1 and translated requests. Gather stores and scatter completed load snapshots, preserving completed-prefix progress across retries. Keep gaps, clipped operands, overlapping addresses and wrapping accesses within their existing bounds and ordering. Coordinate mapped copies and atomics with padded locks keyed by resolved host storage, including GPU aliases. Pin trusted driver RAM for direct copies and retain fault protection for application memory. Release locks on faults and bound bulk transfers to individual transactions. Keep contiguous and monotone strided paths inline, avoid common staging and span allocations, and separate single-line copies from bulk loops. Use a native 32-bit Spinlock wait word to avoid shared proxy-counter traffic. Select backing-owned transaction observation policy independently of cache geometry, and let each CU inherit it. Use the gfx1250 protocol model and retain existing 128-byte compatibility behavior for physically unqualified targets. Exercise 64-byte boundaries without changing target defaults or claiming a universal hardware contract. Gather boundary-crossing lane fragments into their surrounding transactions. Retain translated backend batching for default lane-ordered runs. Cover lane permutations, inactive lanes, clipping, aliases, independent L2s, partial retries, fault recovery, and 64/128-byte transaction boundaries within unchanged cache lines. Document the evidence and qualification limits.
kuhar
force-pushed
the
users/kuhar/rocjitsu-communication-transactions
branch
from
October 23, 2026 19:47
7c8e801 to
cac5bd1
Compare
github-actions
Bot
added
the
documentation
atgutier
approved these changes
Oct 23, 2026
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
…14198) The narrow-policy and alias-exclusion fixtures issue 128-byte copies at nonzero host offsets. With only 128-byte backing alignment, these copies can straddle a host page and be rejected by try_copy_contiguous before either lock stripe is taken. Page-align both buffers so the tests consistently exercise publication across lock stripes. Fixes a flake introduced by #14170
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
Transaction snapshots introduced in 68b6345 (#14170) add overhead when overlapping operands take the sorting path and stripe releases notify shared standard-library waiter buckets. The same workloads also spend time walking byte-validity ranges. Prove overlap before sorting and retain the original lane-order fallback. Check and mark byte-validity ranges with word masks, preserving cheap single-byte probes and every bit outside the requested range. Park Linux storage-stripe waiters on private futex words with bounded timeouts. Release ownership with a store and recheck it after every wait return, so missed wakeups delay progress without losing it. Retain the existing Spinlock on other hosts and preserve transaction geometry, mapping lifetime, fault protection and retry progress. Cover range boundaries, overlap and callback ordering, publication through failed try_lock probes, and interrupted waits. Keep failure handling bounded without weakening the publication oracle. Route host futex waits and wakes through the resolved libc passthrough table rather than the interposed syscall entry. Keep host API selection outside util, and retain bounded progress before host initialization. Validation of host routing: 402 focused tests, 450 threaded executions, 39 selected TSan tests and a 22-test replay. Earlier validation covered independent sanitizer properties and all 66 communication cases, with the same 33 failures as the pristine baseline in the broader suite. Before the host API routing update, five balanced process pairs against the Oct 23 baseline reduce geometric mean ML process time by 1.28%, ML warm-kernel time by 0.65%, and CDNA3/4 small-kernel warm time by 6.99%. The IREE f32 gfx1100 t1 and Tensile subtile gfx1250 t16 regression probes improve by 8.98% and 5.26%; atomic-add warm time falls by 11.08% on gfx942 and 11.15% on gfx950. Measurements use GCC 16 Release/O3, x86-64-v4, fixed 2.5 GHz reserved CPUs 48-95, both workload locks, and wait checking enabled. These are paired time reductions across matching cases. Some other IREE and FP8 cases are 1.3-2.6% slower; the change does not improve every workload. Two final routing pairs pass all checked workloads. Relative to that optimized image, IREE gfx1100 t1 and gfx1201 t8 take 1.28% and 2.04% more time, Tensile takes 0.42% less time, and small-kernel warm time falls 0.70%. These narrow checks do not replace the earlier full-matrix measurements. Fixes a performance regression introduced by #14170.
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
Transaction snapshots introduced in 68b6345 (#14170) add overhead when overlapping operands take the sorting path and stripe releases notify shared standard-library waiter buckets. The same workloads also spend time walking byte-validity ranges. Prove overlap before sorting and retain the original lane-order fallback. Check and mark byte-validity ranges with word masks, preserving cheap single-byte probes and every bit outside the requested range. Park Linux storage-stripe waiters on private futex words with bounded timeouts. Release ownership with a store and recheck it after every wait return, so missed wakeups delay progress without losing it. Retain the existing Spinlock on other hosts and preserve transaction geometry, mapping lifetime, fault protection and retry progress. Cover range boundaries, overlap and callback ordering, publication through failed try_lock probes, and interrupted waits. Keep failure handling bounded without weakening the publication oracle. Route host futex waits and wakes through the resolved libc passthrough table rather than the interposed syscall entry. Keep host API selection outside util, and retain bounded progress before host initialization. Validation of host routing: 402 focused tests, 450 threaded executions, 39 selected TSan tests and a 22-test replay. Earlier validation covered independent sanitizer properties and all 66 communication cases, with the same 33 failures as the pristine baseline in the broader suite. Before the host API routing update, five balanced process pairs against the Oct 23 baseline reduce geometric mean ML process time by 1.28%, ML warm-kernel time by 0.65%, and CDNA3/4 small-kernel warm time by 6.99%. The IREE f32 gfx1100 t1 and Tensile subtile gfx1250 t16 regression probes improve by 8.98% and 5.26%; atomic-add warm time falls by 11.08% on gfx942 and 11.15% on gfx950. Measurements use GCC 16 Release/O3, x86-64-v4, fixed 2.5 GHz reserved CPUs 48-95, both workload locks, and wait checking enabled. These are paired time reductions across matching cases. Some other IREE and FP8 cases are 1.3-2.6% slower; the change does not improve every workload. Two final routing pairs check IREE gfx1100 t1, Tensile gfx1250 t16 and the CDNA3/4 small suite. Relative to the optimized image, process time changes by +1.28% and -0.42% for IREE and Tensile; small warm time falls 0.70%. Exclude a gfx1201 IREE control that failed to load its module. These narrow checks do not replace the earlier full-matrix measurements. Fixes a performance regression introduced by #14170.
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
Transaction snapshots introduced in 68b6345 (#14170) add overhead when overlapping operands take the sorting path and stripe releases notify shared standard-library waiter buckets. The same workloads also spend time walking byte-validity ranges. Prove overlap before sorting and retain the original lane-order fallback. Check and mark byte-validity ranges with word masks, preserving cheap single-byte probes and every bit outside the requested range. Park Linux storage-stripe waiters on private futex words with bounded timeouts. Release ownership with a store and recheck it after every wait return, so missed wakeups delay progress without losing it. Retain the existing Spinlock on other hosts and preserve transaction geometry, mapping lifetime, fault protection and retry progress. Cover range boundaries, overlap and callback ordering, publication through failed try_lock probes, and interrupted waits. Keep failure handling bounded without weakening the publication oracle. Route host futex waits and wakes through the resolved libc passthrough table rather than the interposed syscall entry. Keep host API selection outside util, and retain bounded progress before host initialization. Validation of host routing: 402 focused tests, 450 threaded executions, 39 selected TSan tests and a 22-test replay. Earlier validation covered independent sanitizer properties and all 66 communication cases, with the same 33 failures as the pristine baseline in the broader suite. Before the host API routing update, five balanced process pairs against the Oct 23 baseline reduce geometric mean ML process time by 1.28%, ML warm-kernel time by 0.65%, and CDNA3/4 small-kernel warm time by 6.99%. The IREE f32 gfx1100 t1 and Tensile subtile gfx1250 t16 regression probes improve by 8.98% and 5.26%; atomic-add warm time falls by 11.08% on gfx942 and 11.15% on gfx950. Measurements use GCC 16 Release/O3, x86-64-v4, fixed 2.5 GHz reserved CPUs 48-95, both workload locks, and wait checking enabled. These are paired time reductions across matching cases. Some other IREE and FP8 cases are 1.3-2.6% slower; the change does not improve every workload. Two final routing pairs check IREE gfx1100 t1, Tensile gfx1250 t16 and the CDNA3/4 small suite. Relative to the optimized image, process time changes by +1.28% and -0.42% for IREE and Tensile; small warm time falls 0.70%. Exclude a gfx1201 IREE control that failed to load its module. These narrow checks do not replace the earlier full-matrix measurements. Fixes a performance regression introduced by #14170.
kuhar
added a commit
that referenced
this pull request
Oct 24, 2026
Transaction snapshots introduced in #14170 add overhead when overlapping operands take the sorting path and stripe releases notify shared standard-library waiter buckets. The same workloads also spend time walking byte-validity ranges. Prove overlap before sorting and retain the original lane-order fallback. Check and mark byte-validity ranges with word masks, preserving cheap single-byte probes and every bit outside the requested range. Park Linux storage-stripe waiters on private futex words with bounded timeouts. Release ownership with a store and recheck it after every wait return, so missed wakeups delay progress without losing it. Retain the existing Spinlock on other hosts and preserve transaction geometry, mapping lifetime, fault protection and retry progress. Cover range boundaries, overlap and callback ordering, publication through failed try_lock probes, and interrupted waits. Keep failure handling bounded without weakening the publication oracle. Five balanced process pairs against the Oct 23 baseline reduce geometric mean ML process time by 1.28%, ML warm-kernel time by 0.65%, and CDNA3/4 small-kernel warm time by 6.99%. The IREE f32 gfx1100 t1 and Tensile subtile gfx1250 t16 regression probes improve by 8.98% and 5.26%; atomic-add warm time falls by 11.08% on gfx942 and 11.15% on gfx950. Measurements use GCC 16 Release/O3, x86-64-v4, fixed 2.5 GHz reserved CPUs 48-95, both workload locks, and wait checking enabled. These are paired time reductions across matching cases. Some other IREE and FP8 cases are 1.3-2.6% slower; the change does not improve every workload. Validation: 400 focused tests, 430 threaded executions, 37 selected TSan tests, independent sanitizer properties and all 66 communication cases. The broader suite retains the same 33 baseline failures. Fixes a performance regression introduced by #14170.
Contributor
TheRock Submodule Bump ActivityNewest first
|
kuhar
added a commit
that referenced
this pull request
Oct 26, 2026
Transaction-aware coalescing introduced in #14170, and optimized in #14215, still scans ordinary contiguous lane addresses one at a time. Check full Wave32 operands with a private VM stride predicate before the scalar scan. Select AVX-512 or AVX2 from the compile target without per-wave CPU queries or dispatch. Reject wrapping sequences and keep vector loads within the active addresses. Other masks, Wave64 and scalar-only builds retain the existing scan. Keep the standard address sort in one non-inline function shared by callback instantiations. Preserve transaction policy, overlap fallback and callback order without adding a custom sorter or an arith dependency. Five paired sweeps of the original stride binaries across seven small kernels show 0.43% fewer Wave32 instructions per warm launch on default V3 and 0.46% fewer on V4. Matched V3 builds use 0.43% fewer cycles; copy and gather use 1.04% and 0.63% fewer cycles. V3 atomic- add uses 0.12% more cycles. The measured original V4 binaries show 2.96% fewer Wave32 cycles, but that observation is sensitive to binary layout and does not isolate an algorithmic gain of that size. Wave64 cycle changes are likewise layout-sensitive and are not treated as benefits of this optimization. Measurements use GCC 16 Release/O3, matched baseline/candidate CPU targets, fixed 2.5 GHz reserved CPUs, both workload locks and wait checking enabled. Counters cover checked warm launches and synchronization, excluding startup and JIT. These are host CPU counts, not whole-application speedups. The original stride performance receipts predate the shared host- target prerequisite. They retain all 280 measured processes and their numerical, runtime, affinity and nonmultiplexed-counter proofs. No new performance measurements are attributed to the capability cleanup. Validate the shared-capability integration with V3/V4 builds, 255 CTests, 15 FP-mode helpers and changed-file hooks. Check 24 native compiler/target contexts, 212 include-order/override combinations, 25 synthetic probes, 20 property executables including ASan/UBSan, and 5,875,080 stride cases. All 144 header-only override checks pass; four full-SIMD Clang forced-zero contexts remain unsupported by the existing toolchain. Compare 18 stride assembly pairs byte-for-byte against the original. Preserve the lock's pause instruction and diagnose missing header dependencies in its header-first unit test. Run correctness checks under both workload locks outside the expired reservation, as requested. Two full local Peanut review rounds and focused follow-ups cover the prerequisite and its consumers.
This was referenced Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane-order coalescing splits contiguous records when active lane addresses are permuted, and independent XCD/L2 instances copy shared host backing without coordinating access. Both can expose torn payloads to communication protocols that accept a flag and payload from the same load.
Group contiguous, non-overlapping bytes by transaction, including fragments of 12-byte or unaligned 16-byte operands. Gather stores and scatter completed load snapshots while preserving progress across retries. Coordinate mapped copies and atomics through padded locks keyed by resolved host storage, including aliases; retain mapping lifetime and host-fault protection. Keep ordinary contiguous/strided paths inline and use a native Spinlock wait word to avoid shared proxy-counter traffic.
Separate transaction observation policy from cache geometry. The backing store owns the policy and each CU inherits it. Production targets retain the 128-byte default; an internal 64-byte profile exercises narrower boundaries without changing target defaults. Document the RCCL evidence and hardware qualification limits. Preserve default translated-backend batching to avoid extra transport/DMA round trips.
Validation: 358 focused CTests, 21 selected TSan tests, GCC warnings-as-errors checks on the three changed test translation units, and changed-file pre-commit pass. Three ASan/UBSan property probes cover 380,000 seeded cases. All 66 communication cases complete, with zero corruption across 4.56 million positive observations and the expected deliberately split failures.
The broader standard CTest sweep retains 32 documented baseline/configuration failures: tensor-DMA initialization, DBT/code-object layout checks, and gfx1250-only HIP fixtures run under fixed cdna2/cdna4 configurations. This is not a clean full-suite claim.
A matched comparison against the previously pushed implementation improves eight of ten microbenchmark cases; strided loads/stores are 1.94%/2.80% slower. Measurements use CPU affinity and a shared workload lock, without exclusive CPU reservation or fixed clocks. This is not a claim of universal performance neutrality. One full local Peanut review and focused verification of its substantive fixes precede the push.