Repository navigation
[rocjitsu] Classify full Wave32 runs with vector stride checks - #14250
Merged
Merged
Conversation
SIMD helpers separately combine architecture, compiler, ISA and standard library checks to select backends and mask fallbacks. Make util/host_target.h own these decisions, including block conversions, qualified AVX intrinsic backends, SIMD TS availability and the native 64-bit mask fallback. Include <version> before identifying the standard library so detection is independent of include order. Preserve existing command-line mask overrides under the RJ_ prefix and target-selection behavior. Use RJ_-prefixed capability checks at call sites and remove local detection macros. Follow the header style with #pragma once. Runtime CPU dispatch and scalar execution controls stay separate. Compile the lock's header-first unit test with undefined feature checks as errors, so missing capability dependencies cannot silently remove architecture spin hints.
Transaction-aware coalescing introduced in #14170, and optimized in #14215, still scans ordinary contiguous lane addresses one at a time. Check full Wave32 operands with a private VM stride predicate before the scalar scan. Select AVX-512 or AVX2 from the compile target without per-wave CPU queries or dispatch. Reject wrapping sequences and keep vector loads within the active addresses. Other masks, Wave64 and scalar-only builds retain the existing scan. Keep the standard address sort in one non-inline function shared by callback instantiations. Preserve transaction policy, overlap fallback and callback order without adding a custom sorter or an arith dependency. Five paired sweeps of the original stride binaries across seven small kernels show 0.43% fewer Wave32 instructions per warm launch on default V3 and 0.46% fewer on V4. Matched V3 builds use 0.43% fewer cycles; copy and gather use 1.04% and 0.63% fewer cycles. V3 atomic- add uses 0.12% more cycles. The measured original V4 binaries show 2.96% fewer Wave32 cycles, but that observation is sensitive to binary layout and does not isolate an algorithmic gain of that size. Wave64 cycle changes are likewise layout-sensitive and are not treated as benefits of this optimization. Measurements use GCC 16 Release/O3, matched baseline/candidate CPU targets, fixed 2.5 GHz reserved CPUs, both workload locks and wait checking enabled. Counters cover checked warm launches and synchronization, excluding startup and JIT. These are host CPU counts, not whole-application speedups. The original stride performance receipts predate the shared host- target prerequisite. They retain all 280 measured processes and their numerical, runtime, affinity and nonmultiplexed-counter proofs. No new performance measurements are attributed to the capability cleanup. Validate the shared-capability integration with V3/V4 builds, 255 CTests, 15 FP-mode helpers and changed-file hooks. Check 24 native compiler/target contexts, 212 include-order/override combinations, 25 synthetic probes, 20 property executables including ASan/UBSan, and 5,875,080 stride cases. All 144 header-only override checks pass; four full-SIMD Clang forced-zero contexts remain unsupported by the existing toolchain. Compare 18 stride assembly pairs byte-for-byte against the original. Preserve the lock's pause instruction and diagnose missing header dependencies in its header-first unit test. Run correctness checks under both workload locks outside the expired reservation, as requested. Two full local Peanut review rounds and focused follow-ups cover the prerequisite and its consumers.
bjacob
approved these changes
Oct 26, 2026
bjacob
left a comment
bjacob
left a comment
Contributor
There was a problem hiding this comment.
Reviewed the SIMD arithmetic, unsigned wrap guard, load bounds, and full-Wave32 integration. Boundary and randomized checks passed across scalar, AVX2, AVX-512F-only, and Zen 4 builds with GCC and Clang, including sanitizer checks and comparisons with the previous coalescer. Replayed the relevant gist inputs and confirmed matching runs and callback behavior.
✅ All Policy Checks Passed
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
Contributor
TheRock Submodule Bump ActivityNewest first
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transaction-aware coalescing introduced in #14170, and optimized in
#14215, still scans ordinary contiguous lane addresses one at a time.
Check full Wave32 operands with a private VM stride predicate before
the scalar scan. Select AVX-512 or AVX2 from the compile target
without per-wave CPU queries or dispatch. Reject wrapping sequences
and keep vector loads within the active addresses. Other masks, Wave64
and scalar-only builds retain the existing scan.
Keep the standard address sort in one non-inline function shared by
callback instantiations. Preserve transaction policy, overlap fallback
and callback order without adding a custom sorter or an arith
dependency.
Five paired sweeps of the original stride binaries across seven small
kernels show 0.43% fewer Wave32 instructions per warm launch on
default V3 and 0.46% fewer on V4. Matched V3 builds use 0.43% fewer
cycles; copy and gather use 1.04% and 0.63% fewer cycles. V3 atomic-
add uses 0.12% more cycles.
The measured original V4 binaries show 2.96% fewer Wave32 cycles, but
that observation is sensitive to binary layout and does not isolate an
algorithmic gain of that size. Wave64 cycle changes are likewise
layout-sensitive and are not treated as benefits of this optimization.
Measurements use GCC 16 Release/O3, matched baseline/candidate CPU
targets, fixed 2.5 GHz reserved CPUs, both workload locks and wait
checking enabled. Counters cover checked warm launches and
synchronization, excluding startup and JIT. These are host CPU counts,
not whole-application speedups.
The original stride performance receipts predate the shared host-
target prerequisite. They retain all 280 measured processes and their
numerical, runtime, affinity and nonmultiplexed-counter proofs. No new
performance measurements are attributed to the capability cleanup.
Validate the shared-capability integration with V3/V4 builds, 255 CTests,
15 FP-mode helpers and changed-file hooks. Check 24 native compiler/target
contexts, 212 include-order/override combinations, 25 synthetic probes,
20 property executables including ASan/UBSan, and 5,875,080 stride cases.
All 144 header-only override checks pass; four full-SIMD Clang forced-zero
contexts remain unsupported by the existing toolchain. Compare 18 stride
assembly pairs byte-for-byte against the original. Preserve the lock's
pause instruction and diagnose missing header dependencies in its
header-first unit test.
Run correctness checks under both workload locks outside the expired
reservation, as requested. Two full local Peanut review rounds and
focused follow-ups cover the prerequisite and its consumers.