Skip to content

test(rocke): pin the llvm23 datalayout m:e mangling spec and complete its support - #10260

Merged
shumway merged 1 commit into
developfrom
users/shumway/rocke-llvm23-datalayout-mangling
Aug 3, 2026
Merged

shumway merged 1 commit into
developfrom
users/shumway/rocke-llvm23-datalayout-mangling

Conversation

@shumway

@shumway shumway commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

ISSUE ID : AICK-1517

Summary

llvm23 support (#9583) shipped _DATALAYOUT_LLVM23 as an alias of the llvm22 layout, with a comment to re-derive it on a real LLVM-23 host and split it out if drift was found. On a real LLVM-23 host (ROCm 7.13, AMD clang 23.0.0git) the AMDGPU datalayout includes the ELF symbol-mangling spec m:e (e- → e-m:e-) that LLVM 20/22 omit, so the aliased stub disagrees with what the toolchain emits and the drift guard fails on every wired arch. The stub was harmless at runtime only because the comgr SOURCE path overrides the module datalayout; bitcode input or a stricter verifier would reject it.

Changes

  • Split _DATALAYOUT_LLVM23 out from the llvm22 alias with the m:e spec, in lockstep across all three engine mirror sites (_DATALAYOUT_LLVM23, ROCKE_LL_DATALAYOUT_LLVM23, ROCKE_DATALAYOUT_LLVM23) plus the header, and regenerate the llvm23 golden IR hashes (only llvm23's 79 hashes change; llvm20/llvm22 untouched).
  • Add an always-run host-only test asserting llvm23 == llvm22 + m:e, so an accidental re-alias fails on the bare CI gate rather than only on a ROCm-7.13 box with hipcc. This is the gap that let the stub land green: byte-identity is self-consistent with the stub, and the toolchain drift guard skips without a real LLVM-23 toolchain.
  • Add an m:e per-flavor pin to the C++ flavor-datalayout SSOT test, mirroring the existing p8 pin, so the new drift axis has the same structural guard.
  • Flip the drift-guard's pre-drift pin to assert the proven llvm22 + m:e relationship (derived from the llvm22 constant, not a second copy).
  • Correct datalayout comments: p8 is the buffer-resource address space (128-bit buffer descriptor), not the buffer fat pointer (that is p7); the LlvmDatalayoutKind docstring now reflects that two fields drift (p8 shape and m:e), with the kind grouping flavors by p8 shape / intrinsic-declare compatibility rather than full datalayout identity.

Local validation (real gfx1151, ROCm 7.13 / clang 23)

  • Full C++ build: clean, 0 errors/warnings, all targets incl. the rocke_engine pybind extension.
  • ctest: 5/5 passed, including the new rocke_flavor_datalayout_ssot m:e pin.
  • Byte-identity gate: GREEN — 68 families, 0 divergences (C++ ↔ Python .ll byte-identical, incl. the llvm23 m:e path).
  • pytest (single-engine and ROCKE_BACKEND=both differential): 546 passed, 32 skipped (all torch-gated, expected in the torch-free venv).
  • The llvm23 drift guard (test_datalayout_matches_hipcc_emitted_ir) now passes against the installed toolchain on every wired arch.

🤖 Generated with Claude Code

llvm23 support (#9583) shipped _DATALAYOUT_LLVM23 as an alias of the
llvm22 layout, with a comment to re-derive it on a real LLVM-23 host and
split it out if drift was found. On a real LLVM-23 host (ROCm 7.13, AMD
clang 23.0.0git) the AMDGPU datalayout includes the ELF symbol-mangling
spec `m:e` (e- -> e-m:e-) that LLVM 20/22 omit, so the aliased stub
disagrees with what the toolchain emits and the drift guard fails on
every wired arch. The stub was harmless at runtime only because the comgr
SOURCE path overrides the module datalayout; bitcode input or a stricter
verifier would reject it.

Split _DATALAYOUT_LLVM23 out from the llvm22 alias with the m:e spec, in
lockstep across all three engine mirror sites (python _DATALAYOUT_LLVM23,
cpp ROCKE_LL_DATALAYOUT_LLVM23, cpp ROCKE_DATALAYOUT_LLVM23) plus the
header, and regenerate the llvm23 golden IR hashes (only llvm23's 79
hashes change; llvm20/llvm22 untouched).

Harden the guards so this can't land green again:
- Add an always-run host-only test asserting llvm23 == llvm22 + m:e, so
  an accidental re-alias fails on the bare CI gate rather than only on a
  ROCm-7.13 box with hipcc. This is the gap that let the stub pass:
  byte-identity is self-consistent with the stub, and the toolchain drift
  guard skips without a real LLVM-23 toolchain.
- Add an m:e per-flavor pin to the C++ flavor-datalayout SSOT test,
  mirroring the existing p8 pin, so the new drift axis has the same
  structural guard.

Also correct the datalayout comments: p8 is the buffer-resource address
space (128-bit buffer descriptor), not the buffer fat pointer (that is
p7), and the LlvmDatalayoutKind docstring now reflects that two fields
drift (p8 shape and m:e), with the kind grouping flavors by p8 shape /
intrinsic-declare compatibility rather than full datalayout identity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@therock-pr-bot

therock-pr-bot Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@tenpercent

tenpercent commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

(gen'd by gpt 5.6 sol)

One important Windows AUTO flavor path may need follow-up now that llvm23 is observably distinct from llvm22 through the added m:e datalayout component.

The native C++ ll_resolve_flavor() discovery currently appears POSIX-specific: it searches /opt/rocm, <root>/lib/libamd_comgr.so, and /-style paths, but does not inspect Windows HIP_PATH/ROCM_PATH locations such as bin/amd_comgr*.dll. If discovery fails, it falls back to llvm22. This affects direct C++ and pybind entrypoints that pass ROCKE_LLVM_FLAVOR_AUTO; on a Windows LLVM 23 installation, those paths could therefore emit the LLVM 22 datalayout without m:e unless users explicitly set ROCKE_LLVM_FLAVOR=llvm23.

The torch-free Python path seems analogous: it can locate _rocm_sdk_core/bin/amd_comgr*.dll, but its version resolution does not appear to use rocm-sdk-core distribution metadata and may likewise fall back to llvm22.

This behavior was effectively hidden while the two flavors produced the same result. Could we teach both AUTO resolvers to discover Windows DLL layouts and determine their version, then add Windows-shaped unit coverage for native and torch-free AUTO resolution? The current Windows coverage exercises explicit LLVM 23 constants/SSOT, but does not appear to exercise these AUTO paths.

@shumway
shumway merged commit 5912f34 into develop Aug 3, 2026
39 of 42 checks passed
@shumway
shumway deleted the users/shumway/rocke-llvm23-datalayout-mangling branch August 3, 2026 22:01
yraparti added a commit that referenced this pull request Aug 5, 2026
Pinning the gfx1250 WMMA forward IR added three attention_wmma cases to
the parity harness and the golden. That pinning belongs with the kernel's
own change rather than the dispatcher refactor this PR is scoped to, so
both files return to their develop state: 82 cases per flavor back to 79.

Removing them also clears the drift CI reported. Develop's #10260 split
_DATALAYOUT_LLVM23 out of its llvm22 alias to add the ELF m:e mangling
spec and re-blessed all 79 llvm23 digests. These three cases did not
exist in develop's copy of the golden, so that re-bless could not reach
them and they kept their pre-m:e hashes across the rebase.

The manifest-runner probe fix from the same commit is unrelated to
gfx1250 and stays.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants