[AArch64] Make the list of LSE supported operations explicit #171126
Merged
+1,609
−473
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Similar to llvm#167760 this makes the list of LSE atomics explicit in case new operations are added in the future. UIncWrap, UDecWrap, USubCond and USubSat are excluded.
Member
|
@llvm/pr-subscribers-backend-aarch64 Author: David Green (davemgreen) ChangesSimilar to #167760 this makes the list of LSE atomics explicit in case new operations are added in the future. UIncWrap, UDecWrap, USubCond and USubSat are excluded. Patch is 94.98 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/171126.diff 2 Files Affected:
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index 7199319ccdd9f..85997de77d902 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -29739,12 +29739,26 @@ AArch64TargetLowering::shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const {
AI->getOperation() == AtomicRMWInst::FMinimum))
return AtomicExpansionKind::None;
- // Nand is not supported in LSE.
// Leave 128 bits to LLSC or CmpXChg.
- if (AI->getOperation() != AtomicRMWInst::Nand && Size < 128 &&
- !AI->isFloatingPointOperation()) {
- if (Subtarget->hasLSE())
- return AtomicExpansionKind::None;
+ if (Size < 128 && !AI->isFloatingPointOperation()) {
+ if (Subtarget->hasLSE()) {
+ // Nand is not supported in LSE.
+ switch (AI->getOperation()) {
+ case AtomicRMWInst::Xchg:
+ case AtomicRMWInst::Add:
+ case AtomicRMWInst::Sub:
+ case AtomicRMWInst::And:
+ case AtomicRMWInst::Or:
+ case AtomicRMWInst::Xor:
+ case AtomicRMWInst::Max:
+ case AtomicRMWInst::Min:
+ case AtomicRMWInst::UMax:
+ case AtomicRMWInst::UMin:
+ return AtomicExpansionKind::None;
+ default:
+ break;
+ }
+ }
if (Subtarget->outlineAtomics()) {
// [U]Min/[U]Max RWM atomics are used in __sync_fetch_ libcalls so far.
// Don't outline them unless
diff --git a/llvm/test/CodeGen/AArch64/atomic-ops.ll b/llvm/test/CodeGen/AArch64/atomic-ops.ll
index 1c2edd39e268d..adfb4ab10108a 100644
--- a/llvm/test/CodeGen/AArch64/atomic-ops.ll
+++ b/llvm/test/CodeGen/AArch64/atomic-ops.ll
@@ -1,6 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -aarch64-enable-sink-fold=true < %s | FileCheck %s --check-prefixes=CHECK,INLINE_ATOMICS
-; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+outline-atomics -aarch64-enable-sink-fold=true < %s | FileCheck %s --check-prefixes=CHECK,OUTLINE_ATOMICS
+; RUN: llc -mtriple=aarch64-none-linux-gnu -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,INLINE_ATOMICS
+; RUN: llc -mtriple=aarch64-none-linux-gnu -verify-machineinstrs -mattr=+outline-atomics < %s | FileCheck %s --check-prefixes=CHECK,OUTLINE_ATOMICS
+; RUN: llc -mtriple=aarch64-none-linux-gnu -verify-machineinstrs -mattr=+lse < %s | FileCheck %s --check-prefixes=CHECK,LSE
@var8 = dso_local global i8 0
@var16 = dso_local global i16 0
@@ -30,6 +31,13 @@ define dso_local i8 @test_atomic_load_add_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_add_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var8
+; LSE-NEXT: add x8, x8, :lo12:var8
+; LSE-NEXT: ldaddalb w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset seq_cst
ret i8 %old
}
@@ -57,6 +65,13 @@ define dso_local i16 @test_atomic_load_add_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd2_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_add_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var16
+; LSE-NEXT: add x8, x8, :lo12:var16
+; LSE-NEXT: ldaddah w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset acquire
ret i16 %old
}
@@ -84,6 +99,13 @@ define dso_local i32 @test_atomic_load_add_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd4_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_add_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var32
+; LSE-NEXT: add x8, x8, :lo12:var32
+; LSE-NEXT: ldaddl w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset release
ret i32 %old
}
@@ -111,6 +133,13 @@ define dso_local i64 @test_atomic_load_add_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd8_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_add_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var64
+; LSE-NEXT: add x8, x8, :lo12:var64
+; LSE-NEXT: ldadd x0, x0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset monotonic
ret i64 %old
}
@@ -139,6 +168,14 @@ define dso_local i8 @test_atomic_load_sub_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd1_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_sub_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: neg w8, w0
+; LSE-NEXT: adrp x9, var8
+; LSE-NEXT: add x9, x9, :lo12:var8
+; LSE-NEXT: ldaddb w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset monotonic
ret i8 %old
}
@@ -167,6 +204,14 @@ define dso_local i16 @test_atomic_load_sub_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd2_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_sub_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: neg w8, w0
+; LSE-NEXT: adrp x9, var16
+; LSE-NEXT: add x9, x9, :lo12:var16
+; LSE-NEXT: ldaddlh w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset release
ret i16 %old
}
@@ -195,6 +240,14 @@ define dso_local i32 @test_atomic_load_sub_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd4_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_sub_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: neg w8, w0
+; LSE-NEXT: adrp x9, var32
+; LSE-NEXT: add x9, x9, :lo12:var32
+; LSE-NEXT: ldadda w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset acquire
ret i32 %old
}
@@ -223,6 +276,14 @@ define dso_local i64 @test_atomic_load_sub_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_sub_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: neg x8, x0
+; LSE-NEXT: adrp x9, var64
+; LSE-NEXT: add x9, x9, :lo12:var64
+; LSE-NEXT: ldaddal x8, x0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset seq_cst
ret i64 %old
}
@@ -251,6 +312,14 @@ define dso_local i8 @test_atomic_load_and_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldclr1_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_and_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: mvn w8, w0
+; LSE-NEXT: adrp x9, var8
+; LSE-NEXT: add x9, x9, :lo12:var8
+; LSE-NEXT: ldclrlb w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset release
ret i8 %old
}
@@ -279,6 +348,14 @@ define dso_local i16 @test_atomic_load_and_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldclr2_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_and_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: mvn w8, w0
+; LSE-NEXT: adrp x9, var16
+; LSE-NEXT: add x9, x9, :lo12:var16
+; LSE-NEXT: ldclrh w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset monotonic
ret i16 %old
}
@@ -307,6 +384,14 @@ define dso_local i32 @test_atomic_load_and_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_and_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: mvn w8, w0
+; LSE-NEXT: adrp x9, var32
+; LSE-NEXT: add x9, x9, :lo12:var32
+; LSE-NEXT: ldclral w8, w0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset seq_cst
ret i32 %old
}
@@ -335,6 +420,14 @@ define dso_local i64 @test_atomic_load_and_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldclr8_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_and_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: mvn x8, x0
+; LSE-NEXT: adrp x9, var64
+; LSE-NEXT: add x9, x9, :lo12:var64
+; LSE-NEXT: ldclra x8, x0, [x9]
+; LSE-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset acquire
ret i64 %old
}
@@ -362,6 +455,13 @@ define dso_local i8 @test_atomic_load_or_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_or_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var8
+; LSE-NEXT: add x8, x8, :lo12:var8
+; LSE-NEXT: ldsetalb w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset seq_cst
ret i8 %old
}
@@ -389,6 +489,13 @@ define dso_local i16 @test_atomic_load_or_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldset2_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_or_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var16
+; LSE-NEXT: add x8, x8, :lo12:var16
+; LSE-NEXT: ldseth w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset monotonic
ret i16 %old
}
@@ -416,6 +523,13 @@ define dso_local i32 @test_atomic_load_or_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldset4_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_or_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var32
+; LSE-NEXT: add x8, x8, :lo12:var32
+; LSE-NEXT: ldseta w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset acquire
ret i32 %old
}
@@ -443,6 +557,13 @@ define dso_local i64 @test_atomic_load_or_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldset8_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_or_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var64
+; LSE-NEXT: add x8, x8, :lo12:var64
+; LSE-NEXT: ldsetl x0, x0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset release
ret i64 %old
}
@@ -470,6 +591,13 @@ define dso_local i8 @test_atomic_load_xor_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldeor1_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xor_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var8
+; LSE-NEXT: add x8, x8, :lo12:var8
+; LSE-NEXT: ldeorab w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset acquire
ret i8 %old
}
@@ -497,6 +625,13 @@ define dso_local i16 @test_atomic_load_xor_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldeor2_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xor_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var16
+; LSE-NEXT: add x8, x8, :lo12:var16
+; LSE-NEXT: ldeorlh w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset release
ret i16 %old
}
@@ -524,6 +659,13 @@ define dso_local i32 @test_atomic_load_xor_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xor_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var32
+; LSE-NEXT: add x8, x8, :lo12:var32
+; LSE-NEXT: ldeoral w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset seq_cst
ret i32 %old
}
@@ -551,6 +693,13 @@ define dso_local i64 @test_atomic_load_xor_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_ldeor8_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xor_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var64
+; LSE-NEXT: add x8, x8, :lo12:var64
+; LSE-NEXT: ldeor x0, x0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset monotonic
ret i64 %old
}
@@ -577,6 +726,13 @@ define dso_local i8 @test_atomic_load_xchg_i8(i8 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_swp1_relax
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xchg_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var8
+; LSE-NEXT: add x8, x8, :lo12:var8
+; LSE-NEXT: swpb w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset monotonic
ret i8 %old
}
@@ -603,6 +759,13 @@ define dso_local i16 @test_atomic_load_xchg_i16(i16 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_swp2_acq_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xchg_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var16
+; LSE-NEXT: add x8, x8, :lo12:var16
+; LSE-NEXT: swpalh w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset seq_cst
ret i16 %old
}
@@ -630,6 +793,13 @@ define dso_local i32 @test_atomic_load_xchg_i32(i32 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_swp4_rel
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xchg_i32:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var32
+; LSE-NEXT: add x8, x8, :lo12:var32
+; LSE-NEXT: swpl w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset release
ret i32 %old
}
@@ -656,633 +826,1556 @@ define dso_local i64 @test_atomic_load_xchg_i64(i64 %offset) nounwind {
; OUTLINE_ATOMICS-NEXT: bl __aarch64_swp8_acq
; OUTLINE_ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_xchg_i64:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var64
+; LSE-NEXT: add x8, x8, :lo12:var64
+; LSE-NEXT: swpa x0, x0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset acquire
ret i64 %old
}
define dso_local i8 @test_atomic_load_min_i8(i8 %offset) nounwind {
-; CHECK-LABEL: test_atomic_load_min_i8:
-; CHECK: // %bb.0:
-; CHECK-NEXT: adrp x9, var8
-; CHECK-NEXT: add x9, x9, :lo12:var8
-; CHECK-NEXT: .LBB24_1: // %atomicrmw.start
-; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: ldaxrb w10, [x9]
-; CHECK-NEXT: sxtb w8, w10
-; CHECK-NEXT: cmp w8, w0, sxtb
-; CHECK-NEXT: csel w10, w10, w0, le
-; CHECK-NEXT: stxrb w11, w10, [x9]
-; CHECK-NEXT: cbnz w11, .LBB24_1
-; CHECK-NEXT: // %bb.2: // %atomicrmw.end
-; CHECK-NEXT: mov w0, w8
-; CHECK-NEXT: ret
+; INLINE_ATOMICS-LABEL: test_atomic_load_min_i8:
+; INLINE_ATOMICS: // %bb.0:
+; INLINE_ATOMICS-NEXT: adrp x9, var8
+; INLINE_ATOMICS-NEXT: add x9, x9, :lo12:var8
+; INLINE_ATOMICS-NEXT: .LBB24_1: // %atomicrmw.start
+; INLINE_ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
+; INLINE_ATOMICS-NEXT: ldaxrb w10, [x9]
+; INLINE_ATOMICS-NEXT: sxtb w8, w10
+; INLINE_ATOMICS-NEXT: cmp w8, w0, sxtb
+; INLINE_ATOMICS-NEXT: csel w10, w10, w0, le
+; INLINE_ATOMICS-NEXT: stxrb w11, w10, [x9]
+; INLINE_ATOMICS-NEXT: cbnz w11, .LBB24_1
+; INLINE_ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
+; INLINE_ATOMICS-NEXT: mov w0, w8
+; INLINE_ATOMICS-NEXT: ret
+;
+; OUTLINE_ATOMICS-LABEL: test_atomic_load_min_i8:
+; OUTLINE_ATOMICS: // %bb.0:
+; OUTLINE_ATOMICS-NEXT: adrp x9, var8
+; OUTLINE_ATOMICS-NEXT: add x9, x9, :lo12:var8
+; OUTLINE_ATOMICS-NEXT: .LBB24_1: // %atomicrmw.start
+; OUTLINE_ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
+; OUTLINE_ATOMICS-NEXT: ldaxrb w10, [x9]
+; OUTLINE_ATOMICS-NEXT: sxtb w8, w10
+; OUTLINE_ATOMICS-NEXT: cmp w8, w0, sxtb
+; OUTLINE_ATOMICS-NEXT: csel w10, w10, w0, le
+; OUTLINE_ATOMICS-NEXT: stxrb w11, w10, [x9]
+; OUTLINE_ATOMICS-NEXT: cbnz w11, .LBB24_1
+; OUTLINE_ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
+; OUTLINE_ATOMICS-NEXT: mov w0, w8
+; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_min_i8:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var8
+; LSE-NEXT: add x8, x8, :lo12:var8
+; LSE-NEXT: ldsminab w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset acquire
ret i8 %old
}
define dso_local i16 @test_atomic_load_min_i16(i16 %offset) nounwind {
-; CHECK-LABEL: test_atomic_load_min_i16:
-; CHECK: // %bb.0:
-; CHECK-NEXT: adrp x9, var16
-; CHECK-NEXT: add x9, x9, :lo12:var16
-; CHECK-NEXT: .LBB25_1: // %atomicrmw.start
-; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: ldxrh w10, [x9]
-; CHECK-NEXT: sxth w8, w10
-; CHECK-NEXT: cmp w8, w0, sxth
-; CHECK-NEXT: csel w10, w10, w0, le
-; CHECK-NEXT: stlxrh w11, w10, [x9]
-; CHECK-NEXT: cbnz w11, .LBB25_1
-; CHECK-NEXT: // %bb.2: // %atomicrmw.end
-; CHECK-NEXT: mov w0, w8
-; CHECK-NEXT: ret
+; INLINE_ATOMICS-LABEL: test_atomic_load_min_i16:
+; INLINE_ATOMICS: // %bb.0:
+; INLINE_ATOMICS-NEXT: adrp x9, var16
+; INLINE_ATOMICS-NEXT: add x9, x9, :lo12:var16
+; INLINE_ATOMICS-NEXT: .LBB25_1: // %atomicrmw.start
+; INLINE_ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
+; INLINE_ATOMICS-NEXT: ldxrh w10, [x9]
+; INLINE_ATOMICS-NEXT: sxth w8, w10
+; INLINE_ATOMICS-NEXT: cmp w8, w0, sxth
+; INLINE_ATOMICS-NEXT: csel w10, w10, w0, le
+; INLINE_ATOMICS-NEXT: stlxrh w11, w10, [x9]
+; INLINE_ATOMICS-NEXT: cbnz w11, .LBB25_1
+; INLINE_ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
+; INLINE_ATOMICS-NEXT: mov w0, w8
+; INLINE_ATOMICS-NEXT: ret
+;
+; OUTLINE_ATOMICS-LABEL: test_atomic_load_min_i16:
+; OUTLINE_ATOMICS: // %bb.0:
+; OUTLINE_ATOMICS-NEXT: adrp x9, var16
+; OUTLINE_ATOMICS-NEXT: add x9, x9, :lo12:var16
+; OUTLINE_ATOMICS-NEXT: .LBB25_1: // %atomicrmw.start
+; OUTLINE_ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
+; OUTLINE_ATOMICS-NEXT: ldxrh w10, [x9]
+; OUTLINE_ATOMICS-NEXT: sxth w8, w10
+; OUTLINE_ATOMICS-NEXT: cmp w8, w0, sxth
+; OUTLINE_ATOMICS-NEXT: csel w10, w10, w0, le
+; OUTLINE_ATOMICS-NEXT: stlxrh w11, w10, [x9]
+; OUTLINE_ATOMICS-NEXT: cbnz w11, .LBB25_1
+; OUTLINE_ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
+; OUTLINE_ATOMICS-NEXT: mov w0, w8
+; OUTLINE_ATOMICS-NEXT: ret
+;
+; LSE-LABEL: test_atomic_load_min_i16:
+; LSE: // %bb.0:
+; LSE-NEXT: adrp x8, var16
+; LSE-NEXT: add x8, x8, :lo12:var16
+; LSE-NEXT: ldsminlh w0, w0, [x8]
+; LSE-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset release
ret i16 %old
}
define dso_local i32 @test_atomic_load_min_i32(i32 %offset) nounwind {
-; CHECK-LABEL: test_atomic_load_min_i32:
-; CHECK: // %bb.0:
-; CHECK-NEXT: adrp x9, var32
-; CHECK-NEXT: add x9, x9, :lo12:var32
-; CHECK-NEXT: .LBB26_1: // %atomicrmw.start
-; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: ldxr w8, [x9]
-; CHECK-NEXT: cmp w8, w0
-; CHECK-NEXT: csel w10, w8, w0, le
-; CHECK-NEXT: stxr w11, w10, [x9]
-; CHECK-NEXT: cbnz w11, .LBB26_1
-; CHECK-NEXT: // %bb.2: // %atomicrmw.end
-; CHECK-NEXT: mov w0, w8
-; CHECK-NEXT: ret
+; INLINE_ATOMICS-LABEL: test_atomic...
[truncated]
|
efriedma-quic
approved these changes
Dec 8, 2025
| ; LSE-LABEL: test_atomic_load_uinc_wrap_i8: | ||
| ; LSE: // %bb.0: | ||
| ; LSE-NEXT: adrp x8, var8 | ||
| ; LSE-NEXT: adrp x9, var8 |
Collaborator
There was a problem hiding this comment.
The double adrp looks silly, but I guess that's a different issue.
Collaborator
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/134/builds/31186 Here is the relevant piece of the build log for the reference |
Collaborator
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/66/builds/23414 Here is the relevant piece of the build log for the reference |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Similar to #167760 this makes the list of LSE atomics explicit in case new operations are added in the future. UIncWrap, UDecWrap, USubCond and USubSat are excluded.
Fixes #170450