[LLVM] New NoDivergenceSource function attribute #111832
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A call to a function that has this attribute is not a source of divergence, as used by UniformityAnalysis. That allows a front-end to use known-name calls as an instruction extension mechanism (e.g. https://github.com/GPUOpen-Drivers/llvm-dialects ) without such a call being a source of divergence.
|
@llvm/pr-subscribers-llvm-analysis @llvm/pr-subscribers-llvm-ir Author: Tim Renouf (trenouf) ChangesA call to a function that has this attribute is not a source of divergence, as used by UniformityAnalysis. That allows a front-end to use known-name calls as an instruction extension mechanism (e.g. https://github.com/GPUOpen-Drivers/llvm-dialects ) without such a call being a source of divergence. Full diff: https://github.com/llvm/llvm-project/pull/111832.diff 9 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 0c7279de06cd68..4c94ffbc1d7376 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -2082,6 +2082,16 @@ example:
function call, use of ``longjmp``, or other means. It is a compiler hint that
is used at module level to improve dataflow analysis, dropped during linking,
and has no effect on functions defined in the current module.
+``nodivergencesource``
+ A call to this function is not a source of divergence. In uniformity
+ analysis, a *source of divergence* is an instruction that generates
+ divergence even if its inputs are uniform. A call with no further information
+ would normally be considered a source of divergence; setting this attribute
+ on a function means that a call to it is not a source of divergence.
+
+ This is useful where known-name calls are being used as an extension
+ mechanism for instructions, as for example in `llvm-dialects
+ <https://github.com/GPUOpen-Drivers/llvm-dialects>`.
``noduplicate``
This attribute indicates that calls to the function cannot be
duplicated. A call to a ``noduplicate`` function may be moved
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index cbd92fd52fc75a..a6959d9b62742a 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -764,6 +764,7 @@ enum AttributeKindCodes {
ATTR_KIND_SANITIZE_REALTIME_UNSAFE = 97,
ATTR_KIND_CORO_ELIDE_SAFE = 98,
ATTR_KIND_NO_EXT = 99,
+ ATTR_KIND_NO_DIVERGENCE_SOURCE = 100,
};
enum ComdatSelectionKindCodes {
diff --git a/llvm/include/llvm/IR/Attributes.td b/llvm/include/llvm/IR/Attributes.td
index d05a6ca92aaba0..b6d36a5f7ae4fb 100644
--- a/llvm/include/llvm/IR/Attributes.td
+++ b/llvm/include/llvm/IR/Attributes.td
@@ -183,6 +183,9 @@ def NoCallback : EnumAttr<"nocallback", IntersectAnd, [FnAttr]>;
/// Function creates no aliases of pointer.
def NoCapture : EnumAttr<"nocapture", IntersectAnd, [ParamAttr]>;
+/// Function is not a source of divergence.
+def NoDivergenceSource : EnumAttr<"nodivergencesource", IntersectAnd, [FnAttr]>;
+
/// Call cannot be duplicated.
def NoDuplicate : EnumAttr<"noduplicate", IntersectPreserve, [FnAttr]>;
diff --git a/llvm/lib/Analysis/UniformityAnalysis.cpp b/llvm/lib/Analysis/UniformityAnalysis.cpp
index 2d617db431c588..b0be6379a15125 100644
--- a/llvm/lib/Analysis/UniformityAnalysis.cpp
+++ b/llvm/lib/Analysis/UniformityAnalysis.cpp
@@ -32,7 +32,14 @@ bool llvm::GenericUniformityAnalysisImpl<SSAContext>::markDefsDivergent(
template <> void llvm::GenericUniformityAnalysisImpl<SSAContext>::initialize() {
for (auto &I : instructions(F)) {
- if (TTI->isSourceOfDivergence(&I))
+ bool NoDivergenceSource = false;
+ if (auto Call = dyn_cast<CallInst>(&I)) {
+ if (Function *Callee = Call->getCalledFunction()) {
+ if (Callee->hasFnAttribute(Attribute::NoDivergenceSource))
+ NoDivergenceSource = true;
+ }
+ }
+ if (!NoDivergenceSource && TTI->isSourceOfDivergence(&I))
markDivergent(I);
else if (TTI->isAlwaysUniform(&I))
addUniformOverride(I);
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 6f997510b03609..b8dc5ecbf7abda 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2048,6 +2048,8 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoCallback;
case bitc::ATTR_KIND_NO_CAPTURE:
return Attribute::NoCapture;
+ case bitc::ATTR_KIND_NO_DIVERGENCE_SOURCE:
+ return Attribute::NoDivergenceSource;
case bitc::ATTR_KIND_NO_DUPLICATE:
return Attribute::NoDuplicate;
case bitc::ATTR_KIND_NOFREE:
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index d9086bfebbd2a9..a44b6a8614357d 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -761,6 +761,8 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_CALLBACK;
case Attribute::NoCapture:
return bitc::ATTR_KIND_NO_CAPTURE;
+ case Attribute::NoDivergenceSource:
+ return bitc::ATTR_KIND_NO_DIVERGENCE_SOURCE;
case Attribute::NoDuplicate:
return bitc::ATTR_KIND_NO_DUPLICATE;
case Attribute::NoFree:
diff --git a/llvm/lib/Transforms/Utils/CodeExtractor.cpp b/llvm/lib/Transforms/Utils/CodeExtractor.cpp
index f58448dd9562d5..5b42099398f8ff 100644
--- a/llvm/lib/Transforms/Utils/CodeExtractor.cpp
+++ b/llvm/lib/Transforms/Utils/CodeExtractor.cpp
@@ -917,6 +917,7 @@ Function *CodeExtractor::constructFunction(const ValueSet &inputs,
case Attribute::NoFPClass:
case Attribute::CoroDestroyOnlyWhenComplete:
case Attribute::CoroElideSafe:
+ case Attribute::NoDivergenceSource:
continue;
// Those attributes should be safe to propagate to the extracted function.
case Attribute::AlwaysInline:
diff --git a/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll b/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll
new file mode 100644
index 00000000000000..9c893ac3ba76a0
--- /dev/null
+++ b/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll
@@ -0,0 +1,16 @@
+; RUN: opt -mtriple amdgcn-- -passes='print<uniformity>' -disable-output %s 2>&1 | FileCheck %s
+
+; CHECK: DIVERGENT: %divergentval
+; CHECK-NOT: DIVERGENT: %uniformval
+; CHECK: %uniformval
+define void @test() {
+ %divergentval = call i32 @normalfunc()
+ %uniformval = call i32 @nodivergencesourcefunc()
+ ret void
+}
+
+declare i32 @normalfunc() #0
+declare i32 @nodivergencesourcefunc() #1
+
+attributes #0 = { nounwind }
+attributes #1 = { nounwind nodivergencesource }
diff --git a/llvm/test/Bitcode/attributes.ll b/llvm/test/Bitcode/attributes.ll
index a66eda19ff5735..737f49aa86a7ba 100644
--- a/llvm/test/Bitcode/attributes.ll
+++ b/llvm/test/Bitcode/attributes.ll
@@ -537,6 +537,11 @@ define void @f91(ptr dead_on_unwind %p) {
ret void
}
+; CHECK: define void @f94() [[NODIVERGENCESOURCE:#[0-9]+]]
+define void @f94() nodivergencesource {
+ ret void;
+}
+
; CHECK: define range(i32 -1, 42) i32 @range_attribute(<4 x i32> range(i32 -1, 42) %a)
define range(i32 -1, 42) i32 @range_attribute(<4 x i32> range(i32 -1, 42) %a) {
ret i32 0
@@ -615,4 +620,5 @@ define void @initializes(ptr initializes((-4, 0), (4, 8)) %a) {
; CHECK: attributes [[FNRETTHUNKEXTERN]] = { fn_ret_thunk_extern }
; CHECK: attributes [[SKIPPROFILE]] = { skipprofile }
; CHECK: attributes [[OPTDEBUG]] = { optdebug }
+; CHECK: attributes [[NODIVERGENCESOURCE]] = { nodivergencesource }
; CHECK: attributes #[[NOBUILTIN]] = { nobuiltin }
|
|
@llvm/pr-subscribers-backend-amdgpu Author: Tim Renouf (trenouf) ChangesA call to a function that has this attribute is not a source of divergence, as used by UniformityAnalysis. That allows a front-end to use known-name calls as an instruction extension mechanism (e.g. https://github.com/GPUOpen-Drivers/llvm-dialects ) without such a call being a source of divergence. Full diff: https://github.com/llvm/llvm-project/pull/111832.diff 9 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 0c7279de06cd68..4c94ffbc1d7376 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -2082,6 +2082,16 @@ example:
function call, use of ``longjmp``, or other means. It is a compiler hint that
is used at module level to improve dataflow analysis, dropped during linking,
and has no effect on functions defined in the current module.
+``nodivergencesource``
+ A call to this function is not a source of divergence. In uniformity
+ analysis, a *source of divergence* is an instruction that generates
+ divergence even if its inputs are uniform. A call with no further information
+ would normally be considered a source of divergence; setting this attribute
+ on a function means that a call to it is not a source of divergence.
+
+ This is useful where known-name calls are being used as an extension
+ mechanism for instructions, as for example in `llvm-dialects
+ <https://github.com/GPUOpen-Drivers/llvm-dialects>`.
``noduplicate``
This attribute indicates that calls to the function cannot be
duplicated. A call to a ``noduplicate`` function may be moved
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index cbd92fd52fc75a..a6959d9b62742a 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -764,6 +764,7 @@ enum AttributeKindCodes {
ATTR_KIND_SANITIZE_REALTIME_UNSAFE = 97,
ATTR_KIND_CORO_ELIDE_SAFE = 98,
ATTR_KIND_NO_EXT = 99,
+ ATTR_KIND_NO_DIVERGENCE_SOURCE = 100,
};
enum ComdatSelectionKindCodes {
diff --git a/llvm/include/llvm/IR/Attributes.td b/llvm/include/llvm/IR/Attributes.td
index d05a6ca92aaba0..b6d36a5f7ae4fb 100644
--- a/llvm/include/llvm/IR/Attributes.td
+++ b/llvm/include/llvm/IR/Attributes.td
@@ -183,6 +183,9 @@ def NoCallback : EnumAttr<"nocallback", IntersectAnd, [FnAttr]>;
/// Function creates no aliases of pointer.
def NoCapture : EnumAttr<"nocapture", IntersectAnd, [ParamAttr]>;
+/// Function is not a source of divergence.
+def NoDivergenceSource : EnumAttr<"nodivergencesource", IntersectAnd, [FnAttr]>;
+
/// Call cannot be duplicated.
def NoDuplicate : EnumAttr<"noduplicate", IntersectPreserve, [FnAttr]>;
diff --git a/llvm/lib/Analysis/UniformityAnalysis.cpp b/llvm/lib/Analysis/UniformityAnalysis.cpp
index 2d617db431c588..b0be6379a15125 100644
--- a/llvm/lib/Analysis/UniformityAnalysis.cpp
+++ b/llvm/lib/Analysis/UniformityAnalysis.cpp
@@ -32,7 +32,14 @@ bool llvm::GenericUniformityAnalysisImpl<SSAContext>::markDefsDivergent(
template <> void llvm::GenericUniformityAnalysisImpl<SSAContext>::initialize() {
for (auto &I : instructions(F)) {
- if (TTI->isSourceOfDivergence(&I))
+ bool NoDivergenceSource = false;
+ if (auto Call = dyn_cast<CallInst>(&I)) {
+ if (Function *Callee = Call->getCalledFunction()) {
+ if (Callee->hasFnAttribute(Attribute::NoDivergenceSource))
+ NoDivergenceSource = true;
+ }
+ }
+ if (!NoDivergenceSource && TTI->isSourceOfDivergence(&I))
markDivergent(I);
else if (TTI->isAlwaysUniform(&I))
addUniformOverride(I);
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 6f997510b03609..b8dc5ecbf7abda 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2048,6 +2048,8 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoCallback;
case bitc::ATTR_KIND_NO_CAPTURE:
return Attribute::NoCapture;
+ case bitc::ATTR_KIND_NO_DIVERGENCE_SOURCE:
+ return Attribute::NoDivergenceSource;
case bitc::ATTR_KIND_NO_DUPLICATE:
return Attribute::NoDuplicate;
case bitc::ATTR_KIND_NOFREE:
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index d9086bfebbd2a9..a44b6a8614357d 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -761,6 +761,8 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_CALLBACK;
case Attribute::NoCapture:
return bitc::ATTR_KIND_NO_CAPTURE;
+ case Attribute::NoDivergenceSource:
+ return bitc::ATTR_KIND_NO_DIVERGENCE_SOURCE;
case Attribute::NoDuplicate:
return bitc::ATTR_KIND_NO_DUPLICATE;
case Attribute::NoFree:
diff --git a/llvm/lib/Transforms/Utils/CodeExtractor.cpp b/llvm/lib/Transforms/Utils/CodeExtractor.cpp
index f58448dd9562d5..5b42099398f8ff 100644
--- a/llvm/lib/Transforms/Utils/CodeExtractor.cpp
+++ b/llvm/lib/Transforms/Utils/CodeExtractor.cpp
@@ -917,6 +917,7 @@ Function *CodeExtractor::constructFunction(const ValueSet &inputs,
case Attribute::NoFPClass:
case Attribute::CoroDestroyOnlyWhenComplete:
case Attribute::CoroElideSafe:
+ case Attribute::NoDivergenceSource:
continue;
// Those attributes should be safe to propagate to the extracted function.
case Attribute::AlwaysInline:
diff --git a/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll b/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll
new file mode 100644
index 00000000000000..9c893ac3ba76a0
--- /dev/null
+++ b/llvm/test/Analysis/UniformityAnalysis/AMDGPU/nodivergencesource.ll
@@ -0,0 +1,16 @@
+; RUN: opt -mtriple amdgcn-- -passes='print<uniformity>' -disable-output %s 2>&1 | FileCheck %s
+
+; CHECK: DIVERGENT: %divergentval
+; CHECK-NOT: DIVERGENT: %uniformval
+; CHECK: %uniformval
+define void @test() {
+ %divergentval = call i32 @normalfunc()
+ %uniformval = call i32 @nodivergencesourcefunc()
+ ret void
+}
+
+declare i32 @normalfunc() #0
+declare i32 @nodivergencesourcefunc() #1
+
+attributes #0 = { nounwind }
+attributes #1 = { nounwind nodivergencesource }
diff --git a/llvm/test/Bitcode/attributes.ll b/llvm/test/Bitcode/attributes.ll
index a66eda19ff5735..737f49aa86a7ba 100644
--- a/llvm/test/Bitcode/attributes.ll
+++ b/llvm/test/Bitcode/attributes.ll
@@ -537,6 +537,11 @@ define void @f91(ptr dead_on_unwind %p) {
ret void
}
+; CHECK: define void @f94() [[NODIVERGENCESOURCE:#[0-9]+]]
+define void @f94() nodivergencesource {
+ ret void;
+}
+
; CHECK: define range(i32 -1, 42) i32 @range_attribute(<4 x i32> range(i32 -1, 42) %a)
define range(i32 -1, 42) i32 @range_attribute(<4 x i32> range(i32 -1, 42) %a) {
ret i32 0
@@ -615,4 +620,5 @@ define void @initializes(ptr initializes((-4, 0), (4, 8)) %a) {
; CHECK: attributes [[FNRETTHUNKEXTERN]] = { fn_ret_thunk_extern }
; CHECK: attributes [[SKIPPROFILE]] = { skipprofile }
; CHECK: attributes [[OPTDEBUG]] = { optdebug }
+; CHECK: attributes [[NODIVERGENCESOURCE]] = { nodivergencesource }
; CHECK: attributes #[[NOBUILTIN]] = { nobuiltin }
|
jayfoad
reviewed
Oct 10, 2024
arsenm
reviewed
Oct 10, 2024
That interpretation of |
Oh, I see. It's a replacement for the SourceOfDivergence searchable table, rather than an annotation of the return value |
|
(I will squash down to one commit when I merge.) |
jayfoad
reviewed
Oct 11, 2024
nikic
reviewed
Oct 11, 2024
jayfoad
approved these changes
Oct 11, 2024
arsenm
reviewed
Oct 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A call to a function that has this attribute is not a source of divergence, as used by UniformityAnalysis. That allows a front-end to use known-name calls as an instruction extension mechanism (e.g. https://github.com/GPUOpen-Drivers/llvm-dialects ) without such a call being a source of divergence.