Conversation
llvmbot
added
backend:ARM
backend:AArch64
backend:RISC-V
vectorizers
llvm:analysis
Member
|
@llvm/pr-subscribers-llvm-analysis @llvm/pr-subscribers-backend-risc-v Author: Luke Lau (lukel97) ChangesStacked on #183729 After #144963 and #183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow. With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value) Patch is 246.67 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/183762.diff 38 Files Affected:
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfo.h b/llvm/include/llvm/Analysis/TargetTransformInfo.h
index 18ae6a005d972..476fa057410f8 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfo.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfo.h
@@ -251,10 +251,6 @@ enum class TailFoldingStyle {
/// active.lane.mask to calculate the mask for the next iteration. If the
/// increment overflows, the mask is no longer correct.
DataAndControlFlow,
- /// Use predicate to control both data and control flow, but modify
- /// the trip count so that a runtime overflow check can be avoided
- /// and such that the scalar epilogue loop can always be removed.
- DataAndControlFlowWithoutRuntimeCheck,
/// Use predicated EVL instructions for tail-folding.
/// Indicates that VP intrinsics should be used.
DataWithEVL,
@@ -757,13 +753,7 @@ class TargetTransformInfo {
LLVM_ABI bool preferPredicateOverEpilogue(TailFoldingInfo *TFI) const;
/// Query the target what the preferred style of tail folding is.
- /// \param IVUpdateMayOverflow Tells whether it is known if the IV update
- /// may (or will never) overflow for the suggested VF/UF in the given loop.
- /// Targets can use this information to select a more optimal tail folding
- /// style. The value conservatively defaults to true, such that no assumptions
- /// are made on overflow.
- LLVM_ABI TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow = true) const;
+ LLVM_ABI TailFoldingStyle getPreferredTailFoldingStyle() const;
// Parameters that control the loop peeling transformation
struct PeelingPreferences {
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index e062b70be6b59..6e5d7d308be21 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -278,8 +278,7 @@ class TargetTransformInfoImplBase {
return false;
}
- virtual TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow = true) const {
+ virtual TailFoldingStyle getPreferredTailFoldingStyle() const {
return TailFoldingStyle::DataWithoutLaneMask;
}
diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index 6dcb6f0062a08..1ccc6deb9cebd 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -802,9 +802,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
return BaseT::preferPredicateOverEpilogue(TFI);
}
- TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow = true) const override {
- return BaseT::getPreferredTailFoldingStyle(IVUpdateMayOverflow);
+ TailFoldingStyle getPreferredTailFoldingStyle() const override {
+ return BaseT::getPreferredTailFoldingStyle();
}
std::optional<Instruction *>
diff --git a/llvm/lib/Analysis/TargetTransformInfo.cpp b/llvm/lib/Analysis/TargetTransformInfo.cpp
index 0f97edc424d7e..957f428d9c09c 100644
--- a/llvm/lib/Analysis/TargetTransformInfo.cpp
+++ b/llvm/lib/Analysis/TargetTransformInfo.cpp
@@ -386,9 +386,8 @@ bool TargetTransformInfo::preferPredicateOverEpilogue(
return TTIImpl->preferPredicateOverEpilogue(TFI);
}
-TailFoldingStyle TargetTransformInfo::getPreferredTailFoldingStyle(
- bool IVUpdateMayOverflow) const {
- return TTIImpl->getPreferredTailFoldingStyle(IVUpdateMayOverflow);
+TailFoldingStyle TargetTransformInfo::getPreferredTailFoldingStyle() const {
+ return TTIImpl->getPreferredTailFoldingStyle();
}
std::optional<Instruction *>
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
index f247e9e49e23f..c7716020da13c 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
@@ -464,14 +464,9 @@ class AArch64TTIImpl final : public BasicTTIImplBase<AArch64TTIImpl> {
return ST->hasSVE() ? 5 : 0;
}
- TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow) const override {
- if (ST->hasSVE())
- return IVUpdateMayOverflow
- ? TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck
- : TailFoldingStyle::DataAndControlFlow;
-
- return TailFoldingStyle::DataWithoutLaneMask;
+ TailFoldingStyle getPreferredTailFoldingStyle() const override {
+ return ST->hasSVE() ? TailFoldingStyle::DataAndControlFlow
+ : TailFoldingStyle::DataWithoutLaneMask;
}
bool preferFixedOverScalableIfEqualCost(bool IsEpilogue) const override;
diff --git a/llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp b/llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
index a7766d91af3f2..2a8ca56b17a92 100644
--- a/llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
+++ b/llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
@@ -2670,8 +2670,7 @@ bool ARMTTIImpl::preferPredicateOverEpilogue(TailFoldingInfo *TFI) const {
*LVL->getDominatorTree());
}
-TailFoldingStyle
-ARMTTIImpl::getPreferredTailFoldingStyle(bool IVUpdateMayOverflow) const {
+TailFoldingStyle ARMTTIImpl::getPreferredTailFoldingStyle() const {
if (!ST->hasMVEIntegerOps() || !EnableTailPredication)
return TailFoldingStyle::DataWithoutLaneMask;
diff --git a/llvm/lib/Target/ARM/ARMTargetTransformInfo.h b/llvm/lib/Target/ARM/ARMTargetTransformInfo.h
index 94804152d96ec..75a32d81a44aa 100644
--- a/llvm/lib/Target/ARM/ARMTargetTransformInfo.h
+++ b/llvm/lib/Target/ARM/ARMTargetTransformInfo.h
@@ -432,8 +432,7 @@ class ARMTTIImpl final : public BasicTTIImplBase<ARMTTIImpl> {
TTI::UnrollingPreferences &UP,
OptimizationRemarkEmitter *ORE) const override;
- TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow = true) const override;
+ TailFoldingStyle getPreferredTailFoldingStyle() const override;
void getPeelingPreferences(Loop *L, ScalarEvolution &SE,
TTI::PeelingPreferences &PP) const override;
diff --git a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
index 424f9fe52c59e..97a364d741314 100644
--- a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
+++ b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
@@ -122,8 +122,7 @@ class RISCVTTIImpl final : public BasicTTIImplBase<RISCVTTIImpl> {
bool preferPredicateOverEpilogue(TailFoldingInfo *TFI) const override {
return ST->hasVInstructions();
}
- TailFoldingStyle
- getPreferredTailFoldingStyle(bool IVUpdateMayOverflow) const override {
+ TailFoldingStyle getPreferredTailFoldingStyle() const override {
return ST->hasVInstructions() ? TailFoldingStyle::DataWithEVL
: TailFoldingStyle::None;
}
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 0debab4a2a0ee..c6e0247c1c9ea 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -243,9 +243,6 @@ static cl::opt<TailFoldingStyle> ForceTailFoldingStyle(
clEnumValN(TailFoldingStyle::DataAndControlFlow, "data-and-control",
"Create lane mask using active.lane.mask intrinsic, and use "
"it for both data and control flow"),
- clEnumValN(TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck,
- "data-and-control-without-rt-check",
- "Similar to data-and-control, but remove the runtime check"),
clEnumValN(TailFoldingStyle::DataWithEVL, "data-with-evl",
"Use predicated EVL instructions for tail folding. If EVL "
"is unsupported, fallback to data-without-lane-mask.")));
@@ -1327,11 +1324,10 @@ class LoopVectorizationCostModel {
}
/// Returns the TailFoldingStyle that is best for the current loop.
- TailFoldingStyle getTailFoldingStyle(bool IVUpdateMayOverflow = true) const {
+ TailFoldingStyle getTailFoldingStyle() const {
if (!ChosenTailFoldingStyle)
return TailFoldingStyle::None;
- return IVUpdateMayOverflow ? ChosenTailFoldingStyle->first
- : ChosenTailFoldingStyle->second;
+ return *ChosenTailFoldingStyle;
}
/// Selects and saves TailFoldingStyle for 2 options - if IV update may
@@ -1341,20 +1337,16 @@ class LoopVectorizationCostModel {
void setTailFoldingStyles(bool IsScalableVF, unsigned UserIC) {
assert(!ChosenTailFoldingStyle && "Tail folding must not be selected yet.");
if (!Legal->canFoldTailByMasking()) {
- ChosenTailFoldingStyle = {TailFoldingStyle::None, TailFoldingStyle::None};
+ ChosenTailFoldingStyle = TailFoldingStyle::None;
return;
}
// Default to TTI preference, but allow command line override.
- ChosenTailFoldingStyle = {
- TTI.getPreferredTailFoldingStyle(/*IVUpdateMayOverflow=*/true),
- TTI.getPreferredTailFoldingStyle(/*IVUpdateMayOverflow=*/false)};
+ ChosenTailFoldingStyle = TTI.getPreferredTailFoldingStyle();
if (ForceTailFoldingStyle.getNumOccurrences())
- ChosenTailFoldingStyle = {ForceTailFoldingStyle.getValue(),
- ForceTailFoldingStyle.getValue()};
+ ChosenTailFoldingStyle = ForceTailFoldingStyle.getValue();
- if (ChosenTailFoldingStyle->first != TailFoldingStyle::DataWithEVL &&
- ChosenTailFoldingStyle->second != TailFoldingStyle::DataWithEVL)
+ if (ChosenTailFoldingStyle != TailFoldingStyle::DataWithEVL)
return;
// Override EVL styles if needed.
// FIXME: Investigate opportunity for fixed vector factor.
@@ -1366,10 +1358,9 @@ class LoopVectorizationCostModel {
// if it's allowed, or DataWithoutLaneMask otherwise.
if (ScalarEpilogueStatus == CM_ScalarEpilogueAllowed ||
ScalarEpilogueStatus == CM_ScalarEpilogueNotNeededUsePredicate)
- ChosenTailFoldingStyle = {TailFoldingStyle::None, TailFoldingStyle::None};
+ ChosenTailFoldingStyle = TailFoldingStyle::None;
else
- ChosenTailFoldingStyle = {TailFoldingStyle::DataWithoutLaneMask,
- TailFoldingStyle::DataWithoutLaneMask};
+ ChosenTailFoldingStyle = TailFoldingStyle::DataWithoutLaneMask;
LLVM_DEBUG(
dbgs() << "LV: Preference for VP intrinsics indicated. Will "
@@ -1381,8 +1372,6 @@ class LoopVectorizationCostModel {
/// Returns true if all loop blocks should be masked to fold tail loop.
bool foldTailByMasking() const {
- // TODO: check if it is possible to check for None style independent of
- // IVUpdateMayOverflow flag in getTailFoldingStyle.
return getTailFoldingStyle() != TailFoldingStyle::None;
}
@@ -1393,8 +1382,7 @@ class LoopVectorizationCostModel {
return false;
TailFoldingStyle TF = getTailFoldingStyle();
- return TF == TailFoldingStyle::DataAndControlFlow ||
- TF == TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck;
+ return TF == TailFoldingStyle::DataAndControlFlow;
}
/// Return maximum safe number of elements to be processed per vector
@@ -1608,8 +1596,7 @@ class LoopVectorizationCostModel {
/// Control finally chosen tail folding style. The first element is used if
/// the IV update may overflow, the second element - if it does not.
- std::optional<std::pair<TailFoldingStyle, TailFoldingStyle>>
- ChosenTailFoldingStyle;
+ std::optional<TailFoldingStyle> ChosenTailFoldingStyle;
/// true if scalable vectorization is supported and enabled.
std::optional<bool> IsScalableVectorizationAllowed;
@@ -2098,13 +2085,11 @@ class GeneratedRTChecks {
static bool useActiveLaneMask(TailFoldingStyle Style) {
return Style == TailFoldingStyle::Data ||
- Style == TailFoldingStyle::DataAndControlFlow ||
- Style == TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck;
+ Style == TailFoldingStyle::DataAndControlFlow;
}
static bool useActiveLaneMaskForControlFlow(TailFoldingStyle Style) {
- return Style == TailFoldingStyle::DataAndControlFlow ||
- Style == TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck;
+ return Style == TailFoldingStyle::DataAndControlFlow;
}
// Return true if \p OuterLp is an outer loop annotated with hints for explicit
@@ -8194,7 +8179,7 @@ VPlanPtr LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(
for (ElementCount VF : Range)
IVUpdateMayOverflow |= !isIndvarOverflowCheckKnownFalse(&CM, VF);
- TailFoldingStyle Style = CM.getTailFoldingStyle(IVUpdateMayOverflow);
+ TailFoldingStyle Style = CM.getTailFoldingStyle();
// Use NUW for the induction increment if we proved that it won't overflow in
// the vector loop or when not folding the tail. In the later case, we know
// that the canonical induction increment will not overflow as the vector trip
@@ -8406,10 +8391,7 @@ VPlanPtr LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(
// TODO: Move checks to VPlanTransforms::addActiveLaneMask once
// TailFoldingStyle is visible there.
bool ForControlFlow = useActiveLaneMaskForControlFlow(Style);
- bool WithoutRuntimeCheck =
- Style == TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck;
- VPlanTransforms::addActiveLaneMask(*Plan, ForControlFlow,
- WithoutRuntimeCheck);
+ VPlanTransforms::addActiveLaneMask(*Plan, ForControlFlow);
}
VPlanTransforms::optimizeInductionExitUsers(*Plan, IVEndValues, PSE);
diff --git a/llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp b/llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
index 4c393f2f6f05a..4206dc1143b7c 100644
--- a/llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
@@ -2825,33 +2825,23 @@ void VPlanTransforms::optimize(VPlan &Plan) {
// dropped from the canonical IV increment. Return the created
// VPActiveLaneMaskPHIRecipe.
//
-// The function uses the following definitions:
-//
-// %TripCount = DataWithControlFlowWithoutRuntimeCheck ?
-// calculate-trip-count-minus-VF (original TC) : original TC
-// %IncrementValue = DataWithControlFlowWithoutRuntimeCheck ?
-// CanonicalIVPhi : CanonicalIVIncrement
-// %StartV is the canonical induction start value.
-//
// The function adds the following recipes:
//
// vector.ph:
-// %TripCount = calculate-trip-count-minus-VF (original TC)
-// [if DataWithControlFlowWithoutRuntimeCheck]
-// %EntryInc = canonical-iv-increment-for-part %StartV
-// %EntryALM = active-lane-mask %EntryInc, %TripCount
+// %EntryInc = canonical-iv-increment-for-part CanonicalIVStart
+// %EntryALM = active-lane-mask %EntryInc, TC
//
// vector.body:
// ...
// %P = active-lane-mask-phi [ %EntryALM, %vector.ph ], [ %ALM, %vector.body ]
// ...
-// %InLoopInc = canonical-iv-increment-for-part %IncrementValue
-// %ALM = active-lane-mask %InLoopInc, TripCount
+// %InLoopInc = canonical-iv-increment-for-part CanonicalIVIncrement
+// %ALM = active-lane-mask %InLoopInc, TC
// %Negated = Not %ALM
// branch-on-cond %Negated
//
-static VPActiveLaneMaskPHIRecipe *addVPLaneMaskPhiAndUpdateExitBranch(
- VPlan &Plan, bool DataAndControlFlowWithoutRuntimeCheck) {
+static VPActiveLaneMaskPHIRecipe *
+addVPLaneMaskPhiAndUpdateExitBranch(VPlan &Plan) {
VPRegionBlock *TopRegion = Plan.getVectorLoopRegion();
VPBasicBlock *EB = TopRegion->getExitingBasicBlock();
auto *CanonicalIVPHI = TopRegion->getCanonicalIV();
@@ -2871,24 +2861,8 @@ static VPActiveLaneMaskPHIRecipe *addVPLaneMaskPhiAndUpdateExitBranch(
// Create the ActiveLaneMask instruction using the correct start values.
VPValue *TC = Plan.getTripCount();
- VPValue *VFxUF = &Plan.getVFxUF();
VPValue *VF = &Plan.getVF();
- VPValue *TripCount, *IncrementValue;
- if (!DataAndControlFlowWithoutRuntimeCheck) {
- // When the loop is guarded by a runtime overflow check for the loop
- // induction variable increment by VF, we can increment the value before
- // the get.active.lane mask and use the unmodified tripcount.
- IncrementValue = CanonicalIVIncrement;
- TripCount = TC;
- } else {
- // When avoiding a runtime check, the active.lane.mask inside the loop
- // uses a modified trip count and the induction variable increment is
- // done after the active.lane.mask intrinsic is called.
- IncrementValue = CanonicalIVPHI;
- TripCount = Builder.createNaryOp(VPInstruction::CalculateTripCountMinusVF,
- {TC, VFxUF}, DL);
- }
auto *EntryIncrement = Builder.createOverflowingOp(
VPInstruction::CanonicalIVIncrementForPart, {StartV, VF}, {false, false},
DL, "index.part.next");
@@ -2912,10 +2886,10 @@ static VPActiveLaneMaskPHIRecipe *addVPLaneMaskPhiAndUpdateExitBranch(
Builder.setInsertPoint(OriginalTerminator);
auto *InLoopIncrement = Builder.createOverflowingOp(
VPInstruction::CanonicalIVIncrementForPart,
- {IncrementValue, &Plan.getVF()}, {false, false}, DL);
+ {CanonicalIVIncrement, &Plan.getVF()}, {false, false}, DL);
auto *ALM = Builder.createNaryOp(VPInstruction::ActiveLaneMask,
- {InLoopIncrement, TripCount, ALMMultiplier},
- DL, "active.lane.mask.next");
+ {InLoopIncrement, TC, ALMMultiplier}, DL,
+ "active.lane.mask.next");
LaneMaskPhi->addOperand(ALM);
// Replace the original terminator with BranchOnCond. We have to invert the
@@ -2926,14 +2900,8 @@ static VPActiveLaneMaskPHIRecipe *addVPLaneMaskPhiAndUpdateExitBranch(
return LaneMaskPhi;
}
-void VPlanTransforms::addActiveLaneMask(
- VPlan &Plan, bool UseActiveLaneMaskForControlFlow,
- bool DataAndControlFlowWithoutRuntimeCheck) {
- assert((!DataAndControlFlowWithoutRuntimeCheck ||
- UseActiveLaneMaskForControlFlow) &&
- "DataAndControlFlowWithoutRuntimeCheck implies "
- "UseActiveLaneMaskForControlFlow");
-
+void VPlanTransforms::addActiveLaneMask(VPlan &Plan,
+ bool UseActiveLaneMaskForControlFlow) {
VPRegionBlock *LoopRegion = Plan.getVectorLoopRegion();
auto *FoundWidenCanonicalIVUser = find_if(
LoopRegion->getCanonicalIV()->users(), IsaPred<VPWidenCanonicalIVRecipe>);
@@ -2944,8 +2912,7 @@ void VPlanTransforms::addActiveLaneMask(
cast<VPWidenCanonicalIVRecipe>(*FoundWidenCanonicalIVUser);
VPSingleDefRecipe *LaneMask;
if (UseActiveLaneMaskForControlFlow) {
- LaneMask = addVPLaneMaskPhiAndUpdateExitBranch(
- Plan, DataAndControlFlowWithoutRuntimeCheck);
+ LaneMask = addVPLaneMaskPhiAndUpdateExitBranch(Plan);
} else {
VPBuilder B = VPBuilder::getToInsertAfter(WideCanonicalIV);
VPValue *ALMMultiplier =
diff --git a/llvm/lib/Transforms/Vectorize/VPlanTransforms.h b/llvm/lib/Transforms/Vectorize/VPlanTransforms.h
index 0dce486cb1c2c..16f7ae2daeb5e 100644
--- a/llvm/lib/Transforms/Vectorize/VPlanTransforms.h
+++ b/llvm/lib/Transforms/Vectorize/VPlanTransforms.h
@@ -253,14 +253,9 @@ struct VPlanTransforms {
/// Replace (ICMP_ULE, wide canonical IV, backedge-taken-count) checks with an
/// (active-lane-mask recipe, wide canonical IV, trip-count). If \p
/// UseActiveLaneMaskForControlFlow is true, introduce an
- /// VPActiveLaneMaskPHIRecipe. If \p DataAndControlFlowWithoutRuntimeCheck is
- /// true, no minimum-iteration runtime check will be created (during skeleton
- /// creation) and instead it is handled using active-lane-mask. \p
- /// DataAndControlFlowWithoutRuntimeCheck implies \p
- /// UseActiveLaneMaskForControlFlow.
+ /// VPActiveLaneMaskPHIRecipe.
static void addActiveLaneMask(VPlan &Plan,
- bool UseActiveLaneMaskForControlFlow,
- bool DataAndControlFlowWithoutRuntimeCheck);
+ bool UseActiveLaneMaskForControlFlow);
/// Insert truncates and extends for any truncated recipe. Redundant casts
/// will be folded later.
diff --git a/llvm/test/Transforms/LoopVectorize/AArch64/conditional-branches-cost.ll b/llvm/test/Transforms/LoopVectorize/AArch64/conditional-branches-cost.ll
in...
[truncated]
|
Stacked on llvm#183729 After llvm#144963 and llvm#183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow. With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value)
lukel97
force-pushed
the
loop-vectorize/remove-DataAndControlFlowWithoutRuntimeCheck
branch
from
032be64 to
17bb026
Compare
paulwalker-arm
approved these changes
Feb 28, 2026
david-arm
reviewed
Mar 2, 2026
| return TailFoldingStyle::None; | ||
| return IVUpdateMayOverflow ? ChosenTailFoldingStyle->first | ||
| : ChosenTailFoldingStyle->second; | ||
| return *ChosenTailFoldingStyle; |
Contributor
There was a problem hiding this comment.
nit: Can this just be a one-liner, i.e.
return ChosenTailFoldingStyle ? *ChosenTailFoldingStyle : TailFoldingStyle::None;
In fact, do we still need ChosenTailFoldingStyle to be optional?
| TailFoldingStyle TF = getTailFoldingStyle(); | ||
| return TF == TailFoldingStyle::DataAndControlFlow || | ||
| TF == TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck; | ||
| return TF == TailFoldingStyle::DataAndControlFlow; |
Contributor
There was a problem hiding this comment.
nit: I think you can just do
return getTailFoldingStyle() != TailFoldingStyle::DataAndControlFlow;
| /// the IV update may overflow, the second element - if it does not. | ||
| std::optional<std::pair<TailFoldingStyle, TailFoldingStyle>> | ||
| ChosenTailFoldingStyle; | ||
| std::optional<TailFoldingStyle> ChosenTailFoldingStyle; |
Contributor
There was a problem hiding this comment.
Same question as above - does this still need to be an optional?
Contributor
Author
There was a problem hiding this comment.
setTailFoldingStyle checks it's unset to make sure it's only set once, but looking through the git blame it looks like it didn't used to be an std::optional before the EVL stuff was added, it just checked it was TailFoldingStyle::None.
Removed the optional in d13671b, good idea thanks! The type's much simpler now
…ze/remove-DataAndControlFlowWithoutRuntimeCheck
nasherm
pushed a commit
to nasherm/llvm-project
that referenced
this pull request
Mar 3, 2026
After llvm#144963 and llvm#183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow. With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value)
sahas3
pushed a commit
to sahas3/llvm-project
that referenced
this pull request
Mar 4, 2026
After llvm#144963 and llvm#183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow. With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value)
sujianIBM
pushed a commit
to sujianIBM/llvm-project
that referenced
this pull request
Mar 5, 2026
After llvm#144963 and llvm#183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow. With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After #144963 and #183292 we never emit the runtime check, so DataAndControlFlowWithoutRuntimeCheck is equivalent to DataAndControlFlow.
With that we only need to store one tail folding style instead of two, because we don't need to distinguish whether or not the IV update overflows (to a non-zero value)