[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. #95305
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -928,8 +928,19 @@ void VPlan::prepareToExecute(Value *TripCountV, Value *VectorTripCountV, | |||||||||||||||||
|
|
||||||||||||||||||
| IRBuilder<> Builder(State.CFG.PrevBB->getTerminator()); | ||||||||||||||||||
| // FIXME: Model VF * UF computation completely in VPlan. | ||||||||||||||||||
|
Collaborator
There was a problem hiding this comment. Would it be possible to handle this FIXME now that VF is assigned a VPValue?
Contributor
Author
There was a problem hiding this comment. Yes, would either require introducing a
Collaborator
There was a problem hiding this comment. ok, so FIXME remains until a symbolic UF VPValue placeholder is introduced, as follow-up, along with a VPInstruction multiplying it with VF (introduced here), possibly subject to constant folding? |
||||||||||||||||||
| Value *RuntimeVF = nullptr; | ||||||||||||||||||
| if (VF.getNumUsers()) { | ||||||||||||||||||
| RuntimeVF = createStepForVF(Builder, TripCountV->getType(), State.VF, 1); | ||||||||||||||||||
|
||||||||||||||||||
| Value *RuntimeVF = nullptr; | |
| if (VF.getNumUsers()) { | |
| RuntimeVF = createStepForVF(Builder, TripCountV->getType(), State.VF, 1); | |
| if (VF.getNumUsers()) { | |
| Value *RuntimeVF = getRuntimeVF(Builder, TripCountV->getType(), State.VF); |
could setting VF and/or VFxUF be done by optimizeForVFAndUF() instead, i.e., as soon as they are fixed for a VPlan?
The term RuntimeVF (here and below) may be confusing, as it relates to the Static (Fixed or Scalable) VF, as indicated in the Type of vector Values, rather than the Dynamic EVL.
There was a problem hiding this comment.
Adjusted, thanks!
could setting VF and/or VFxUF be done by optimizeForVFAndUF() instead, i.e., as soon as they are fixed for a VPlan?
Could do, but may not be ideal to rely on an optimization to set them?
There was a problem hiding this comment.
could setting VF and/or VFxUF be done by optimizeForVFAndUF() instead, i.e., as soon as they are fixed for a VPlan?
Could do, but may not be ideal to rely on an optimization to set them?
How about renaming the optimizeForVFAndUF() VPlanTransform to setVFAndUF(), or setVF() and setUF() VPlanTransforms, where fixing a constant value triggers its folding optimizations?
There was a problem hiding this comment.
Sounds good, can update to setVFAndUF as a start separately?
There was a problem hiding this comment.
Given RuntimeVF is only non-null if VF.getNumUsers() != 0 wouldn't it be neater to simply fold this into the if (VF.getNumUsers()) { block? i.e.
if (VF.getNumUsers()) {
RuntimeVF = createStepForVF(Builder, TripCountV->getType(), State.VF, 1);
VF.setUnderlyingValue(RuntimeVF);
VFxUF.setUnderlyingValue(
State.UF > 1 ? Builder.CreateMul(
VF.getLiveInIRValue(),
ConstantInt::get(TripCountV->getType(), State.UF))
: VF.getLiveInIRValue());
} else {
VFxUF.setUnderlyingValue(
createStepForVF(Builder, TripCountV->getType(), State.VF, State.UF));
}
Outdated
There was a problem hiding this comment.
VFxUF can always be set via createStepForVF(), regardless of UF=1, UF>1, VF having users or not. But we want to set it directly instead, using VF - when the latter is built with getRuntimeVF()? Maybe related to @david-arm's comment below about "... why we need a specialised version for the UF=1 case".
| State.UF > 1 ? Builder.CreateMul( | |
| VF.getLiveInIRValue(), | |
| ConstantInt::get(TripCountV->getType(), State.UF)) | |
| : VF.getLiveInIRValue()); | |
| State.UF > 1 ? Builder.CreateMul( | |
| RuntimeVF, | |
| ConstantInt::get(TripCountV->getType(), State.UF)) | |
| : RuntimeVF); |
There was a problem hiding this comment.
But we want to set it directly instead, using VF - when the latter is built with getRuntimeVF(
yep, to reduce calls to @vscale .
Updated to use RuntimeVF variable, thanks!
There was a problem hiding this comment.
VFxUF is set regardless of having users or not. Be consistent?
There was a problem hiding this comment.
I see earlier that
... Code for VF is only generated if there are users of VF, to avoid unnecessary test changes.
Code for VF must be generated regardless of direct users, issues is its position and possible repetition?
There was a problem hiding this comment.
Yes, VFxUF is used to increment the canonical induction, which is present in almost all cases, but could also be done only if users exist.
... Code for VF is only generated if there are users of VF, to avoid unnecessary test changes.
Code for VF must be generated regardless of direct users, issues is its position and possible repetition?
There are cases where VF separately isn't used, only as part of VFxUF. If only VFxUF is used, the constant multiply of VF * UF is folded, hence always generating VF separately would lead to additional test changes. Some alternatives are mentioned in my latest comment above.
There was a problem hiding this comment.
Yes, VFxUF is used to increment the canonical induction, which is present in almost all cases, but could also be done only if users exist.
Exceptional use-less cases are loops whose trip count is VFxUF - where optimizeForVFAndUF() discards the canonical IV's increment by VFxUF?
So one way to improve consistency, w/o changing too many tests, would be to (also) set VFxUF only if used? Although logically it should always be used - to set the vector trip count(?).
There was a problem hiding this comment.
At the moment, it is always used (added assert in 1a5a1e9).
There was a problem hiding this comment.
Should this condition be an assert?
There was a problem hiding this comment.
Yes, will adjust separately
There was a problem hiding this comment.
I must be honest it's not obvious to me why we need a specialised version for the UF=1 case. Why can't we use VFxUF always given VFx1 is a valid use case? It seems to add a bit of extra complexity. I understand in a loop we may want to calculate both a runtime VF (for a second part or something) and a runtime VF x UF (for a canonical induction variable, etc), but if we've filled out Old2NewVPValues[&VFxUF] with both VFx1 and VFxUF we can just query accordingly, right?
There was a problem hiding this comment.
ut if we've filled out Old2NewVPValues[&VFxUF] with both VFx1 and VFxUF we can just query accordingly, right?
I am not sure what filling them out accordingly means here, perhaps using a pair instead of 2 separate fields?
There are different places that need both VF and VFxUF and to serve them we would need different values I think, but I might be missing something from your suggestion?
There was a problem hiding this comment.
| Old2NewVPValues[&VFxUF] = &NewPlan->VFxUF; | |
| Old2NewVPValues[&VF] = &NewPlan->VF; | |
| Old2NewVPValues[&VF] = &NewPlan->VF; | |
| Old2NewVPValues[&VFxUF] = &NewPlan->VFxUF; |
There was a problem hiding this comment.
Should this condition be an assert?
There was a problem hiding this comment.
Yes, will adjust separately
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -1770,25 +1770,27 @@ class VPWidenIntOrFpInductionRecipe : public VPHeaderPHIRecipe { | |||||||
|
|
||||||||
| public: | ||||||||
| VPWidenIntOrFpInductionRecipe(PHINode *IV, VPValue *Start, VPValue *Step, | ||||||||
| VPValue *VF, const InductionDescriptor &IndDesc) | ||||||||
| : VPHeaderPHIRecipe(VPDef::VPWidenIntOrFpInductionSC, IV, Start), IV(IV), | ||||||||
| Trunc(nullptr), IndDesc(IndDesc) { | ||||||||
| addOperand(Step); | ||||||||
| addOperand(VF); | ||||||||
| } | ||||||||
|
|
||||||||
| VPWidenIntOrFpInductionRecipe(PHINode *IV, VPValue *Start, VPValue *Step, | ||||||||
| VPValue *VF, const InductionDescriptor &IndDesc, | ||||||||
| TruncInst *Trunc) | ||||||||
| : VPHeaderPHIRecipe(VPDef::VPWidenIntOrFpInductionSC, Trunc, Start), | ||||||||
| IV(IV), Trunc(Trunc), IndDesc(IndDesc) { | ||||||||
| addOperand(Step); | ||||||||
| addOperand(VF); | ||||||||
| } | ||||||||
|
|
||||||||
| ~VPWidenIntOrFpInductionRecipe() override = default; | ||||||||
|
|
||||||||
| VPWidenIntOrFpInductionRecipe *clone() override { | ||||||||
| return new VPWidenIntOrFpInductionRecipe( | ||||||||
| IV, getStartValue(), getStepValue(), getOperand(2), IndDesc, Trunc); | ||||||||
|
||||||||
| IV, getStartValue(), getStepValue(), getOperand(2), IndDesc, Trunc); | |
| IV, getStartValue(), getStepValue(), getVFValue(), IndDesc, Trunc); |
There was a problem hiding this comment.
| VPValue VF; | |
| /// Represents the vectorization factor of the loop. | |
| VPValue VF; |
(belongs to the loop Region rather than VPlan itself)
There was a problem hiding this comment.
Added thanks!
VF may also be referenced outside the loop region, so probably should be defined at the top-level and used by the region?
There was a problem hiding this comment.
(This is a follow-up thought, hence originally in brackets)
Regions (and blocks in general) currently model the HCFG only, leaving the def-use graph of values to recipes. A loop region has a canonical IV recipe which typically uses VFxUF as the canonical step controlling the loop. The latter could be a Mul recipe whose operands provide VF and UF, and a loop region could provide getVF() and getUF() to ease their retrieval?
There was a problem hiding this comment.
Agreed, I think ideally the loop region would be a user of both VF, UF (and possibly VF x UF).
Outdated
There was a problem hiding this comment.
| VPValue *getVF() { return &VF; }; | |
| /// Returns the VF of the vector loop region. | |
| VPValue *getVF() { return &VF; }; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1243,14 +1243,6 @@ static Constant *getSignedIntOrFpConstant(Type *Ty, int64_t C) { | |
| : ConstantFP::get(Ty, C); | ||
| } | ||
|
|
||
| void VPWidenIntOrFpInductionRecipe::execute(VPTransformState &State) { | ||
| assert(!State.Instance && "Int or FP induction being replicated."); | ||
|
|
||
|
|
@@ -1308,11 +1300,11 @@ void VPWidenIntOrFpInductionRecipe::execute(VPTransformState &State) { | |
| // Multiply the vectorization factor by the step using integer or | ||
| // floating-point arithmetic as appropriate. | ||
| Type *StepType = Step->getType(); | ||
| Value *RuntimeVF = State.get(getVFValue(), {0, 0}); | ||
| if (Step->getType()->isFloatingPointTy()) | ||
| RuntimeVF = Builder.CreateUIToFP(RuntimeVF, StepType); | ||
| else | ||
| RuntimeVF = Builder.CreateZExtOrTrunc(RuntimeVF, StepType); | ||
| Value *Mul = Builder.CreateBinOp(MulOp, Step, RuntimeVF); | ||
|
|
||
| // Create a vector splat to use in the induction update. | ||
|
|
@@ -1366,6 +1358,9 @@ void VPWidenIntOrFpInductionRecipe::print(raw_ostream &O, const Twine &Indent, | |
|
|
||
| O << ", "; | ||
| getStepValue()->printAsOperand(O, SlotTracker); | ||
|
|
||
| O << ", "; | ||
| getVFValue()->printAsOperand(O, SlotTracker); | ||
| } | ||
| #endif | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,7 +58,8 @@ void VPlanTransforms::VPInstructionsToVPRecipes( | |
| VPValue *Start = Plan->getOrAddLiveIn(II->getStartValue()); | ||
| VPValue *Step = | ||
| vputils::getOrCreateVPValueForSCEVExpr(*Plan, II->getStep(), SE); | ||
| NewRecipe = new VPWidenIntOrFpInductionRecipe(Phi, Start, Step, | ||
| Plan->getVF(), *II); | ||
| } else { | ||
| assert(isa<VPInstruction>(&Ingredient) && | ||
| "only VPInstructions expected here"); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,9 +21,7 @@ define void @clamped_tc_8(ptr nocapture %dst, i32 %n, i64 %val) vscale_range(1,1 | |
| ; CHECK-NEXT: [[TMP8:%.*]] = add <vscale x 4 x i64> [[TMP7]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP9:%.*]] = mul <vscale x 4 x i64> [[TMP8]], shufflevector (<vscale x 4 x i64> insertelement (<vscale x 4 x i64> poison, i64 1, i64 0), <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 4 x i64> zeroinitializer, [[TMP9]] | ||
| ; CHECK-NEXT: [[TMP12:%.*]] = mul i64 1, [[TMP6]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[TMP12]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 4 x i64> [[DOTSPLATINSERT]], <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer | ||
| ; CHECK-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[VAL]], i64 0 | ||
|
|
@@ -112,9 +110,7 @@ define void @clamped_tc_max_8(ptr nocapture %dst, i32 %n, i64 %val) vscale_range | |
| ; CHECK-NEXT: [[TMP8:%.*]] = add <vscale x 4 x i64> [[TMP7]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP9:%.*]] = mul <vscale x 4 x i64> [[TMP8]], shufflevector (<vscale x 4 x i64> insertelement (<vscale x 4 x i64> poison, i64 1, i64 0), <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 4 x i64> zeroinitializer, [[TMP9]] | ||
| ; CHECK-NEXT: [[TMP12:%.*]] = mul i64 1, [[TMP6]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[TMP12]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 4 x i64> [[DOTSPLATINSERT]], <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer | ||
| ; CHECK-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[VAL]], i64 0 | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,9 +24,7 @@ define void @foo() { | |
| ; CHECK-NEXT: [[TMP5:%.*]] = add <vscale x 4 x i64> [[TMP4]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP6:%.*]] = mul <vscale x 4 x i64> [[TMP5]], shufflevector (<vscale x 4 x i64> insertelement (<vscale x 4 x i64> poison, i64 1, i64 0), <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 4 x i64> zeroinitializer, [[TMP6]] | ||
| ; CHECK-NEXT: [[TMP9:%.*]] = mul i64 1, [[TMP19]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[TMP9]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 4 x i64> [[DOTSPLATINSERT]], <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer | ||
| ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -32,9 +32,7 @@ define void @test_no_scalarization(ptr %a, ptr noalias %b, i32 %idx, i32 %n) #0 | |
| ; CHECK-NEXT: [[TMP9:%.*]] = add <vscale x 2 x i32> [[TMP8]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP10:%.*]] = mul <vscale x 2 x i32> [[TMP9]], shufflevector (<vscale x 2 x i32> insertelement (<vscale x 2 x i32> poison, i32 1, i64 0), <vscale x 2 x i32> poison, <vscale x 2 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i32> [[DOTSPLAT]], [[TMP10]] | ||
| ; CHECK-NEXT: [[TMP13:%.*]] = mul i32 1, [[TMP7]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT1:%.*]] = insertelement <vscale x 2 x i32> poison, i32 [[TMP13]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT2:%.*]] = shufflevector <vscale x 2 x i32> [[DOTSPLATINSERT1]], <vscale x 2 x i32> poison, <vscale x 2 x i32> zeroinitializer | ||
| ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -293,10 +293,9 @@ define void @gather_nxv4i32_ind64_stride2(ptr noalias nocapture %a, ptr noalias | |
| ; CHECK-NEXT: [[DOTNEG:%.*]] = mul nsw i64 [[TMP2]], -8 | ||
| ; CHECK-NEXT: [[N_VEC:%.*]] = and i64 [[DOTNEG]], [[N]] | ||
| ; CHECK-NEXT: [[TMP3:%.*]] = call i64 @llvm.vscale.i64() | ||
| ; CHECK-NEXT: [[TMP7:%.*]] = shl nuw nsw i64 [[TMP3]], 2 | ||
| ; CHECK-NEXT: [[TMP4:%.*]] = shl nuw nsw i64 [[TMP3]], 3 | ||
| ; CHECK-NEXT: [[TMP5:%.*]] = call <vscale x 4 x i64> @llvm.experimental.stepvector.nxv4i64() | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[TMP7]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 4 x i64> [[DOTSPLATINSERT]], <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer | ||
| ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,14 +18,14 @@ define void @induction_i7(ptr %dst) #0 { | |
| ; CHECK-NEXT: [[N_VEC:%.*]] = sub i64 64, [[N_MOD_VF]] | ||
| ; CHECK-NEXT: [[IND_END:%.*]] = trunc i64 [[N_VEC]] to i7 | ||
| ; CHECK-NEXT: [[TMP4:%.*]] = call i64 @llvm.vscale.i64() | ||
| ; CHECK-NEXT: [[TMP40:%.*]] = mul i64 [[TMP4]], 2 | ||
| ; CHECK-NEXT: [[TMP5:%.*]] = mul i64 [[TMP40]], 2 | ||
| ; CHECK-NEXT: [[TMP6:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8() | ||
| ; CHECK-NEXT: [[TMP7:%.*]] = trunc <vscale x 2 x i8> [[TMP6]] to <vscale x 2 x i7> | ||
| ; CHECK-NEXT: [[TMP8:%.*]] = add <vscale x 2 x i7> [[TMP7]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP9:%.*]] = mul <vscale x 2 x i7> [[TMP8]], shufflevector (<vscale x 2 x i7> insertelement (<vscale x 2 x i7> poison, i7 1, i64 0), <vscale x 2 x i7> poison, <vscale x 2 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i7> zeroinitializer, [[TMP9]] | ||
| ; CHECK-NEXT: [[TMP11:%.*]] = trunc i64 [[TMP40]] to i7 | ||
| ; CHECK-NEXT: [[TMP12:%.*]] = mul i7 1, [[TMP11]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 2 x i7> poison, i7 [[TMP12]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 2 x i7> [[DOTSPLATINSERT]], <vscale x 2 x i7> poison, <vscale x 2 x i32> zeroinitializer | ||
|
|
@@ -92,14 +92,14 @@ define void @induction_i3_zext(ptr %dst) #0 { | |
| ; CHECK-NEXT: [[N_VEC:%.*]] = sub i64 64, [[N_MOD_VF]] | ||
| ; CHECK-NEXT: [[IND_END:%.*]] = trunc i64 [[N_VEC]] to i3 | ||
| ; CHECK-NEXT: [[TMP4:%.*]] = call i64 @llvm.vscale.i64() | ||
| ; CHECK-NEXT: [[TMP40:%.*]] = mul i64 [[TMP4]], 2 | ||
| ; CHECK-NEXT: [[TMP5:%.*]] = mul i64 [[TMP40]], 2 | ||
|
Contributor
There was a problem hiding this comment. This is interesting. I guess I was expecting that since this patch only creates a runtime VF on demand that total lines of IR would only decrease, rather than increase which is happening here.
Contributor
Author
There was a problem hiding this comment. The reason this was showing up as net increase was that the test case originally didn't check all instructions in vector.ph; now that it does we safe vscale & mul but need a trunc, so it is neutral overall in this case. |
||
| ; CHECK-NEXT: [[TMP6:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8() | ||
| ; CHECK-NEXT: [[TMP7:%.*]] = trunc <vscale x 2 x i8> [[TMP6]] to <vscale x 2 x i3> | ||
| ; CHECK-NEXT: [[TMP8:%.*]] = add <vscale x 2 x i3> [[TMP7]], zeroinitializer | ||
| ; CHECK-NEXT: [[TMP9:%.*]] = mul <vscale x 2 x i3> [[TMP8]], shufflevector (<vscale x 2 x i3> insertelement (<vscale x 2 x i3> poison, i3 1, i64 0), <vscale x 2 x i3> poison, <vscale x 2 x i32> zeroinitializer) | ||
| ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i3> zeroinitializer, [[TMP9]] | ||
| ; CHECK-NEXT: [[TMP11:%.*]] = trunc i64 [[TMP40]] to i3 | ||
| ; CHECK-NEXT: [[TMP12:%.*]] = mul i3 1, [[TMP11]] | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 2 x i3> poison, i3 [[TMP12]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 2 x i3> [[DOTSPLATINSERT]], <vscale x 2 x i3> poison, <vscale x 2 x i32> zeroinitializer | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,9 +26,7 @@ define void @cond_ind64(ptr noalias nocapture %a, ptr noalias nocapture readonly | |
| ; CHECK-NEXT: [[TMP4:%.*]] = call i64 @llvm.vscale.i64() | ||
| ; CHECK-NEXT: [[TMP5:%.*]] = shl i64 [[TMP4]], 2 | ||
| ; CHECK-NEXT: [[TMP6:%.*]] = call <vscale x 4 x i64> @llvm.experimental.stepvector.nxv4i64() | ||
| ; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 4 x i64> poison, i64 [[TMP5]], i64 0 | ||
| ; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 4 x i64> [[DOTSPLATINSERT]], <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer | ||
| ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] | ||
| ; CHECK: vector.body: | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VPWidenIntOrFpInductionRecipe could retrieve VF by looking up Plan.getVF() on demand rather than recording it as on operand, but the latter helps in checking if VF has users, i.e., if any VPWidenIntOrFpInductionRecipe exists?
Surely VF is needed to vectorize any loop, including ones free of VPWidenIntOrFpInductionRecipes. Does it need to be cached somehow, to prevent regeneration?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly, this is used to check whether to generate it or not.
VFxUFis similarly added as operand to the VPInstruction to increment the canonical IV.We could create VF unconditionally, then we would have update all tests with scalable vectors to split up VFxUF computation to
((vscale * VF) * UF)instead of(vscale * (VF * UF))even ifvscale * VFis only used in the multiply byUF.To limit this we could try to fold it back as post-codegen cleanup. Or update all tests, happy to go either way (or leave as is in the current patch for now)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree to model usage and dependence of values directly via explicit operands, rather than by retrieving them from plan (or region).
If/when VFxUF becomes a Mul VPInstruction which uses VF (and UF), will the check for no VF users change to check if VF is used only by this Mul?
Could this folding be done by a subsequent VPlan2VPlan pass? Would indeed be good to reduce amount of test changes...