[LV] Restrict scalable vectorization to targets with power-of-2 vscale by lukel97 · Pull Request #183065 · llvm/llvm-project

lukel97 · 2026-02-24T14:32:49Z

In #145098 we are proposing to restrict vscale to always be a power of 2.

This PR proposes to go ahead and remove support for non-power-of-2 vscales in the loop vectorizer independently of the LangRef. Given we don't have any targets with scalable vector support with non-power-of-2 vscales, this is essentially NFC.

The main benefit of this is that it means the IV can't overflow with tail folding anymore. This remove a lot of checks, both at runtime and statically, and simplifies other areas:

We can remove the vscale-power-of-2 runtime check code, which isn't emitted on any target
We can remove IVUpdateMayOverflow as it's now always false
addMinimumIterationCheck doesn't need to check for IV overflow
We don't need to drop nuw on the canonical IV when tail folding anymore
We don't need to store two separate values for the tail folding style, (one for iv overflow, one without)
I think we can remove TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck, since that's only used by SVE when the IV can overflow. But AFAIK that can't happen on SVE since it has power of two vscale.

I plan to post patches stacked on top of this for these to show how they can be removed. This patch just restricts it for now to show how no tests are affected.

llvmbot · 2026-02-24T14:33:28Z

@llvm/pr-subscribers-vectorizers

@llvm/pr-subscribers-llvm-transforms

Author: Luke Lau (lukel97)

Changes

In #145098 we are proposing to restrict vscale to always be a power of 2.

This PR proposes to go ahead and remove support for non-power-of-2 vscales in the loop vectorizer independently of the LangRef. Given we don't have any targets with scalable vector support with non-power-of-2 vscales, this is essentially NFC.

The main benefit of this is that it means the IV can't overflow with tail folding anymore. This remove a lot of checks, both at runtime and statically, and simplifies other areas:

We can remove the vscale-power-of-2 runtime check code, which isn't emitted on any target
We can remove IVUpdateMayOverflow as it's now always false
addMinimumIterationCheck doesn't need to check for IV overflow
We don't need to drop nuw on the canonical IV when tail folding anymore
We don't need to store two separate values for the tail folding style, (one for iv overflow, one without)
I think we can remove TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck, since that's only used by SVE when the IV can overflow. But AFAIK that can't happen on SVE since it has power of two vscale.

I plan to post patches stacked on top of this for these to show how they can be removed. This patch just restricts it for now to show how no tests are affected.

Full diff: https://github.com/llvm/llvm-project/pull/183065.diff

2 Files Affected:

(modified) llvm/lib/Target/RISCV/RISCVISelLowering.cpp (-3)
(modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+7-3)

diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 77be8cc95b6da..1726963f43e32 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -25717,9 +25717,6 @@ bool RISCVTargetLowering::isVScaleKnownToBeAPowerOfTwo() const {
   // We define vscale to be VLEN/RVVBitsPerBlock.  VLEN is always a power
   // of two >= 64, and RVVBitsPerBlock is 64.  Thus, vscale must be
   // a power of two as well.
-  // FIXME: This doesn't work for zve32, but that's already broken
-  // elsewhere for the same reason.
-  assert(Subtarget.getRealMinVLen() >= 64 && "zve32* unsupported");
   static_assert(RISCV::RVVBitsPerBlock == 64,
                 "RVVBitsPerBlock changed, audit needed");
   return true;
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index b28c3d949c96a..073a6ccef4c93 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -296,9 +296,9 @@ cl::opt<unsigned> llvm::ForceTargetInstructionCost(
 
 static cl::opt<bool> ForceTargetSupportsScalableVectors(
     "force-target-supports-scalable-vectors", cl::init(false), cl::Hidden,
-    cl::desc(
-        "Pretend that scalable vectors are supported, even if the target does "
-        "not support them. This flag should only be used for testing."));
+    cl::desc("Pretend that scalable vectors are supported and vscale is a "
+             "power of two, even if the target does "
+             "not support them. This flag should only be used for testing."));
 
 static cl::opt<unsigned> SmallLoopCost(
     "small-loop-cost", cl::init(20), cl::Hidden,
@@ -3387,6 +3387,10 @@ bool LoopVectorizationCostModel::isScalableVectorizationAllowed() {
   if (!TTI.supportsScalableVectors() && !ForceTargetSupportsScalableVectors)
     return false;
 
+  if (!TTI.isVScaleKnownToBeAPowerOfTwo() &&
+      !ForceTargetSupportsScalableVectors)
+    return false;
+
   if (Hints->isScalableVectorizationDisabled()) {
     reportVectorizationInfo("Scalable vectorization is explicitly disabled",
                             "ScalableVectorizationDisabled", ORE, TheLoop);

llvmbot · 2026-02-24T14:33:29Z

@llvm/pr-subscribers-backend-risc-v

Author: Luke Lau (lukel97)

Changes

In #145098 we are proposing to restrict vscale to always be a power of 2.

This PR proposes to go ahead and remove support for non-power-of-2 vscales in the loop vectorizer independently of the LangRef. Given we don't have any targets with scalable vector support with non-power-of-2 vscales, this is essentially NFC.

The main benefit of this is that it means the IV can't overflow with tail folding anymore. This remove a lot of checks, both at runtime and statically, and simplifies other areas:

We can remove the vscale-power-of-2 runtime check code, which isn't emitted on any target
We can remove IVUpdateMayOverflow as it's now always false
addMinimumIterationCheck doesn't need to check for IV overflow
We don't need to drop nuw on the canonical IV when tail folding anymore
We don't need to store two separate values for the tail folding style, (one for iv overflow, one without)
I think we can remove TailFoldingStyle::DataAndControlFlowWithoutRuntimeCheck, since that's only used by SVE when the IV can overflow. But AFAIK that can't happen on SVE since it has power of two vscale.

I plan to post patches stacked on top of this for these to show how they can be removed. This patch just restricts it for now to show how no tests are affected.

Full diff: https://github.com/llvm/llvm-project/pull/183065.diff

2 Files Affected:

(modified) llvm/lib/Target/RISCV/RISCVISelLowering.cpp (-3)
(modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+7-3)

diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 77be8cc95b6da..1726963f43e32 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -25717,9 +25717,6 @@ bool RISCVTargetLowering::isVScaleKnownToBeAPowerOfTwo() const {
   // We define vscale to be VLEN/RVVBitsPerBlock.  VLEN is always a power
   // of two >= 64, and RVVBitsPerBlock is 64.  Thus, vscale must be
   // a power of two as well.
-  // FIXME: This doesn't work for zve32, but that's already broken
-  // elsewhere for the same reason.
-  assert(Subtarget.getRealMinVLen() >= 64 && "zve32* unsupported");
   static_assert(RISCV::RVVBitsPerBlock == 64,
                 "RVVBitsPerBlock changed, audit needed");
   return true;
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index b28c3d949c96a..073a6ccef4c93 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -296,9 +296,9 @@ cl::opt<unsigned> llvm::ForceTargetInstructionCost(
 
 static cl::opt<bool> ForceTargetSupportsScalableVectors(
     "force-target-supports-scalable-vectors", cl::init(false), cl::Hidden,
-    cl::desc(
-        "Pretend that scalable vectors are supported, even if the target does "
-        "not support them. This flag should only be used for testing."));
+    cl::desc("Pretend that scalable vectors are supported and vscale is a "
+             "power of two, even if the target does "
+             "not support them. This flag should only be used for testing."));
 
 static cl::opt<unsigned> SmallLoopCost(
     "small-loop-cost", cl::init(20), cl::Hidden,
@@ -3387,6 +3387,10 @@ bool LoopVectorizationCostModel::isScalableVectorizationAllowed() {
   if (!TTI.supportsScalableVectors() && !ForceTargetSupportsScalableVectors)
     return false;
 
+  if (!TTI.isVScaleKnownToBeAPowerOfTwo() &&
+      !ForceTargetSupportsScalableVectors)
+    return false;
+
   if (Hints->isScalableVectorizationDisabled()) {
     reportVectorizationInfo("Scalable vectorization is explicitly disabled",
                             "ScalableVectorizationDisabled", ORE, TheLoop);

lukel97 · 2026-02-24T14:34:46Z

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

@@ -25717,9 +25717,6 @@ bool RISCVTargetLowering::isVScaleKnownToBeAPowerOfTwo() const {
  // We define vscale to be VLEN/RVVBitsPerBlock.  VLEN is always a power
  // of two >= 64, and RVVBitsPerBlock is 64.  Thus, vscale must be
  // a power of two as well.
-  // FIXME: This doesn't work for zve32, but that's already broken
-  // elsewhere for the same reason.
-  assert(Subtarget.getRealMinVLen() >= 64 && "zve32* unsupported");


llvm/test/Transforms/LoopVectorize/RISCV/zvl32b.ll triggers this assert because we query isVScaleKnownToBeAPowerOfTwo earlier.

lukel97 · 2026-02-24T14:35:39Z

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

-        "Pretend that scalable vectors are supported, even if the target does "
-        "not support them. This flag should only be used for testing."));
+    cl::desc("Pretend that scalable vectors are supported and vscale is a "
+             "power of two, even if the target does "


This adjusts the wording so that we can assume power-of-2 vscale in the target independent tests

I'd probably revert this specific hunk, since the LangRef change has landed.

paulwalker-arm · 2026-02-24T14:46:49Z

Why do this independently of the LangRef? Why not land the LangRef change first given it has broad acceptance?

lukel97 · 2026-02-24T14:51:48Z

Why do this independently of the LangRef? Why not land the LangRef change first given it has broad acceptance?

I reverse pinged the LangRef PR two weeks ago but there hasn't been any response yet.

This splits off the loop vectorizer part to unblock the cleanup work. One specific thing I'd like to start fixing without being blocked on that PR is preserving the NUW flag on tail folded canonical IVs. We have to work around an assumption in VPlanVerifier otherwise in #182254, see the comment here: #182254 (comment)

paulwalker-arm · 2026-02-24T15:45:57Z

Why do this independently of the LangRef? Why not land the LangRef change first given it has broad acceptance?

I reverse pinged the LangRef PR two weeks ago but there hasn't been any response yet.

I looked at #145098 but the update was not free so I figured it best to try changing the LangRef in isolation and created #183080.

artagnon

This LGTM, kindly wait on other reviewers.

artagnon · 2026-02-25T08:38:46Z

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

-        "Pretend that scalable vectors are supported, even if the target does "
-        "not support them. This flag should only be used for testing."));
+    cl::desc("Pretend that scalable vectors are supported and vscale is a "
+             "power of two, even if the target does "


I'd probably revert this specific hunk, since the LangRef change has landed.

david-arm · 2026-02-25T08:51:57Z

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

  if (!TTI.supportsScalableVectors() && !ForceTargetSupportsScalableVectors)
    return false;

+  if (!TTI.isVScaleKnownToBeAPowerOfTwo() &&


I think it probably makes sense to now just remove this hook completely, right?

I am working on PR to do this. I just wanted to land the LangRef change first to unblock other related work.

#183292

paulwalker-arm · 2026-02-25T11:10:46Z

Does this PR have any value after the LangRef change? I saw it more as a show of intent. Presumably we can now move straight to the functional changes?

lukel97 · 2026-02-25T13:29:27Z

Does this PR have any value after the LangRef change? I saw it more as a show of intent. Presumably we can now move straight to the functional changes?

Yup this was just to show the intent, thanks for landing the langref change. Closing now

[LV] Restrict scalable vectorization to targets with power-of-2 vscale

071bd1b

lukel97 requested review from MacDue, davemgreen, david-arm, fhahn, sdesmalen-arm and topperc February 24, 2026 14:32

llvmbot added backend:RISC-V vectorizers llvm:transforms labels Feb 24, 2026

lukel97 commented Feb 24, 2026

View reviewed changes

lukel97 mentioned this pull request Feb 24, 2026

[LV] Remove CheckNeededWithTailFolding from addMinimumIterationCheck. NFC #183066

Merged

paulwalker-arm mentioned this pull request Feb 24, 2026

[LLVM][LangRef] Restrict vscale to be a signed power-of-two integer. #183080

Merged

artagnon reviewed Feb 25, 2026

View reviewed changes

david-arm reviewed Feb 25, 2026

View reviewed changes

lukel97 closed this Feb 25, 2026

Conversation

lukel97 commented Feb 24, 2026

Uh oh!

llvmbot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Feb 24, 2026

Uh oh!

lukel97 Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

lukel97 Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

artagnon Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

paulwalker-arm commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lukel97 commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paulwalker-arm commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

artagnon left a comment

Choose a reason for hiding this comment

Uh oh!

artagnon Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

david-arm Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

paulwalker-arm Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulwalker-arm commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lukel97 commented Feb 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

llvmbot commented Feb 24, 2026 •

edited

Loading

paulwalker-arm commented Feb 24, 2026 •

edited

Loading

lukel97 commented Feb 24, 2026 •

edited

Loading

paulwalker-arm commented Feb 24, 2026 •

edited

Loading

paulwalker-arm Feb 25, 2026 •

edited

Loading

paulwalker-arm commented Feb 25, 2026 •

edited

Loading