[RISCV][FPEnv] Lowering of fpmode intrinsics #148569

spavloff · 2025-07-14T06:29:30Z

The change implements custom lowering of get_fpmode, set_fpmode and reset_fpmode for RISCV target. The implementation is aligned with the functions fegetmode and fesetmode in GLIBC.

The change implements custom lowering of `get_fpmode`, `set_fpmode` and `reset_fpmode` for RISCV target. The implementation is aligned with the functions `fegetmode` and `fesetmode` in GLIBC.

llvmbot · 2025-07-14T06:30:04Z

@llvm/pr-subscribers-backend-risc-v

Author: Serge Pavlov (spavloff)

Changes

The change implements custom lowering of get_fpmode, set_fpmode and reset_fpmode for RISCV target. The implementation is aligned with the functions fegetmode and fesetmode in GLIBC.

Full diff: https://github.com/llvm/llvm-project/pull/148569.diff

5 Files Affected:

(modified) llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h (+11)
(modified) llvm/lib/Target/RISCV/RISCVISelLowering.cpp (+58)
(modified) llvm/lib/Target/RISCV/RISCVISelLowering.h (+3)
(modified) llvm/lib/Target/RISCV/RISCVInstrInfo.td (+54)
(modified) llvm/test/CodeGen/RISCV/fpenv-xlen.ll (+35)

diff --git a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h
index f41ad419db1a7..29261ccb39848 100644
--- a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h
+++ b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h
@@ -495,6 +495,17 @@ inline static bool isValidRoundingMode(unsigned Mode) {
 }
 } // namespace RISCVVXRndMode
 
+namespace RISCVExceptFlags {
+enum ExceptionFlag {
+  NX = 0x01,  // Inexact
+  UF = 0x02,  // Underflow
+  OF = 0x04,  // Overflow
+  DZ = 0x08,  // Divide by zero
+  NV = 0x10,  // Invalid operation
+  ALL = 0x1F  // Mask for all accrued exception flags
+};
+}
+
 //===----------------------------------------------------------------------===//
 // Floating-point Immediates
 //
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 7c72d074a35b6..dbc6d684f997d 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -656,6 +656,9 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::GET_FPENV, XLenVT, Custom);
     setOperationAction(ISD::SET_FPENV, XLenVT, Custom);
     setOperationAction(ISD::RESET_FPENV, MVT::Other, Custom);
+    setOperationAction(ISD::GET_FPMODE, XLenVT, Custom);
+    setOperationAction(ISD::SET_FPMODE, XLenVT, Custom);
+    setOperationAction(ISD::RESET_FPMODE, MVT::Other, Custom);
   }
 
   setOperationAction({ISD::GlobalAddress, ISD::BlockAddress, ISD::ConstantPool,
@@ -8226,6 +8229,12 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
     return lowerSET_FPENV(Op, DAG);
   case ISD::RESET_FPENV:
     return lowerRESET_FPENV(Op, DAG);
+  case ISD::GET_FPMODE:
+    return lowerGET_FPMODE(Op, DAG);
+  case ISD::SET_FPMODE:
+    return lowerSET_FPMODE(Op, DAG);
+  case ISD::RESET_FPMODE:
+    return lowerRESET_FPMODE(Op, DAG);
   case ISD::EH_DWARF_CFA:
     return lowerEH_DWARF_CFA(Op, DAG);
   case ISD::VP_MERGE:
@@ -13998,6 +14007,55 @@ SDValue RISCVTargetLowering::lowerRESET_FPENV(SDValue Op,
                      EnvValue);
 }
 
+const uint64_t ModeMask64 = ~RISCVExceptFlags::ALL;
+const uint32_t ModeMask32 = ~RISCVExceptFlags::ALL;
+
+SDValue RISCVTargetLowering::lowerGET_FPMODE(SDValue Op,
+                                             SelectionDAG &DAG) const {
+  const MVT XLenVT = Subtarget.getXLenVT();
+  const uint64_t ModeMaskValue = Subtarget.is64Bit() ? ModeMask64 : ModeMask32;
+  SDLoc DL(Op);
+  SDValue Chain = Op->getOperand(0);
+  SDValue SysRegNo = DAG.getTargetConstant(RISCVSysReg::fcsr, DL, XLenVT);
+  SDValue ModeMask = DAG.getConstant(ModeMaskValue, DL, XLenVT);
+  SDVTList VTs = DAG.getVTList(XLenVT, MVT::Other);
+  SDValue Result = DAG.getNode(RISCVISD::READ_CSR, DL, VTs, Chain, SysRegNo);
+  Chain = Result.getValue(1);
+  Result = DAG.getNode(ISD::AND, DL, XLenVT, Result, ModeMask);
+  return DAG.getMergeValues({Result, Chain}, DL);
+}
+
+SDValue RISCVTargetLowering::lowerSET_FPMODE(SDValue Op,
+                                             SelectionDAG &DAG) const {
+  const MVT XLenVT = Subtarget.getXLenVT();
+  const uint64_t ModeMaskValue = Subtarget.is64Bit() ? ModeMask64 : ModeMask32;
+  SDLoc DL(Op);
+  SDValue Chain = Op->getOperand(0);
+  SDValue EnvValue = Op->getOperand(1);
+  SDValue SysRegNo = DAG.getTargetConstant(RISCVSysReg::fcsr, DL, XLenVT);
+  SDValue ModeMask = DAG.getConstant(ModeMaskValue, DL, XLenVT);
+
+  EnvValue = DAG.getNode(ISD::ZERO_EXTEND, DL, XLenVT, EnvValue);
+  EnvValue = DAG.getNode(ISD::AND, DL, XLenVT, EnvValue, ModeMask);
+  Chain = DAG.getNode(RISCVISD::CLEAR_CSR, DL, MVT::Other, Chain, SysRegNo,
+                      ModeMask);
+  return DAG.getNode(RISCVISD::SET_CSR, DL, MVT::Other, Chain, SysRegNo,
+                     EnvValue);
+}
+
+SDValue RISCVTargetLowering::lowerRESET_FPMODE(SDValue Op,
+                                               SelectionDAG &DAG) const {
+  const MVT XLenVT = Subtarget.getXLenVT();
+  const uint64_t ModeMaskValue = Subtarget.is64Bit() ? ModeMask64 : ModeMask32;
+  SDLoc DL(Op);
+  SDValue Chain = Op->getOperand(0);
+  SDValue SysRegNo = DAG.getTargetConstant(RISCVSysReg::fcsr, DL, XLenVT);
+  SDValue ModeMask = DAG.getConstant(ModeMaskValue, DL, XLenVT);
+
+  return DAG.getNode(RISCVISD::CLEAR_CSR, DL, MVT::Other, Chain, SysRegNo,
+                     ModeMask);
+}
+
 SDValue RISCVTargetLowering::lowerEH_DWARF_CFA(SDValue Op,
                                                SelectionDAG &DAG) const {
   MachineFunction &MF = DAG.getMachineFunction();
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.h b/llvm/lib/Target/RISCV/RISCVISelLowering.h
index 00e969056df7d..1963cfeca59c8 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.h
@@ -562,6 +562,9 @@ class RISCVTargetLowering : public TargetLowering {
   SDValue lowerGET_FPENV(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerSET_FPENV(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerRESET_FPENV(SDValue Op, SelectionDAG &DAG) const;
+  SDValue lowerGET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
+  SDValue lowerSET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
+  SDValue lowerRESET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
 
   SDValue lowerEH_DWARF_CFA(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerCTLZ_CTTZ_ZERO_UNDEF(SDValue Op, SelectionDAG &DAG) const;
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfo.td b/llvm/lib/Target/RISCV/RISCVInstrInfo.td
index f63531a0109b0..653607827282e 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfo.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfo.td
@@ -120,6 +120,20 @@ def riscv_swap_csr  : RVSDNode<"SWAP_CSR",
                                                     SDTCisInt<2>]>,
                                [SDNPHasChain]>;
 
+// Clear bits of CSR. The first operand is the address of the required CSR,
+// the second is the bitmask of cleared bits.
+def riscv_clear_csr  : RVSDNode<"CLEAR_CSR",
+                               SDTypeProfile<0, 2, [SDTCisInt<0>,
+                                                    SDTCisInt<1>]>,
+                               [SDNPHasChain]>;
+
+// Set bits of CSR. The first operand is the address of the required CSR,
+// the second is the bitmask of bits to set.
+def riscv_set_csr  : RVSDNode<"SET_CSR",
+                               SDTypeProfile<0, 2, [SDTCisInt<0>,
+                                                    SDTCisInt<1>]>,
+                               [SDNPHasChain]>;
+
 // A read of the 64-bit counter CSR on a 32-bit target (returns (Lo, Hi)).
 // It takes a chain operand and another two target constant operands (the
 // CSR numbers of the low and high parts of the counter).
@@ -2038,6 +2052,42 @@ class SwapSysRegImm<SysReg SR, list<Register> Regs>
   let Defs = Regs;
 }
 
+class ClearSysReg<SysReg SR, list<Register> Regs>
+  : Pseudo<(outs), (ins GPR:$val),
+           [(riscv_clear_csr (XLenVT SR.Encoding), (XLenVT GPR:$val))]>,
+    PseudoInstExpansion<(CSRRC X0, SR.Encoding, GPR:$val)> {
+  let hasSideEffects = 0;
+  let Uses = Regs;
+  let Defs = Regs;
+}
+
+class ClearSysRegImm<SysReg SR, list<Register> Regs>
+  : Pseudo<(outs), (ins uimm5:$val),
+           [(riscv_clear_csr (XLenVT SR.Encoding), uimm5:$val)]>,
+    PseudoInstExpansion<(CSRRCI X0, SR.Encoding, uimm5:$val)> {
+  let hasSideEffects = 0;
+  let Uses = Regs;
+  let Defs = Regs;
+}
+
+class SetSysReg<SysReg SR, list<Register> Regs>
+  : Pseudo<(outs), (ins GPR:$val),
+           [(riscv_set_csr (XLenVT SR.Encoding), (XLenVT GPR:$val))]>,
+    PseudoInstExpansion<(CSRRS X0, SR.Encoding, GPR:$val)> {
+  let hasSideEffects = 0;
+  let Uses = Regs;
+  let Defs = Regs;
+}
+
+class SetSysRegImm<SysReg SR, list<Register> Regs>
+  : Pseudo<(outs), (ins uimm5:$val),
+           [(riscv_set_csr (XLenVT SR.Encoding), uimm5:$val)]>,
+    PseudoInstExpansion<(CSRRSI X0, SR.Encoding, uimm5:$val)> {
+  let hasSideEffects = 0;
+  let Uses = Regs;
+  let Defs = Regs;
+}
+
 def ReadFRM : ReadSysReg<SysRegFRM, [FRM]>;
 let hasPostISelHook = 1 in {
 def WriteFRM : WriteSysReg<SysRegFRM, [FRM]>;
@@ -2056,6 +2106,10 @@ let hasPostISelHook = 1 in {
 def ReadFCSR : ReadSysReg<SysRegFCSR, [FRM, FFLAGS]>;
 def WriteFCSR : WriteSysReg<SysRegFCSR, [FRM, FFLAGS]>;
 def WriteFCSRImm : WriteSysRegImm<SysRegFCSR, [FRM, FFLAGS]>;
+def ClearFCSR : ClearSysReg<SysRegFCSR, [FRM, FFLAGS]>;
+def ClearFCSRImm : ClearSysRegImm<SysRegFCSR, [FRM, FFLAGS]>;
+def SetFCSR : SetSysReg<SysRegFCSR, [FRM, FFLAGS]>;
+def SetFCSRImm : SetSysRegImm<SysRegFCSR, [FRM, FFLAGS]>;
 }
 
 /// Other pseudo-instructions
diff --git a/llvm/test/CodeGen/RISCV/fpenv-xlen.ll b/llvm/test/CodeGen/RISCV/fpenv-xlen.ll
index 148186b21c125..c762f2ba41043 100644
--- a/llvm/test/CodeGen/RISCV/fpenv-xlen.ll
+++ b/llvm/test/CodeGen/RISCV/fpenv-xlen.ll
@@ -35,3 +35,38 @@ entry:
   call void @llvm.reset.fpenv()
   ret void
 }
+
+define iXLen @func_get_fpmode() {
+; CHECK-LABEL: func_get_fpmode:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    frcsr a0
+; CHECK-NEXT:    andi  a0, a0, -32
+; CHECK-NEXT:    ret
+entry:
+  %fpenv = call iXLen @llvm.get.fpmode.iXLen()
+  ret iXLen %fpenv
+}
+
+define void @func_set_fpmode(iXLen %fpmode) {
+; CHECK-LABEL: func_set_fpmode:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li    a1, -32
+; CHECK-NEXT:    csrc  fcsr, a1
+; CHECK-NEXT:    andi  a0, a0, -32
+; CHECK-NEXT:    csrs  fcsr, a0
+; CHECK-NEXT:    ret
+entry:
+  call void @llvm.set.fpmode.iXLen(iXLen %fpmode)
+  ret void
+}
+
+define void @func_reset_fpmode() {
+; CHECK-LABEL: func_reset_fpmode:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li   a0, -32
+; CHECK-NEXT:    csrc fcsr, a0
+; CHECK-NEXT:    ret
+entry:
+  call void @llvm.reset.fpmode()
+  ret void
+}

github-actions · 2025-07-14T06:34:34Z

✅ With the latest revision this PR passed the C/C++ code formatter.

topperc

LGTM

lenary · 2025-07-15T03:14:53Z

Can you point to where the glibc sources are masking out the exception bits?

spavloff · 2025-07-15T03:33:27Z

Here they are: https://github.com/bminor/glibc/blob/0263528f8dd60cf58976e2d516b7c9edb16ae6f8/sysdeps/riscv/rvf/fesetmode.c#L25-L28.

lenary · 2025-07-15T03:43:51Z

Ok, so the set masks, but fegetmode doesn't seem to: https://github.com/bminor/glibc/blob/0263528f8dd60cf58976e2d516b7c9edb16ae6f8/sysdeps/riscv/rvf/fegetmode.c (unless it's hidden in that macro? I'm not sure)

spavloff · 2025-07-15T05:19:09Z

Yes, you are right. The value returned by fegetmode contains the entire FP state not only control modes. Probably this is because the result of fegetmode is intended for use in calls of fesetmode only, which already cleans the status bits. In general these unrelated bits could complicate some operations but I dont't how if this really creates problems.

lenary

LGTM.

Please sort the clang-format before you land this.

It was added by me in 905bb5b, which committed PR llvm#148569.

It was added by me in 905bb5b, which committed PR #148569.

It was added by me in 905bb5b, which committed PR llvm/llvm-project#148569.

[RISCV][FPEnv] Lowering of fpmode intrinsics

d2eb527

The change implements custom lowering of `get_fpmode`, `set_fpmode` and `reset_fpmode` for RISCV target. The implementation is aligned with the functions `fegetmode` and `fesetmode` in GLIBC.

spavloff requested review from lenary, tclin914 and topperc July 14, 2025 06:29

spavloff self-assigned this Jul 14, 2025

llvmbot added the backend:RISC-V label Jul 14, 2025

topperc approved these changes Jul 14, 2025

View reviewed changes

Make implementation more close to GLIBC

f62c7d4

lenary approved these changes Jul 15, 2025

View reviewed changes

Fix clang-format errors

46a95d6

spavloff merged commit 905bb5b into llvm:main Jul 16, 2025
9 checks passed

spavloff added a commit to spavloff/llvm-project that referenced this pull request Jul 16, 2025

[RISCV][FPE] Remove unused variable

4b143bd

It was added by me in 905bb5b, which committed PR llvm#148569.

spavloff mentioned this pull request Jul 16, 2025

[RISCV][FPE] Remove unused variable #149054

Merged

spavloff added a commit that referenced this pull request Jul 16, 2025

[RISCV][FPE] Remove unused variable (#149054)

c71b92d

It was added by me in 905bb5b, which committed PR #148569.

llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Jul 16, 2025

Automerge: [RISCV][FPE] Remove unused variable (#149054)

886c5bf

It was added by me in 905bb5b, which committed PR llvm/llvm-project#148569.

This was referenced Jul 23, 2025

test abhinavgaba/llvm-project#2

Closed

Add dataFence plugin interface abhinavgaba/llvm-project#3

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[RISCV][FPEnv] Lowering of fpmode intrinsics #148569

[RISCV][FPEnv] Lowering of fpmode intrinsics #148569

Uh oh!

spavloff commented Jul 14, 2025

Uh oh!

llvmbot commented Jul 14, 2025

Uh oh!

github-actions bot commented Jul 14, 2025 •

edited

Loading

Uh oh!

topperc left a comment

Uh oh!

lenary commented Jul 15, 2025

Uh oh!

spavloff commented Jul 15, 2025

Uh oh!

lenary commented Jul 15, 2025

Uh oh!

spavloff commented Jul 15, 2025

Uh oh!

lenary left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[RISCV][FPEnv] Lowering of fpmode intrinsics #148569

[RISCV][FPEnv] Lowering of fpmode intrinsics #148569

Uh oh!

Conversation

spavloff commented Jul 14, 2025

Uh oh!

llvmbot commented Jul 14, 2025

Uh oh!

github-actions bot commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

topperc left a comment

Choose a reason for hiding this comment

Uh oh!

lenary commented Jul 15, 2025

Uh oh!

spavloff commented Jul 15, 2025

Uh oh!

lenary commented Jul 15, 2025

Uh oh!

spavloff commented Jul 15, 2025

Uh oh!

lenary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Jul 14, 2025 •

edited

Loading