Unable to load crypto library with otp version 26.2.5 when built erlang with fips-enabled

[HarinadhD]

I am trying to upgrade erlang version from 25.1.2 to 26.2.5 , erlang 26.X require to build rabbirmq-server 3.13.X

But got an error "Unable to load crypto library" when executing crypto:version(). in erl shell with OTP version 26.2.5

This problem is noticed with 26.2.5 , same issue was not observed with erlang 25.1.2
openssl installed version in both the cases 3.0.13

OTP 26.2.5:

root@vm [ws ]# erl
Erlang/OTP 26 [erts-14.2.5] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]
Eshell V14.2.5 (press Ctrl+G to abort, type help(). for help)
1> crypto:version().
=ERROR REPORT==== 11-Jun-2024::12:45:57.772369 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."
=WARNING REPORT==== 11-Jun-2024::12:45:57.777686 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}
** exception error: undefined function crypto:version/0
2

### NOTE:
The above issue got resolved after installing openssl-fips-provider package , but OTP 25.1.2 was working without openssl-fips-provider package.

OTP 25.1.2:

root@photon4 [ ~ ]# erl
Erlang/OTP 25 [erts-13.1.2] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]
Eshell V13.1.2 (abort with ^G)
1> crypto:version().
"5.1.2"
2>

To Reproduce
install openssl-devel package
build OTP with fips enabled
execute crypto:version().

root@vm[/ws/ ]# erl
Erlang/OTP 26 [erts-14.2.5] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]

Eshell V14.2.5 (press Ctrl+G to abort, type help(). for help)
1> crypto:version().
=ERROR REPORT==== 11-Jun-2024::12:45:57.772369 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

=WARNING REPORT==== 11-Jun-2024::12:45:57.777686 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}

** exception error: undefined function crypto:version/0

Expected behavior
erlang should load crypto library without installing openssl-fips-provider package.

Affected versions
verified Only with OTP 26.2.5

[sshedi]

Following patch fixes the issue. (Patch is generated from 26.2.5 version of erlang).

---
diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c
@@ -224,7 +224,9 @@ static int initialize(ErlNifEnv* env, ERL_NIF_TERM load_info)
 #ifdef HAS_3_0_API
     prov_cnt = 0;
 # ifdef FIPS_SUPPORT
-    if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "fips"))) return __LINE__;
+    if (FIPS_MODE()) {
+        if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "fips"))) return __LINE__;
+    }
 #endif
     if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "default"))) return __LINE__;
     if ((prov_cnt<MAX_NUM_PROVIDERS) && !(prov[prov_cnt++] = OSSL_PROVIDER_load(NULL, "base"))) return __LINE__;
--

This patch makes erlang load "fips.so" library only when openssl fips is enabled in the system.
Distros like Fedora have fips.so available with openssl-libs package.
In some of the distros where fips.so is not available by default (PhotonOS, Ubuntu for example), crypto module doesn't load at all because fips.so is not present, hence the error. Loading fips.so only when fips enabled seems to be the right way to fix this issue.

Can someone from erlang upstream please take this patch? I can't send a PR due to some restrictions at my end. Sorry about that.

[nickva]

I think I see the same issue when building with --enable-fips with Erlang 26 on Debian Bookworm.

~/otp_src_26.2.5.2$ openssl version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)

./bin/cerl 
Erlang/OTP 26 [erts-14.2.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V14.2.5.2 (press Ctrl+G to abort, type help(). for help)
1> crypto:strong_rand_bytes(1).
=ERROR REPORT==== 11-Jul-2024::19:02:53.521863 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

=WARNING REPORT==== 11-Jul-2024::19:02:53.525380 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}

** exception error: undefined function crypto:strong_rand_bytes/1
2> 

The build string is:

./configure 
   --without-javac --without-wx --without-odbc  --without-debugger 
   --without-observer --without-et  --without-cosEvent --without-cosEventDomain 
   --without-cosFileTransfer  --without-cosNotification --without-cosProperty 
   --without-cosTime --without-cosTransactions --without-orber
   --enable-fips

If I drop --enable-fips it seems to work.

Was wondering what is the intent behind enable-fips option? Maybe we're using it incorrectly? To me seems it should be adding the ability to enable FIPS after the VM is compiled using -crypto fips_mode true|false but by default for false the system should still be able to call crypto:strong_rand_bytes/1?