benchmarks: provide a digest benchmark which uses various implementations

[hannesm]

I wanted to easily compare different implementations, and started with hashing / digests.

In this PR, first the bench/speed.exe has a split-off helper.ml (no surprises) ;)

Then, I developed bench/digests.exe which tests nocrypto (https://github.com/hannesm/ocaml-nocrypto/tree/compiles - 0.5.4-2 plus allowing newer cstruct releases to avoid lots of changes in my switch), mirage-crypto (commit 6fa26bd), digestif 1.1.4, ocaml-sha 1.15.4, cryptokit 1.19, OCaml 4.14.1 and openssl 3.0.12

All tests are run on my i7-5600U CPU @ 2.60GHz using FreeBSD-14, OCaml 4.14.1 (happy to see numbers on other architectures / processors):

Resuls:

MD5

size	nocrypto	mirage-cry	digestif	stdlib	cryptokit	openssl
16	51.682	63.650	78.144	* 107.970*	90.273	59.678
64	148.387	172.181	195.855	* 230.107*	205.056	169.489
256	313.978	332.245	357.565	* 385.844*	361.911	364.497
1024	433.760	445.244	455.512	464.499	454.579	* 513.199*
8192	486.566	489.529	491.404	492.987	490.263	* 587.799*

First column in bytes, all others in MB/s (1MB = 1024 * 1024)

SHA1

size	nocrypto	mirage-cry	digestif	ocaml-sha	cryptokit	openssl
16	43.659	54.273	65.635	* 74.074*	52.385	56.167
64	122.322	142.711	161.865	* 176.676*	116.828	164.608
256	260.846	279.374	298.677	315.168	199.342	* 398.186*
1024	364.759	375.541	385.924	393.510	245.253	* 614.528*
8192	416.407	415.313	420.857	430.265	261.766	* 736.483*

First column in bytes, all others in MB/s (1MB = 1024 * 1024)

SHA256

size	nocrypto	mirage-cry	digestif	ocaml-sha	cryptokit	openssl
16	26.480	31.774	34.243	36.839	34.678	* 38.340*
64	69.416	77.409	82.467	81.558	76.503	* 99.244*
256	131.065	139.294	142.987	137.850	129.533	* 219.244*
1024	171.909	173.974	175.883	163.997	157.563	* 305.108*
8192	188.027	188.982	188.704	180.706	168.621	* 341.120*

First column in bytes, all others in MB/s (1MB = 1024 * 1024)

SHA512

size	nocrypto	mirage-cry	digestif	ocaml-sha	cryptokit	openssl
16	23.323	25.765	25.326	23.991	25.605	* 30.038*
64	92.719	103.203	99.902	104.559	110.152	* 120.589*
256	165.876	167.433	158.540	156.484	167.372	* 244.509*
1024	243.956	237.645	235.108	230.279	234.339	* 387.885*
8192	282.661	269.457	270.386	253.435	266.440	* 487.495*

First column in bytes, all others in MB/s (1MB = 1024 * 1024)

[hannesm]

A table about X25519:

X25519

	mirage-cryp	callipyge	rfc7748	hacl_x25519	openssl
1	15598.650	294.174	795.586	15579.182	23468.535

Note, the hacl_star package doesn't build on FreeBSD, thus it is not present above. And I patched the hacl_x25519 to use Cstruct.length (and thus a recent cstruct).

[dinosaure]

On AMD Ryzen 9 7950X (4.5 GHz):

SHA1

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	79.460	113.065	160.568	188.077	61.715	213.591
64	239.724	330.320	412.439	438.737	131.987	665.491
256	590.134	704.546	792.350	789.971	224.407	1545.151
1024	932.007	994.807	1024.803	973.503	264.234	2322.035
8192	1100.019	1110.046	1136.005	1040.819	282.564	2680.815

SHA256

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	56.059	63.366	85.261	95.510	86.249	203.169
64	150.514	176.322	191.899	211.134	188.266	632.563
256	304.330	328.814	340.239	364.059	316.101	1460.627
1024	412.551	421.731	419.593	445.085	382.492	2173.267
8192	456.972	456.577	451.432	476.765	413.332	2551.422

SHA512

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	48.668	54.247	70.859	75.767	68.872	91.953
64	191.719	213.347	280.200	299.890	295.584	368.394
256	373.461	395.111	440.088	460.356	445.883	678.256
1024	587.753	606.742	620.436	657.357	614.305	1001.971
8192	705.427	714.307	707.548	750.250	692.852	1171.290

[palainp]

According to /dev/cpuinfo my vcpus are i7-1365U running at 2688.006Hz, and the laptop was wallpluged when benchmarking.
Here are my results but it's hard to understand the different behavior, MD5 results, as well as nocrypto, mirage-crypto, digestif and cruptokit, are quite similar to @hannesm, but openssl significantly outperforms on the SHA1 and 256 suites:

MD5

size	nocrypto	mirage-cryp	digestif	stdlib	cryptokit	openssl
16	46.065	60.942	77.967	* 87.102*	78.350	70.518
64	132.995	159.852	191.998	* 198.498*	186.323	169.594
256	278.367	309.204	339.583	* 345.299*	334.919	330.738
1024	396.499	413.291	420.163	424.478	420.437	* 447.872*
8192	441.823	450.029	451.587	454.835	454.077	* 499.858*

SHA1

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	52.550	54.621	69.878	69.416	33.155	* 78.529*
64	117.249	141.327	165.596	155.706	70.543	* 285.192*
256	249.779	275.922	300.234	265.001	117.018	* 793.100*
1024	353.590	370.176	376.592	321.868	140.868	* 1426.052*
8192	399.865	404.476	405.558	343.124	149.753	* 1859.459*

SHA256

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	27.268	32.401	36.046	39.772	36.702	* 76.896*
64	68.729	76.204	80.636	88.043	78.884	* 271.642*
256	130.334	136.683	139.198	151.284	131.043	* 721.106*
1024	168.792	172.998	171.136	183.977	158.319	* 1229.280*
8192	186.274	187.169	183.687	196.986	168.498	* 1547.014*

SHA512

size	nocrypto	mirage-cryp	digestif	ocaml-sha	cryptokit	openssl
16	21.449	23.902	26.861	* 30.971*	30.004	26.553
64	84.842	96.364	107.193	* 124.293*	120.362	108.819
256	160.448	168.171	174.807	189.678	182.127	* 217.158*
1024	240.432	250.623	248.739	266.476	250.449	* 347.056*
8192	288.568	289.504	287.682	302.394	282.529	* 419.953*

[hannesm]

hmm, was there some trick to add some private opam package with the dependencies to get this compiling in CI -- while "private" means that a dune-publish does not accidentally publish that opam package? or is that just something I dreamed of - and we can either live with "ocaml-ci" not finding the dependencies and failing, or add another opam package where we need to be careful to never publish it in a release?

[hannesm]

since digest algorithms have been removed from mirage-crypto, this is no longer very relevant. developing benchmarks for the various ocaml libraries and openssl as baseline is still on our list, and will hopefully land soon in this repository :)