OpenSSL v3 Perf Regressions (from v1.1)

[nibanks]

Describe the bug

As mentioned in #3388 OpenSSL v3 has some perf regressions compared to v1.1.

Affected OS

• Windows
• Linux
• macOS
• Other (specify below)

Additional OS information

No response

MsQuic version

main

Steps taken to reproduce bug

Build with OpenSSL v3.
Run all tests (functional and performance).

Expected behavior

No regressions compared to v1.1

Actual outcome

Regressions.

Additional details

No response

[nibanks]

Updating this bug to reflect just the perf regressions, since the functional issues have been fixed (thanks @wfurt!)

[wfurt]

adding specific numbers for visibility

v1.1.1

Test Run Took 00:00:13.9332980
Run 1: 5627775 kbps
Test Run Took 00:00:14.0494522
Run 2: 5535505 kbps
Test Run Took 00:00:13.8022312
Run 3: 5621339 kbps
Test Run Took 00:00:14.0703712
Run 4: 5354296 kbps
Test Run Took 00:00:14.0590394
Run 5: 5633485 kbps
Median: 5621339 kbps

v3.0

Test Run Took 00:00:14.1823326
Run 1: 4650012 kbps
Test Run Took 00:00:13.9744334
Run 2: 4620731 kbps
Test Run Took 00:00:14.1512972
Run 3: 4618331 kbps
Test Run Took 00:00:13.9970742
Run 4: 4698704 kbps
Test Run Took 00:00:14.2107039
Run 5: 4655014 kbps
Median: 4650012 kbps

Linux HPS Perf v1.1.1

Test Run Took 00:00:10.9550682
Run 1: 9764 HPS
Test Run Took 00:00:10.9642312
Run 2: 9990 HPS
Test Run Took 00:00:11.0954863
Run 3: 10086 HPS
Test Run Took 00:00:10.9084721
Run 4: 10276 HPS
Test Run Took 00:00:10.9453742
Run 5: 10166 HPS
Median: 10086 HPS

Linux HPS Perf v3.0

Test Run Took 00:00:11.1963015
Run 1: 1891 HPS
Test Run Took 00:00:11.4757571
Run 2: 1949 HPS
Test Run Took 00:00:11.3739960
Run 3: 1879 HPS
Test Run Took 00:00:11.3007463
Run 4: 1888 HPS
Test Run Took 00:00:11.4470352
Run 5: 1978 HPS
Median: 1891 HPS

[nibanks]

@wfurt so what are we doing to do about this for the 2.2 release? Just live with it for now?

[wfurt]

Probably. There are some hints from OpenSSL community but I don't have much time because of other priorities.
We would like to run HTTP/3 benchmarks to see impact on .NET but that did not happen yet AFAIK.
cc: @ManickaP @CarnaViire

[ManickaP]

We should get the OpenSSL 3.0 support into the next release as we need to support that for .NET 8.0 - OpenSSL 3 is becoming the default for the new versions of most distros.
We also need to get it in in time for us to properly test with it (stress, some baking time in CI etc.)

If there will be any OpenSSL 3 perf fixes that will require msquic being rebuild, we should be ready to service that into whatever release gets used with .NET 8.0.

I don't know what are your time frames for the next releases, but we'll need this many weeks, preferable 2+ months before feature complete which is in the middle of July.

cc @karelz

[nibanks]

Resolving this as generally 3.1 fixed most issues.