[✨ Triage] dotnet/runtime#119641 by rzikm - ML-DSA certificates cannot be used with QUIC and HTTP/3

[MihuBot]

Triage for dotnet/runtime#119641.
Repo filter: All networking issues.
MihuBot version: 706ff6.
Ping MihaZupan for any issues.

This is a test triage report generated by AI, aimed at helping the triage team quickly identify past issues/PRs that may be related.
Take any conclusions with a large grain of salt.

Tool logs

dotnet/runtime#119641: ML-DSA certificates cannot be used with QUIC and HTTP/3 by rzikm
[Tool] Searching for ML-DSA certificate, QUIC, HTTP/3, MsQuic system libssl, quictls OpenSSL fork, TLS certificate support (IncludeOpen=True, IncludeClosed=True, IncludeIssues=True, IncludePullRequests=True, Repository=dotnet/runtime)
[Tool] Found 129 issues, 351 comments, 40 returned results (12208 ms)

Here are the most relevant related issues and discussions for issue #119641 ('ML-DSA certificates cannot be used with QUIC and HTTP/3'):

---

1. PR #119265 (September 2025) - Use forTls parameter to enable TLS-compatible certificate key types while maintaining platform compatibility

• Summary: This PR introduced a forTls parameter to certificate generation logic, allowing tests to select key types compatible with TLS and platform support. The discussion revealed that:
  • On Windows, SChannel does not support ML-DSA certificates for TLS.
  • On Linux, with OpenSSL 3.5.0+, ML-DSA is supported for HTTP/1.1 and HTTP/2 (which use system libssl), but not for QUIC/HTTP/3 (which uses a statically linked, older quictls fork).
  • The PR disables SLH-DSA for Linux due to lack of OpenSSL provider support in TLS-SIGALG.
  • The issue with ML-DSA and QUIC/HTTP/3 is confirmed to be due to the statically linked, pre-3.5.0 quictls in MsQuic.
  • There is an expectation that when MsQuic is built to use system libssl (OpenSSL 3.5.0+), this limitation will be resolved.

---

2. Issue #114911 (April 2025) - [QUIC] Test MsQuic with OpenSSL 3.5

• Summary: This issue tracks the adoption of MsQuic builds using official OpenSSL 3.5 for QUIC. The discussion notes that:
  • MsQuic is moving toward using system OpenSSL 3.5, which is required for PQC (post-quantum cryptography) algorithms like ML-DSA.
  • There is ongoing work to test and integrate MsQuic with OpenSSL 3.5 in CI, which is a prerequisite for resolving the ML-DSA/QUIC/HTTP/3 compatibility gap.

---

3. Issue #81801 (February 2023, closed May 2023) - Quic should support OpenSSL 3.x

• Summary: This issue tracked the work to support OpenSSL 3.x in MsQuic and .NET. It was resolved after MsQuic and .NET added support for OpenSSL 3.x, but the discussion predates OpenSSL 3.5 and the introduction of ML-DSA. It establishes the groundwork for future PQC support in QUIC.

---

4. PR #69603 (May 2022) - Support optional client certificates for MsQuicConnection

• Summary: This PR and its discussion are relevant for understanding how MsQuic and .NET handle certificates for QUIC. It highlights the dependency on MsQuic's OpenSSL layer and the need for up-to-date MsQuic builds for new crypto features.

---

5. Issue #113244 (March 2025, closed) - Ship Openssl MsQuic dll and utilize it in environments where schannel doesn't support tls1.3

• Summary: This issue discusses the challenges of shipping OpenSSL-linked MsQuic binaries due to security and update concerns. It confirms that .NET will not ship OpenSSL MsQuic DLLs, and that supporting new OpenSSL features (like ML-DSA) in QUIC/HTTP/3 depends on MsQuic being built against the system OpenSSL.

---

6. Issue #101602 (April 2024) - [QUIC] Eliminate peer X509 certificate parsing on Linux

• Summary: This issue discusses the technical details of MsQuic's linkage to OpenSSL and the implications for certificate handling. It notes that recent MsQuic versions can use system libcrypto, which is necessary for supporting new certificate types like ML-DSA in QUIC.

---

7. Issue #55544 (July 2021) - [QUIC] Certificate Validation with OpenSSL on Windows

• Summary: This older issue discusses the challenges of supporting different certificate formats and validation paths in MsQuic/OpenSSL/SChannel, which is relevant background for the current ML-DSA situation.

---

Key Conclusions

• Root Cause: ML-DSA certificates are not supported for QUIC/HTTP/3 on Linux because MsQuic (as used by .NET) is currently built with a statically linked, pre-3.5.0 fork of OpenSSL (quictls), which does not recognize ML-DSA for TLS. HTTP/1.1 and HTTP/2 use the system libssl, which may be OpenSSL 3.5.0+ and does support ML-DSA.
• Resolution Path: This limitation will be resolved when MsQuic is built to use the system OpenSSL 3.5.0+ (dynamic linking), which exposes the necessary APIs and algorithm support for ML-DSA in QUIC/HTTP/3.
• Tracking: The transition to system OpenSSL 3.5.0+ in MsQuic is being tracked in issues like #114911 and is a known work item for the .NET networking team.
• Workarounds: There is no workaround in .NET until MsQuic is updated and distributed with system OpenSSL 3.5.0+ support.

---

If you need more details on any specific PR or issue, let me know!