Explicitly manage TLS/SSL CA paths for remote I/O

[kingcrimsontianyu]

Background

libcurl have two path parameters related to the certificate authority (CA):

• CURLOPT_CAINFO, which specifies the CA bundle file path.
• CURLOPT_CAPATH, which specifies the directory of individual CA certificates with hash-based naming.

The default paths are determined at compile-time, which can cause issues if the Linux distributions where libcurl is built and run are different (e.g. on Rocky Linux vs Ubuntu as in our CI vs our lab system), and the certificates files are likely at different locations. This problem has been observed in KvikIO's wheel distribution, where HTTPS would fail with the message:

error setting certificate verify locations: CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: /etc/ssl/certs

This PR

This PR addresses this problem. The certificate path is now explicitly searched for in the following order. The compile-time parameters, if any, are still used but treated with lowest priority.

• CA bundle file: Check env vars CURL_CA_BUNDLE, and SSL_CERT_FILE
• CA directory: Check env vars SSL_CERT_DIR
• CA bundle file: Search a set of distribution-specific locations for accessible bundle
• CA directory: Search a set of distribution-specific locations for accessible directory
• CA bundle file: Check if the compile-time path is given and accessible
• CA directory: Check if the compile-time parameter is given and accessible

Depends on #819 for the use of static structured binding which is only available in C++ >=20

Closes #711

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[kingcrimsontianyu]

/ok to test 9cb4137

[kingcrimsontianyu]

Verification

A libcurl intentionally built with bad CA bundle file and CA directory paths was used, and a toy code that uses the search method in this PR was tested under various conditions:

• No env vars specified (i.e. searching standard system locations)
• Correct CURL_CA_BUNDLE
• Correct SSL_CERT_FILE
• Correct SSL_CERT_DIR

The toy program is able to locate and use the correct certificate in all the conditions above.

[kingcrimsontianyu]

static structured binding is not supported in C++17, and this helped to identify an oversight to be fixed by #819.

[kingcrimsontianyu]

/ok to test 1903a7b

[madsbk]

Nice work

[kingcrimsontianyu]

/merge