error trying to connect: TLS handshake failed: cert verification failed

[ivan]

Hello,

Thank you very much for publishing this, as it may be invaluable for surveying the web without tripping up Cloudflare and other WAFs. I am trying to use it (my test repo), but it seems to be failing where reqwest would not:

test-reqwest-impersonate$ ./target/debug/test-reqwest-impersonate https://letsencrypt.org/
Error: error sending request for url (https://letsencrypt.org/): error trying to connect: TLS handshake failed: cert verification failed - certificate has expired [CERTIFICATE_VERIFY_FAILED]

Caused by:
    0: error trying to connect: TLS handshake failed: cert verification failed - certificate has expired [CERTIFICATE_VERIFY_FAILED]
    1: TLS handshake failed: cert verification failed - certificate has expired [CERTIFICATE_VERIFY_FAILED]
    2: [CERTIFICATE_VERIFY_FAILED]
    3: [CERTIFICATE_VERIFY_FAILED]

(Debian 11.4 with ca-certificates installed, rustc 1.63.0)

When used on NixOS 22.05 which has a different /etc/ssl, it seems to fail on all https:// sites with a different error:

test-reqwest-impersonate# ./target/debug/test-reqwest-impersonate https://www.google.com/
Error: error sending request for url (https://www.google.com/): error trying to connect: TLS handshake failed: cert verification failed - unable to get local issuer certificate [CERTIFICATE_VERIFY_FAILED]

Caused by:
    0: error trying to connect: TLS handshake failed: cert verification failed - unable to get local issuer certificate [CERTIFICATE_VERIFY_FAILED]
    1: TLS handshake failed: cert verification failed - unable to get local issuer certificate [CERTIFICATE_VERIFY_FAILED]
    2: [CERTIFICATE_VERIFY_FAILED]
    3: [CERTIFICATE_VERIFY_FAILED]

Details

# ls -lR /etc/ssl
/etc/ssl:
total 4,096
drwxr-xr-x 1 root root 64 2022-09-17 10:19 certs/
lrwxrwxrwx 1 root root 28 2022-09-17 10:19 trust-source -> /etc/static/ssl/trust-source/

/etc/ssl/certs:
total 8,192
lrwxrwxrwx 1 root root 35 2022-09-17 10:19 ca-bundle.crt -> /etc/static/ssl/certs/ca-bundle.crt*
lrwxrwxrwx 1 root root 41 2022-09-17 10:19 ca-certificates.crt -> /etc/static/ssl/certs/ca-certificates.crt*

# ls -l /etc/static/ssl/certs
total 8,192
lrwxrwxrwx 1 root root 87 1970-01-01 00:00 ca-bundle.crt -> /nix/store/sg1d11cjmc63iz6krh8mray50fkk0gqk-nss-cacert-3.80/etc/ssl/certs/ca-bundle.crt*
lrwxrwxrwx 1 root root 87 1970-01-01 00:00 ca-certificates.crt -> /nix/store/sg1d11cjmc63iz6krh8mray50fkk0gqk-nss-cacert-3.80/etc/ssl/certs/ca-bundle.crt*

# ls -l /nix/store/sg1d11cjmc63iz6krh8mray50fkk0gqk-nss-cacert-3.80/etc/ssl/certs
total 495,616
-r-xr-xr-x 3 root root 494,993 1970-01-01 00:00 ca-bundle.crt*

Is there something I should be doing to give reqwest-impersonate/BoringSSL a Chrome-like set of trusted certificate authorities? It would be very helpful if this were documented, or if it could trust all the certificates in webpki-roots or similar.

[0xFar5eer]

Did you try adding these patches to Cargo.toml?

[patch.crates-io]
hyper = { git = "https://github.com/4JX/hyper.git", branch = "0.14.x-patched" }
h2 = { git = "https://github.com/4JX/h2.git", branch = "maybefix" }

Then run cargo update

[ivan]

Thanks. Yeah, I had the patches in https://github.com/ludios/test-reqwest-impersonate/blob/master/Cargo.toml. I tried maybefix instead of imp just now and after cargo update and build I still see the same error with https://letsencrypt.org/ when testing on Debian.

(I also see the same error on Debian after removing that openssl = that shouldn't be there.)

[4JX]

It would be very helpful if this were documented, or if it could trust all the certificates in webpki-roots or similar.

It works oob on my arch machine, but being able to use the webpki roots should be easy enough to add. That being said I'm currently unable to use one of my hands so it'll take some time before I can tackle this.

For now you could try overriding the tls client used like

reqwest-impersonate/src/browser/chrome/ver/v104.rs

Line 29 in fa5287b

fn create_ssl_connector() -> SslConnectorBuilder {

with use_boring_tls and work from there.

[n1ght-hunter]

for anyone still have this issue. this is a fix
https://github.com/sfackler/rust-openssl/pull/535/files
if i get time i will try integrating this into the lib