Bug when using an HTTPS Proxy and disabling proxy certificate verification

[ramsestom]

I tried to configure okhttp (v 4.10.0) to make a GET request to an url through an https proxy that use a self-signed certificate.
I saw these issues: #6561 and #3787 that are quite similar exept that they do not address the case where you want to disable the certificate verification during the SSL handshake with the https proxy (because in my case, the proxy is using a self-signed certificate, that may change quite frequently. So requiering to have it downloaded in a client keystore is not a practical option).

Here is the code that I currently have (which is basically the same as the one from @lpuglia in #6561 with just a TrustManager that will accept all server's certificates added) :

String desturl = "https://api64.ipify.org/?format=json"; //"https://myip.com" //"http://neverssl.com" //"http://silverfinebeautifulsecret.neverssl.com/online/"

OkHttpClient.Builder clientb = new OkHttpClient.Builder().proxy(new java.net.Proxy(java.net.Proxy.Type.HTTP, new InetSocketAddress("shoutouttomyex.icu", 443))) ;

// Create a trust manager that does not validate certificate chains
final TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
     @Override public void checkClientTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException { System.out.println("checkClientTrusted"); return;}
     @Override public void checkServerTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException { System.out.println("checkServerTrusted"); return;}
     @Override public java.security.cert.X509Certificate[] getAcceptedIssuers() { System.out.println("getAcceptedIssuers");  return new java.security.cert.X509Certificate[]{}; }
} };
		  
final SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
final SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
		  		  
clientb.socketFactory(new DelegatingSocketFactory(sslSocketFactory));
clientb.hostnameVerifier(new HostnameVerifier() {
     @Override public boolean verify(String hostname, SSLSession session) { System.out.println("Verify "+hostname); return true; }
});

okhttp3.Request.Builder rbuilder = new Request.Builder().url(desturl).get();
 
Response response = clientb.build().newCall(rbuilder.build()).execute();
System.out.println("Answer: "+response.body().string());

(Note: I don't need any Authenticator as my https proxy is an opened proxy to which you can connect without any username and password)

the problem is that when requesting an https url with this code, I systematically get an answer with a 404 error. Here is the output that I have:

checkServerTrusted
getAcceptedIssuers
java.io.IOException: Unexpected response code for CONNECT: 404
	at [email protected]/okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.kt:483)
	at [email protected]/okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.kt:262)
	at [email protected]/okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:201)
	at [email protected]/okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
	at [email protected]/okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
	at [email protected]/okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
	at [email protected]/okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
	at [email protected]/okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
	at [email protected]/okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at [email protected]/okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at [email protected]/okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at [email protected]/okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at [email protected]/okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at [email protected]/okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at [email protected]/okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at [email protected]/okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at [email protected]/okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)

and if I request an http url (like http://silverfinebeautifulsecret.neverssl.com/online/) I don't have any error but I get an empty answer (which is also a bug as I should receive the page body). Here is the output with an http destination url:

checkServerTrusted
getAcceptedIssuers
Answer: 

The same chain of request (requesting an url through my https proxy with proxy's certificate verification disabled) is working just fine with curl.
Here is the output that I get with this curl command : curl -x https://shoutouttomyex.icu:443 https://api64.ipify.org/?format=json --proxy-insecure -v :

*   Trying 65.49.2.58:443...
* Connected to shoutouttomyex.icu (65.49.2.58) port 443 (#0)
* ALPN: offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted http/1.1
* Proxy certificate:
*  subject: CN=shoutouttomyex.icu
*  start date: Mar 21 03:28:06 2023 GMT
*  expire date: Jun 19 03:28:05 2023 GMT
*  issuer: OU=generated by Avast Antivirus for SSL/TLS scanning; O=Avast Web/Mail Shield; CN=Avast Web/Mail Shield Root
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* allocate connect buffer
* Establish HTTP proxy tunnel to api64.ipify.org:443
> CONNECT api64.ipify.org:443 HTTP/1.1
> Host: api64.ipify.org:443
> User-Agent: curl/8.0.1
> Proxy-Connection: Keep-Alive
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.0 200 OK
< X-RS: rs71
<
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: C:\Development\utils\curl-8.0.1_4-win64-mingw\bin\curl-ca-bundle.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.ipify.org
*  start date: Feb  7 00:00:00 2023 GMT
*  expire date: Feb 18 23:59:59 2024 GMT
*  subjectAltName: host "api64.ipify.org" matched cert's "*.ipify.org"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /?format=json]
* h2h3 [:scheme: https]
* h2h3 [:authority: api64.ipify.org]
* h2h3 [user-agent: curl/8.0.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x2bbe2549f10)
> GET /?format=json HTTP/2
> Host: api64.ipify.org
> user-agent: curl/8.0.1
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200
< content-type: application/json
< date: Mon, 27 Mar 2023 20:52:58 GMT
< vary: Origin
< content-length: 21
<
{"ip":"64.62.219.63"}* Connection #0 to host shoutouttomyex.icu left intact

[yschimke]

@ramsestom apologies for the delay, interesting problem.

This seems to be an SNI related issue. Change the following

  clientb.socketFactory(object : DelegatingSocketFactory(sslSocketFactory) {
    override fun configureSocket(socket: Socket): Socket {
      val sslSocket = socket as SSLSocket

      val parameters = sslSocket.sslParameters
      val sni = parameters.serverNames
      parameters.serverNames = mutableListOf<SNIServerName>(SNIHostName("shoutouttomyex.icu"))
      sslSocket.sslParameters = parameters

      return socket
    }
  })