Support certificates hot reload

[Horcrux7]

Because certificates from Let's Encrypt has a lifetime of 90 days only a automatic renew is required without a server restart. This should be handle from Jetty server self.

A generic implementation of the ACME protocol should be the best solution. There are Java client libraries for the ACME protocol.

The minimum is a dynamic replacement of the current used certificate.

[sbordet]

See also: https://dev.eclipse.org/mhonarc/lists/jetty-users/msg07260.html

[Horcrux7]

@sbordet Do you means this is the recommended solution also for the future?

[sbordet]

@Horcrux7, no I just meant to gather together references to the same problem.
We will analyze all data, proposed solutions, etc. and come up with the right solution.

[fhossain]

I have a similar situation with Jetty using short lived certificates. In my case I am not using ACME directly as it is an internal private CA for service to service mutual authentication. I have control over the client portion and can deal with short lived certificates but the Jetty server portion needs a solution. I have dynamic keystores that update the key/certificate and can make a callback call when they are ready to use. What I am missing is a mechanism for the callback to notify Jetty listeners to reinitialize the SSLContext.

[sbordet]

@Horcrux7 for Jetty implementing the ACME protocol I think it's overkill. We are not in that business, and Let's Encrypt has already provided tools that people can use alongside with Jetty to implement the certificate renewal/revocation.

The idea is that you will install Jetty and a certificate management agent (as defined by Let's Encrypt); the latter will take care of the ACME part and generate the new keystore.

We will improve Jetty's SslContextFactory with a way to be reconfigured and reloaded so that the the certificate management agent, once received the new certificate and generated the new keystore, can reconfigure Jetty's SslContextFactory with the new keystore path, password and other configuration options, and then reload the new configuration.

This will allow new connections to use the new key material, while older connections will continue to use the old key material until closed.
The ServerConnector will not be stopped, and will atomically switch to the new key material.

I think this will cover @fhossain use case, as well as the original poster on the mailing list (thread referenced above), and with a bit of glue code, also @Horcrux7 use case.

[sbordet]

Introduced SslContextFactory.reload(Consumer<SslContextFactory>) to allow reconfiguration and reload of the SslContextFactory.

There is a gotcha, though.

The JDK by default supports TLS session resumption.
A Java client connects to a TLS server with the pair (host, port). When the first connection to this pair completes successfully and the connection is then closed, the TLS session ID is cached by the JDK.
Subsequent connections make use of the cached TLS session ID and will be able to perform the "fast" TLS handshake via session resumption.

A new connection established by the client will use session resumption; the client will also check that the certificate received by the server is equal to the certificate that it cached previously.
This check may fail when the certificate on the server has changed as part of a reload.

Turns out that the JDK chokes if this check fails, and throws a SSLHandshakeException.

On the other hand, browsers are perfectly capable of handling this case by falling back to the full TLS handshake instead of the fast one that uses session resumption.

@gregw has already verified that the JDK offers no API to disable the caching of TLS sessions (see #519).
The closest solution is to use [SSLSocket|SSLEngine].getSession().invalidate() but that may not apply to all cases.

In summary:

• It is possible to change the server keystore without restarting or suspending the ServerConnector via the new SslContextFactory.reload(...) method.
• Browser clients should work fine (we have tested Firefox).
• JDK clients must be prepared to get an SSLHandshakeException if that happens, and somehow try to invalidate the TLS session to avoid session resumption.

I would appreciate if people can test this new feature and report their feedback.

Thanks !

[Horcrux7]

@sbordet Sounds you should open a bug report/feature request for Java by Oracle.

[fhossain]

@sbordet thanks for addressing it. I have tested it out and it works like a charm. I am using this inside SpringFramework and have wired it to use both Keystore from disk as well as use SslStoreProvider() which allows me to have in memory version of Keystore. They both work great. I tested with standard HttpsURLConnection but could not reproduce the problem you mentioned. I ran a loop with System.setProperty("http.keepAlive", "false"); on the client with -Dssl.debug=true -Djavax.net.debug=ssl:handshake. I can see repeated calls with logs like %% Try resuming [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384] from port 61259. In each round the port number seems to change but not the Session number. However, after a SslContextFactory.reload(...) I can see the client session number going up by one. The exception seems to be handled transparently from the client side without any exception propagated to the client. Let me know if you think I made any mistakes in my testing. Otherwise, everything LGTM 👍

[sbordet]

@fhossain, glad it works for you.

It is strange that the TLS session number goes up by one: it should be a random-like number every time it is generated by the server.

Can you share your code that uses HttpsURLConnection ?

Other testers ?

[fhossain]

Here is the gut of my test.

public void goRun(String url, int count, SSLSocketFactory sslSocketFactory) throws Exception {
    System.setProperty("http.keepAlive", "false");
    long start = System.currentTimeMillis();
    for (int i = 0; i < count; i++) {
        URL myurl = new URL(url);
        HttpsURLConnection con = (HttpsURLConnection) myurl.openConnection();
        //con.setRequestProperty("Connection", "close");
        con.setSSLSocketFactory(sslSocketFactory);

        InputStream ins = con.getInputStream();
        InputStreamReader isr = new InputStreamReader(ins);
        BufferedReader in = new BufferedReader(isr);
        String inputLine;
        while ((inputLine = in.readLine()) != null) {
            if (count == 1) {
                System.out.print(inputLine);
            }
        }
        in.close();
        Thread.sleep(5000);
    }
    System.out.println("Total time: " + (System.currentTimeMillis() - start) + " ms");
}

[sbordet]

What happens if you uncomment

con.setRequestProperty("Connection", "close");

?

[fhossain]

It behaves the same as using System.setProperty("http.keepAlive", "false");. In both cases the session re-establish is successful until the certificate is updated. Upon which the session count goes up by one. I think the actual session number might be random but the debug printout sanitized it for readability.
However, there is one change needed in the code. The Thread.sleep() has to be < 5 seconds. Otherwise, the keep-alive timer kicks in and closes connection and you can't see the different between System.setProperty("http.keepAlive", "false")/con.setRequestProperty("Connection", "close"); and commenting out both and just using the default. The system default is keep-alive on. In that case the connection is kept open and certificates change is not detected as expected.

[johngmyers]

Could the server invalidate all TLS session IDs whenever the certificate changes?

[joakime]

@johngmyers there's no API to either purge the TLS session ID cache, set it to 0 entries, or to access the list of sessions and invalidate each.

If you know of a way, let us know.

[amandeep33]

@sbordet wanted to know if we can have the same functionality in 9.2.x release also as version 9.3.x requires JDK 8.

[sbordet]

No, we won't backport this to 9.2.x because Jetty 9.2.x is End-Of-Life.

[ramtaneja]

@sbordet , I couldn't get this work for me,
IN my case I just need same keystore to be reloaded after I replace it manually
so I just used a dummy consumer
sslContextFactory.reload(scf -> {});

I could see log in at INFO level of SslContextFactory for change in X509 for my SslContextFactory, but browser wasn't able to detect change in certs, is there anything else needs to be done

[sbordet]

I could see log in at INFO level of SslContextFactory for change in X509 for my SslContextFactory, but browser wasn't able to detect change in certs, is there anything else needs to be done

So the server did the right thing, but the browser did not.
Perhaps the browser is using existing connections that won't pick up the new configuration (only new connections will).
Have you tried restarting the browser?
Since the server did the right thing, it's not a Jetty issue.

[ramtaneja]

well, actually its working perfectly fine...!
That was my bad, I was running two servers on different ports, reloading on one of them and was browsing the other one...

[nagarjunabattula]

Hi all, cc: @ramtaneja @sbordet ,
We are using the Jetty server with version 9.4.48.v20220622 embedding and core Java 17 with REST code.
We are looking to hot reload the certificates.
For this, we are getting the latest server.jks and trust.jks copying to our cert folder and using
SslContextFactory.reload(...) .

When I am checking using the curl
curl --insecure -vvI https://localhost:/ 2>&1 | awk 'BEGIN { cert=0 } /^* SSL connection/ { cert=1 } /^*/ { if (cert) print }'
I am getting the latest cert info.
But in the browser, I am still seeing the old cert only, tried to signedout and use a private window, and checked all browsers other than Chrome still has no use.
And I have tried to use the wrong certificate still it's working as is without failing.
Could you please help me?

Please find the code and suggest me if any.

   HttpConfiguration http_config = new HttpConfiguration();
   CertificateUtil.downloadCertificates(
   CertificateUtil.copyCertificates(
    http_config.setSecureScheme("https");
    http_config.setSecurePort(JETTY_PORT);
    SslContextFactory sslContextFactory = new SslContextFactory.Server();
    sslContextFactory.setKeyStorePath(SSL_SERVER_KEY_STROKE_PATH);
    sslContextFactory.setCertAlias("server");
    sslContextFactory.setKeyStorePassword(SSL_KEY_STROKE_KEY);
    sslContextFactory.setTrustStorePath(SSL_TRUST_KEY_STROKE_PATH);
    sslContextFactory.setTrustStorePassword(SSL_KEY_STROKE_KEY);
    HttpConfiguration https_config = new HttpConfiguration(http_config);
    https_config.addCustomizer(new SecureRequestCustomizer());
    ServerConnector https = new ServerConnector(server, new SslConnectionFactory(sslContextFactory, HttpVersion.HTTP_1_1.asString()), new HttpConnectionFactory(https_config));
    https.setPort(JETTY_PORT);
    server.setConnectors(new Connector[]{https});

    Timer timer = new Timer();
    timer.schedule(new TimerTask() {
        @Override
        public void run() {
            try {
                logger.info("SSL timer scheduler start");
                    logger.info(sslContextFactory.toString());
                    logger.info("ssl context "+sslContextFactory.getSslContext());
                    // gets latest server.jks and trust.jks and copies to the path
//    CertificateUtil.downloadCertificates();
  //  CertificateUtil.copyCertificates();
                        logger.info("Started certificates reload");
                    sslContextFactory.reload(scf -> logger.info("Certificates reloaded successfully"));
                logger.info("Calling reload end");
            } catch (Exception e) {
                logger.severe("Exception while reloading certs " + e);
            }
        }
    }, 0, CERT_RELOAD_TIME);
    logger.info("Jetty SSL successfully configured..");
} catch (Exception e){
    logger.severe("Error configuring Jetty SSL.."+e);
    throw e;
}
}
...
server.start();
server.join();

[joakime]

We are using the Jetty server with version 9.4.48.v20220622 standalone and core Java 17 with REST code.
We are looking to hot reload the certificates.
For this, we are getting the latest server.jks and trust.jks copying to our cert folder and using
SslContextFactory.reload(...) .

Note: Jetty 9.x is at End of Community Support as of June 2022

• EoCS (End of Community Support) for Jetty 9 - June 2022 #7958

You should be using Jetty 10, Jetty 11, or Jetty 12 at this point in time.

Also note that Jetty 10 and Jetty 11 has started it's Sunsetting and will be at End of Community Support in January 2025

• EoCS (End of Community Support) for Jetty 10 / Jetty 11 - January 2024 #10485

[nagarjunabattula]

Hi @joakime ,
Ok, I will upgrade the jetty to 10. x or very latest also.
Could you please suggest the solution for the issue

[Horcrux7]

@nagarjunabattula The browses cache the displayed certificate infos in standard mode. The certificate what you see is not the used certificate. Use the recovery incognito mode to check the current certificate or use Wireshark to see the current certificate.

[sbordet]

Also note that existing connections are not closed, so they will continue to use the old certificate.

Only new connections established after the reload will use the new certificate.

Depending on the HTTP protocol version used, this may take a long time (typically longer in HTTP/2 than in HTTP/1).

[nagarjunabattula]

@sbordet
Am using HTTP/1.1.
could you please suggest how to close based on the code I have written?
Or you are talking about the browser connection or the rest API connection

[nagarjunabattula]

@Horcrux7
Even if we use private window then also it wont show new cerificate?

[Horcrux7]

Yes, I means the private mode.

[sbordet]

It is not a good idea to close the connections, as requests may be in progress.

They will eventually close, and the new ones will use the new certificate.

[nagarjunabattula]

Only new connections established after the reload will use the new certificate.

yeah tested but its still showing old cer
Any flaw in my code?

[nagarjunabattula]

It is not a good idea to close the connections, as requests may be in progress.

They will eventually close, and the new ones will use the new certificate.

then what could be the alternative? Was there any flaw in my code?
I found in logs that its throwing SSL handshake exception ater reload
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target\n\tat java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)\n\tat java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:378)\n\tat java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)\n\tat java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:316)\n\tat java.base

[nagarjunabattula]

@ramtaneja
Can you also look at the code once?
What i need to do to avoid this error

[sbordet]

The code you posted is not enough to say anything, sorry.
No idea what your CertificateUtil does, if it has bugs, I have no idea why you are using a Timer at all, whether the new certificate is a good one, whether it's under the same alias in the KeyStore, etc. etc. etc. etc.

We have tests in SslContextFactoryReloadTest that prove that the mechanism work.
You can use that class as a starting point for your code, perhaps.

If you can replicate the issue in a small project with very few or no other dependencies, then open a new issue, attach the reproducer project so that we can try it out, and we will continue the discussion there.

[joakime]

If you can replicate the issue in a small project with very few or no other dependencies, then open a new issue, attach the reproducer project so that we can try it out, and we will continue the discussion there.

Only if the example small project is on a supported version of Jetty.

[nagarjunabattula]

The code you posted is not enough to say anything, sorry. No idea what your CertificateUtil does, if it has bugs, I have no idea why you are using a Timer at all, whether the new certificate is a good one, whether it's under the same alias in the KeyStore, etc. etc. etc. etc.

We have tests in SslContextFactoryReloadTest that prove that the mechanism work. You can use that class as a starting point for your code, perhaps.

If you can replicate the issue in a small project with very few or no other dependencies, then open a new issue, attach the reproducer project so that we can try it out, and we will continue the discussion there.

CertificateUtil is a very simple one, downloadcerts will download certs using an API, and copycerts will just copy from the download location to the certs location. As it has some confidential APIs so only I didn't share them.
And I am using a timer for testing certs which i am placing by manual copying by commenting CertificateUtil.copycerts call

[nagarjunabattula]

@sbordet
saw this if we get SSL handshake exception
JDK clients must be prepared to get an SSLHandshakeException if that happens, and somehow try to invalidate the TLS session to avoid session resumption.

Do I need to use [SSLSocket|SSLEngine].getSession().invalidate()?
May I know for org.eclipse.jetty.server.HttpConfiguration and org.eclipse.jetty.util.ssl.SslContextFactory which way i should do connection close or invalidate the session?
Or what ever mentioned is related to the client-side?

[nagarjunabattula]

@sbordet
Whatever I have shared the code you can tell that it should work fine right? I mean especially the reload part ignore other classes, though the jetty version is 9.4.48.v20220622

[nagarjunabattula]

@sbordet
We have a few internal server calls. There we are getting an SSLHandshake exception.
For that, we are using the code as follows

Client client = ClientBuilder.newClient();
WebTarget target = client.target(baseUri);
Invocation.Builder builder = target.request(MediaType.APPLICATION_JSON);

Now when we change this to the following code we are not getting SSLHandshake exception.
Hope I am in the right direction.
Solution:

private static String XXX(String baseUri) throws Exception {
        SSLContext sslContext = SSLContexts.createDefault();
        SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext);
        Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("https", sslSocketFactory)
                .register("http", PlainConnectionSocketFactory.getSocketFactory())
                .build();

        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
        logger.info("got sslSocketFactory");
        // Create the HttpClient with the connection manager
        CloseableHttpClient httpClient = HttpClients.custom()
                .setSSLSocketFactory(sslSocketFactory)
                .setConnectionManager(cm)
                .build();

         HttpGet httpGet = new HttpGet(baseUri);
         CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
         HttpEntity entity = httpResponse.getEntity();
         return EntityUtils.toString(entity);
     }

[joakime]

Go ask the Apache HttpClient folks how their client handles certificate renewals for active connections.

Since curl, Jetty HttpClient, and the new Java java.net.htt.HttpClient all work correctly this is no longer a Jetty issue or question, but a question for your chosen Http Client library.

[nagarjunabattula]

ok.

[nagarjunabattula]

One more thing, now when I am reloading and without reloading the internal API call getting succeeded with out error and i saw the following line now

However, in non-embedded situations (i.e. Jetty started using java start.jar), it requires the creation of a custom module and a jar file that will include the fairly straightforward code that will trigger the reload.

As I am using Jetty standalone and I am not using anything mentioned, could you please let me know what is custom module and what jar I need to use here?

[joakime]

In Jetty 12 (the only supported version of Jetty right now) the ss-reload module already exists, no custom module needed, just use it.

[nagarjunabattula]

Hi @joakime ,
I have migrated to jetty 11 now.
If you see my code jetty embedding previously mentioned initially i have loaded proper certificates (#918 (comment)),
sslContextFactory.reload(scf -> logger.info("Certificates reloaded successfully"));
after this, I printed the certificate and I am getting the expected one.
So as part of the testing I have taken other domain certificates and reloaded them, in this case, it has to fail with the exception "Unable to communicate securely with peer: requested domain name does not match the server's certificate".

But I am not getting this instead call is getting succeeded.

and client-side code is the following.

private static String getResponse(String baseUri) throws Exception {
       System.setProperty("http.keepAlive", "false");
       URL baseUrl = new URL(baseUri);
       HttpsURLConnection httpsURLConnection = (HttpsURLConnection) baseUrl.openConnection();
//        httpsURLConnection.setRequestProperty("Connection", "close");
       httpsURLConnection.setRequestMethod("GET");
       httpsURLConnection.setRequestProperty("X-TENANT-ID", "system");
       httpsURLConnection.setRequestProperty(AUTHORIZATION, RequestUtils.getEncodedDpServicesBasicAuthCreds());
       httpsURLConnection.setSSLSocketFactory(SSLUtils.getSSLContext().getSocketFactory());
       printCertificate(httpsURLConnection);
       StringBuilder result = new StringBuilder();
       InputStream inputStream = httpsURLConnection.getInputStream();

       InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
       BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
       printCertificate(httpsURLConnection);
       String inputLine;
       while ((inputLine = bufferedReader.readLine()) != null) {
           result.append(inputLine);
       }
       bufferedReader.close();
       printCertificate(httpsURLConnection);
       String responseStr = result.toString();
       Thread.sleep(5000);
       printCertificate(httpsURLConnection);
       return responseStr;
   }
private static void printCertificate(HttpsURLConnection httpsURLConnection) {
       try {
           java.security.cert.Certificate[] certificates = httpsURLConnection.getServerCertificates();
           for (Certificate certificate : certificates) {
               logger.info("Http httpsURLConnection server certificates start");
               logger.info(certificate.toString());
               logger.info("Http httpsURLConnection server certificates end");
           }
       } catch (Exception exception) {
           logger.error("exception while getting server cert", exception);
       }
       try {
           java.security.cert.Certificate[] certificates = httpsURLConnection.getLocalCertificates();
           for (Certificate certificate : certificates) {
               logger.info("Http httpsURLConnection Local certificates start");
               logger.info(certificate.toString());
               logger.info("Http httpsURLConnection Local certificates end");
           }
       } catch (Exception exception) {
           logger.error("exception while getting Local cert", exception);
       }
   }

please find sshutil

import oracle.biapps.cxo.logging.CxoLoggerFactory;
import org.slf4j.Logger;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.FileInputStream;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.cert.Certificate;
public class SSLUtils {

   public static final String SSL_SERVER_KEY_STROKE_PATH = "/server.jks";
   public static final String SSL_TRUST_KEY_STROKE_PATH = "/trust.jks";
   public static final String SSL_KEY_STROKE_KEY = "changeit";

   public static SSLContext getSSLContext() {
       char[] keyStoreWord = SSL_KEY_STROKE_KEY.toCharArray();
       SSLContext sslContext = null;
       try {
           KeyStore keyStore = getKeyStore(SSL_SERVER_KEY_STROKE_PATH, keyStoreWord);
           KeyStore trustStore = getKeyStore(SSL_TRUST_KEY_STROKE_PATH, keyStoreWord);

           KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
           keyManagerFactory.init(keyStore, keyStoreWord);

           TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
           trustManagerFactory.init(trustStore);

           sslContext = SSLContext.getInstance("TLS");

           sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
           SSLContext.setDefault(sslContext);
           LOG.info("Getting  SSL Context completed and getting cert");
           try {
               Certificate certificate = keyStore.getCertificate("server");
               LOG.info(certificate.toString());
           } catch (Exception e) {
               LOG.error("Failed to see certificates: " + e);
           }
           return sslContext;

       } catch (Exception exception) {
           LOG.error("Exception while getting SSL context: " + exception);
       }
       return sslContext;
   }

   public static KeyStore getKeyStore(String file, char[] password) throws Exception {
       KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
       try (InputStream in = new FileInputStream(file)) {
           keyStore.load(in, password);
       }

       return keyStore;
   }
}

Here in getResponse call when I am printing certificate info but i am getting an error while getting server or client certs.

So, I am unable to figure out whether the issue is with server reload or client-side

1. if the issue is with the server side
   after sslContextFactory.reload(scf -> logger.info("Certificates reloaded successfully"));
   Do I need to set the connectors again after reloading (please refer to 1st posted code I have written.)
   or
2. If the issue is with client-side is there any issue with SSlUtils.java or the getResponse method
   Please help me out

FYI
In the 1st post, the code is to start the jetty server and the timer will check every 30 minutes whether the new certificate is available if available it just reloads with the following snippet
sslContextFactory.reload(scf -> logger.info("Certificates reloaded successfully"));

Also on the client side when I am using printCertificate to print the certs I am always getting exceptions is this expected how should we know which cert the httpclientconnection using?

[nagarjunabattula]

cert-sh.zip

FYI While starting I am using CertificateUtil.downloadCertificates(); which executes the above sh file and then
after CertificateUtil.copyCertificates(); will copy the .jks files into refered certificates folder.
@joakime @sbordet

[joakime]

Note: Jetty 12 is the only supported version of Jetty at the moment.

And use the KeyStoreScanner from Jetty 12's org.eclipse.jetty.util.ssl.KeyStoreScanner (I don't see that in your code examples)

        keyStoreScanner = new KeyStoreScanner(sslContextFactory);
        keyStoreScanner.setScanInterval(0); // how frequently to scan for changes (in seconds)
        server.addBean(keyStoreScanner);

Don't use java.net.HttpsURLConnection as it has both a SSL cache and a connection cache that you cannot work around reliably.

Since you are on Java 11 now, use java.net.http.HttpClient
You can control the SSL/TLS behaviors with more reliability that way.
Keep in mind that this client in the JVM also supports HTTP/2.

Always use Connection: close for this testing, no matter the HTTP Client you use.
You don't want to reuse a connection that's already established with an old certificate.

Also, keep in mind that you are now in the world of up to date SSL/TLS specs.
That means you MUST use a host name with your client when issuing the remote connection (SSL/TLS does not work with IP Literals, either IPv4 or IPv6, or any local domain, or localhost declaration)
The certificates MUST match that host name used as well.
Otherwise you'll need to disabled host name verification.

[nagarjunabattula]

Ok let me change.
BTW I am using java17 not java11.
Initial code is with HttpClient only as SSLHandshake has come, i have used java.net.HttpsURLConnection.

[nagarjunabattula]

Disable hostname verification will accept the untrusted certificate right, No it should still pass with the proper certificate only.
if it fails that is expected only.

and
keyStoreScanner.setScanInterval(0); will scan for multiple reload ? it seems 0 will stop multiple time scans right? or does it have to be 0 when we load both trust.jks and server.jks? As part of copy certificates, i am copying new trust.jks and server.jks files to the respective directories, whenever the certificate expires and we will use sslContextFactory.reload...

@joakime