PIP-337 changes

pip/pip-333.md

@@ -0,0 +1,387 @@
+# PIP-337: SSL Factory Plugin to customize SSLContext/SSLEngine generation
+
+# Background knowledge
+Apache Pulsar supports TLS encrypted communication between the clients and servers. The TLS encryption setup requires
+loading the TLS certificates and its respective passwords to generate the SSL Context. Pulsar supports loading these 
+certificates and passwords via the filesystem. It supports both Java based Keystores/Truststores and TLS information in 
+".crt", ".pem" & ".key" formats. This information is refreshed based on a configurable interval.
+
+Apache Pulsar internally uses 3 different frameworks for connection management:
+
+- Netty: Connection management for Pulsar server and client that understands Pulsar binary protocol.
+- Jetty: HTTP Server creation for Pulsar Admin and websocket. Jetty Client is used by proxy for admin client calls.
+- AsyncHttpClient: HTTP Client creation for Admin client and HTTP Lookup
+
+Each of the above frameworks supports customizing the generation of the SSL Context and SSL Engine. Currently, Pulsar 
+uses these features to feed the SSL Context via its internal security tools after loading the file based certificates.
+One of the issues of using these features is that pulsar tries to bootstrap the SSL Context in multiple ways to suit
+each framework and file type.
+
+```mermaid
+flowchart TB
+    Proxy.DirectProxyHandler --> NettyClientSslContextRefresher
+    Proxy.DirectProxyHandler --> NettySSLContextAutoRefreshBuilder
+    Proxy.AdminProxyHandler --> KeyStoreSSLContext
+    Proxy.AdminProxyHandler --> SecurityUtility
+    Proxy.ServiceChannelInitializer --> NettySSLContextAutoRefreshBuilder
+    Proxy.ServiceChannelInitializer --> NettyServerSslContextBuilder
+    Broker.PulsarChannelInitializer --> NettyServerSslContextBuilder
+    Broker.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+    Client.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+    Client.PulsarChannelInitializer --> SecurityUtility
+    Broker.WebService --> JettySSlContextFactory
+    Proxy.WebServer --> JettySSlContextFactory
+    PulsarAdmin --> AsyncHttpConnector
+    AsyncHttpConnector --> KeyStoreSSLContext
+    AsyncHttpConnector --> SecurityUtility
+    JettySSlContextFactory --> NetSslContextBuilder
+    JettySSlContextFactory --> DefaultSslContextBuilder
+    NettyClientSslContextRefresher -.-> SslContextAutoRefreshBuilder
+    NettySSLContextAutoRefreshBuilder -.-> SslContextAutoRefreshBuilder
+    NettyServerSslContextBuilder -.-> SslContextAutoRefreshBuilder
+    NetSslContextBuilder -.-> SslContextAutoRefreshBuilder
+    DefaultSslContextBuilder -.-> SslContextAutoRefreshBuilder
+    Client.HttpLookup.HttpClient --> KeyStoreSSLContext
+    Client.HttpLookup.HttpClient --> SecurityUtility
+    SecurityUtility -.-> KeyManagerProxy
+    SecurityUtility -.-> TrustManagerProxy
+```
+The above diagram is an example of the complexity of the TLS encryption setup within Pulsar. The above diagram only
+contains the basic components of Pulsar excluding Websockets, Functions, etc.
+
+Pulsar uses 2 base classes to load the TLS information.
+
+- `SecurityUtility`: It loads files of type ".crt", ".pem" and ".key" and converts it into SSL Context. This SSL Context
+can be of type `io.netty.handler.ssl.SslContext` or `javax.net.ssl.SSLContext` based on the caller. Security Utility 
+can be used to create SSL Context that internally has KeyManager and Trustmanager proxies that load cert changes 
+dynamically.
+- `KeyStoreSSLContext`: It loads files of type Java Keystore/Truststore and converts it into SSL Context. This SSL
+Context will be of type `javax.net.ssl.SSLContext`. This is always used to create the SSL Engine.
+
+Each of the above classes are either directly used by Pulsar Clients or used via implementations of the abstract class 
+`SslContextAutoRefreshBuilder`.
+
+- `SslContextAutoRefreshBuilder` - This abstract class is used to refresh certificates at a configurable interval. It 
+internally provides a public API to return the SSL Context.
+
+There are several implementations of the above abstract class to suit the needs of each of the framework and the
+respective TLS certificate files:
+
+- `NettyClientSslContextRefresher` - It internally creates the `io.netty.handler.ssl.SslContext` using the ".crt", 
+".pem" and ".key" files for the proxy client.
+- `NettySSLContextAutoRefreshBuilder` - It internally creates the `KeyStoreSSLContext` using the Java Keystores.
+- `NettyServerSslContextBuilder` - It internally creates the `io.netty.handler.ssl.SslContext` using the ".crt",
+  ".pem" and ".key" files for the server.
+- `NetSslContextBuilder` - It internally creates the `javax.net.ssl.SSLContext` using the Java Keystores for the web 
+server.
+- `DefaultSslContextBuilder` - It internally creates the `javax.net.ssl.SSLContext` using the ".crt", ".pem" and ".key"
+files for the web server.
+
+# Motivation
+Apache Pulsar's TLS encryption configuration is not pluggable. It only supports file-based certificates. This makes 
+Pulsar difficult to adopt for organizations that require loading TLS certificates by other mechanisms.
+
+# Goals
+The purpose of this PIP is to introduce the following:
+
+- Provide a mechanism to plugin a custom SSL Factory that can generate SSL Context and SSL Engine. 
+- Simplify the Pulsar code base to universally use `javax.net.ssl.SSLContext` and reduce the amount of code required to
+build and configure the SSL context taking into consideration backwards compatibility.
+
+## In Scope
+
+- Creation of a new abstract class `SSLFactory` that can generate a SSL Context, Client SSL Engine and Server SSL 
+Engine. It should support refreshing the certificates at a configurable interval.
+- Creation of a default implementation of `SSLFactory` that supports loading the SSL Context and SSL Engine via
+file-based certificates. Internally it will use the SecurityUtility and KeyStoreSSLContext.
+- Modify the Pulsar Components to support the `SSLFactory` instead of the SslContextAutoRefreshBuilder, SecurityUtility
+and KeyStoreSSLContext.
+- Remove the SslContextAutoRefreshBuilder and all its implementations.
+
+# High Level Design
+```mermaid
+flowchart TB
+    Proxy.DirectProxyHandler --> Sslfactory
+    Proxy.AdminProxyHandler --> Sslfactory
+    Proxy.ServiceChannelInitializer --> Sslfactory
+    Broker.PulsarChannelInitializer --> Sslfactory
+    Client.PulsarChannelInitializer --> Sslfactory
+    Broker.WebService --> JettySSlContextFactory
+    Proxy.WebServer --> JettySSlContextFactory
+    PulsarAdmin --> AsyncHttpConnector
+    AsyncHttpConnector --> Sslfactory
+    JettySSlContextFactory --> Sslfactory
+    Client.HttpLookup.HttpClient --> Sslfactory
+    Sslfactory -.-> DefaultSslFactory
+    Sslfactory -.-> CustomSslFactory
+```
+
+# Detailed Design
+
+## Design and Implementation Details
+
+### Pulsar Common Changes
+
+A new abstract class called `SslFactory` that provides public methods to create a SSL Context, Client SSL Engine and
+Server SSL Engine. The SSL Context class returned will be of type `javax.net.ssl.SSLContext`. This class will refresh 
+the SSL Context at the configured interval. This refresh behavior is the same as the 
+current `SslContextAutoRefreshBuilder`.
+
+```java
+public abstract class SslFactory {
+    public SslFactory(long certRefreshInSec);
+    public abstract SSLEngine getClientSslEngine(String peerHost, int peerPort);
+    public abstract SSLEngine getServerSslEngine() throws GeneralSecurityException, IOException;
+    public abstract boolean needsUpdate();
+    public abstract void update() throws Exception;
+    public abstract void configure(String sslProviderString, 
+                                   Set<String> ciphers, 
+                                   Set<String> protocols, 
+                                   boolean allowInsecureConnection, 
+                                   boolean requireTrustedClientCertOnConnect, 
+                                   AuthenticationDataProvider authData, 
+                                   String tlsCustomParams);
+    // Implementation similar to SslContextAutoRefreshBuilder class which will be threadsafe
+    public SSLContext getInternalSslContext();
+}
+```
+
+A default implementation of the above SSLFactory class called `DefaultSslFactory` that will generate the SSL Context 
+and SSL Engines using File-based Certificates. It will be able to support both Java keystores and "pem/crt/key" files.
+
+```java
+public class DefaultSslFactory extends SslFactory {
+  public DefaultSslFactory(long certRefreshInSec);
+  public void configure(String sslProviderString, String keyStoreTypeString, String keyStore, String keyStorePassword,
+                        String trustStoreTypeString, String trustStore, String trustStorePassword,
+                        Set<String> ciphers, Set<String> protocols, String trustCertsFilePath,
+                        String certificateFilePath, String keyFilePath, boolean allowInsecureConnection,
+                        boolean requireTrustedClientCertOnConnect, AuthenticationDataProvider authData,
+                        boolean isKeyStoreEnabled);
+  public SSLEngine getClientSslEngine(String peerHost, int peerPort);
+  public SSLEngine getServerSslEngine();
+  public boolean needsUpdate();
+  public void update() throws Exception;
+  //The below will be a noop
+  public void configure(String sslProviderString,
+                        Set<String> ciphers,
+                        Set<String> protocols,
+                        boolean allowInsecureConnection,
+                        boolean requireTrustedClientCertOnConnect,
+                        AuthenticationDataProvider authData,
+                        String tlsCustomParams);
+}
+```
+
+### Pulsar Commmon Changes
+
+6 new configurations will need to be added into the Configurations like `ServiceConfiguration`, 
+`ClientConfigurationData`, `ProxyConfiguration`, etc. All of the below will be optional. It will use the default values
+to match the current behavior of Pulsar.
+
+- `sslFactoryPlugin`: SSL Factory Plugin class to provide SSLEngine and SSLContext objects.
+The default class used is `DefaultSslFactory`.
+- `sslFactoryPluginParams`: SSL Factory plugin configuration parameters. It will be of type string. It can be parsed by
+the plugin at its discretion.
+- `sslFactorySslContextInitTimeoutInMs`: SSL Context initialization timeout in ms for the first time it is initialized 
+by the SSLFactory
+
+The below configs will be applicable only to the Pulsar Server components like Broker and Proxy:
+- `brokerClientSslFactoryPlugin`: SSL Factory Plugin class used by internal client to provide SSLEngine and SSLContext 
+objects. The default class used is `DefaultSslFactory`.
+- `brokerClientSslFactoryPluginParams`: SSL Factory plugin configuration parameters used by internal client. It can be 
+parsed by the plugin at its discretion.
+- `brokerClientsslFactorySslContextInitTimeoutInMs`: SSL Context initialization timeout in ms for the first time 
+it is initialized by the SSLFactory in the internal client
+
+`JettySslContextFactory` class will need to be changed to internally use the `SslFactory` class to generate the 
+SslContext.
+
+### SslFactory Usage across Pulsar Netty based server components
+
+Example Changes in broker's `PulsarChannelInitializer` to initialize the SslFactory:
+```java
+sslFactory = (SslFactory) Class.forName(config.getSslFactoryPlugin())
+                    .getDeclaredConstructor(Long.TYPE, LONG.TYPE)
+                    .newInstance(config.getTlsCertRefreshCheckDurationSec(),
+                                 config.getSslFactorySslContextInitTimeoutInMs);
+if (this.sslFactory instanceof DefaultSslFactory) {
+    ((DefaultSslFactory) this.sslFactory).configure(config.getTlsProvider(),
+                                                    config.getTlsKeyStoreType(),
+                                                    config.getTlsKeyStore(),
+                                                    config.getTlsKeyStorePassword(),
+                                                    config.getTlsTrustStoreType(),
+                                                    config.getTlsTrustStore(),
+                                                    config.getTlsTrustStorePassword(),
+                                                    config.getTlsCiphers(),
+                                                    config.getTlsProtocols(),
+                                                    config.getTlsTrustCertsFilePath(),
+                                                    config.getTlsCertificateFilePath(),
+                                                    config.getTlsKeyFilePath(),
+                                                    config.isTlsAllowInsecureConnection(),
+                                                    config.isTlsRequireTrustedClientCertOnConnect(),
+                                                    null,
+                                                    config.isTlsEnabledWithKeyStore());
+} else {
+    this.sslFactory.configure(config.getTlsProvider(),
+                              config.getTlsCiphers(),
+                              config.getTlsProtocols(),
+                              config.isTlsAllowInsecureConnection(),
+                              config.isTlsRequireTrustedClientCertOnConnect(),
+                              null,
+                              config.getSslFactoryPluginParams());
+}
+```
+
+Example changes in `PulsarChannelInitializer` to `initChannel(SocketChannel ch)`:
+```java
+ch.pipeline().addLast(TLS_HANDLER, new SslHandler(this.sslFactory.getServerSslEngine()));
+```
+
+The above changes is similar in all the Pulsar Server components that internally utilize Netty.
+
+### SslFactory Usage across Pulsar Netty based Client components
+
+Example Changes in Client's `PulsarChannelInitializer` to initialize the SslFactory:
+```java
+this.sslFactory = (SslFactory) Class.forName(conf.getSslFactoryPlugin())
+                    .getDeclaredConstructor(Long.TYPE, LONG.TYPE)
+                    .newInstance(conf.getAutoCertRefreshSeconds(), 
+                                 conf.getSslFactorySslContextInitTimeoutInMs);
+if (this.sslFactory instanceof DefaultSslFactory) {
+    ((DefaultSslFactory) this.sslFactory).configure(
+            conf.getSslProvider(),
+            conf.getTlsKeyStoreType(),
+            conf.getTlsKeyStorePath(),
+            conf.getTlsKeyStorePassword(),
+            conf.getTlsTrustStoreType(),
+            conf.getTlsTrustStorePath(),
+            conf.getTlsTrustStorePassword(),
+            conf.getTlsCiphers(),
+            conf.getTlsProtocols(),
+            conf.getTlsTrustCertsFilePath(),
+            conf.getTlsCertificateFilePath(),
+            conf.getTlsKeyFilePath(),
+            conf.isTlsAllowInsecureConnection(),
+            false,
+            conf.getAuthentication().getAuthData(),
+            conf.isUseKeyStoreTls());
+} else {
+        this.sslFactory.configure(
+            conf.getSslProvider(),
+            conf.getTlsCiphers(),
+            conf.getTlsProtocols(),
+            conf.isTlsAllowInsecureConnection(),
+            false,
+            conf.getAuthentication().getAuthData(),
+            conf.getSslFactoryPluginParams());
+}
+```
+
+Example changes in `PulsarChannelInitializer` to `initChannel(SocketChannel ch)`:
+```java
+SslHandler handler = new SslHandler(sslFactory
+                        .getClientSslEngine(sniHost.getHostName(), sniHost.getPort()));
+ch.pipeline().addFirst(TLS_HANDLER, handler);
+```
+
+The above changes is similar in all the Pulsar client components that internally utilize Netty.
+
+### SslFactory Usage across Pulsar Jetty Based Server Components
+
+The initialization of the SslFactory is similar to the [Netty Server initialization.](#sslfactory-usage-across-pulsar-jetty-based-server-components)
+
+The usage of the SSLFactory requires changes in the `JettySslContextFactory`. It will internally accept `SslFactory` as 
+an input and utilize it to create the SSL Context.
+```java
+public class JettySslContextFactory {
+  private static class Server extends SslContextFactory.Server {
+    private final SslFactory sslFactory;
+
+    // New 
+    public Server(String sslProviderString, SslFactory sslFactory,
+                  boolean requireTrustedClientCertOnConnect, Set<String> ciphers, Set<String> protocols) {
+      // Current implementation
+    }
+
+    @Override
+    public SSLContext getSslContext() {
+      return this.sslFactory.getInternalSslContext();
+    }
+  }
+}
+```
+
+The above `JettySslContextFactory` will be used to create the SSL Context within the Jetty Server. This pattern will be 
+common across all Web Server created using Jetty within Pulsar.
+
+### SslFactory Usage across Pulsar AsyncHttpClient based Client Components
+
+The initialization of the SslFactory is similar to the [Netty Server initialization.](#sslfactory-usage-across-pulsar-jetty-based-server-components)
+
+The usage of the SSLFactory requires changes in the `AsyncHttpConnector`. It will internally initialize the `SslFactory`
+and pass it to a new custom `PulsarHttpAsyncSslEngineFactory` that implements `org.asynchttpclient.SSLEngineFactory`.
+This new custom class will incorporate the features of the existing `WithSNISslEngineFactory` and `JsseSslEngineFactory`
+and replace it.
+
+```java
+public class PulsarHttpAsyncSslEngineFactory extends DefaultSslEngineFactory {
+
+    private final SslFactory sslFactory;
+    private final String host;
+
+    public PulsarHttpAsyncSslEngineFactory(SslFactory sslFactory, String host) {
+        this.sslFactory = sslFactory;
+        this.host = host;
+    }
+
+    @Override
+    protected void configureSslEngine(SSLEngine sslEngine, AsyncHttpClientConfig config) {
+        super.configureSslEngine(sslEngine, config);
+        if (StringUtils.isNotBlank(host)) {
+            SSLParameters parameters = sslEngine.getSSLParameters();
+            parameters.setServerNames(Collections.singletonList(new SNIHostName(host)));
+            sslEngine.setSSLParameters(parameters);
+        }
+    }
+
+    @Override
+    public SSLEngine newSslEngine(AsyncHttpClientConfig config, String peerHost, int peerPort) {
+        SSLContext sslContext = this.sslFactory.getInternalSslContext();
+        SSLEngine sslEngine = config.isDisableHttpsEndpointIdentificationAlgorithm()
+                ? sslContext.createSSLEngine() :
+                sslContext.createSSLEngine(domain(peerHost), peerPort);
+        configureSslEngine(sslEngine, config);
+        return  sslEngine;
+    }
+
+}
+```
+
+The above `PulsarHttpAsyncSslEngineFactory` will be passed to the DefaultAsyncHttpClientConfig.Builder while creating 
+the DefaultAsyncHttpClient. This pattern will be common across all HTTP Clients using AsyncHttpClient within Pulsar.
+
+## Public-facing Changes
+
+### Configuration
+
+Same as [Broker Common Changes](#pulsar-commmon-changes)
+
+### CLI
+CLI tools like `PulsarClientTool` and `PulsarAdminTool` will need to be modified to support the new configurations.
+
+# Backward & Forward Compatibility
+
+## Revert
+Rolling back to the previous version of Pulsar will revert to the previous behavior.
+
+## Upgrade
+Upgrading to the version containing the `SslFactory` will not cause any behavior change. The `SslFactory` for the 
+server, client and brokerclient will default to using the `DefaultSslFactory` which will read the TLS certificates
+via the file system.
+
+The Pulsar system will use the custom plugin behavior only if the `sslFactoryPlugin` configuration is set.
+
+# Links
+
+POC Changes:  https://github.com/apache/pulsar/compare/master...Apurva007:pulsar:security_plugin_changes