Support for ECH (Encrypted Client Hello)

[ftasnetamot]

Hi,
I am opening this discussion, as this may be of interest to several sslh users.
Encrypted Client Hello (ECH) is used, to hide the name of the target website in an initial TLS connection. Where we are using today SNI (and sniffing SNI with sslh to redirect certain connections), ECH hides this information in first part.
The real servername is encrypted with an outer key, and first after this is decrypted, the real destination could be read.
Very short overview here: https://en.wikipedia.org/wiki/Server_Name_Indication#Encrypted_Client_Hello
Longer information here: https://blog.cloudflare.com/announcing-encrypted-client-hello/ and https://developers.cloudflare.com/ssl/edge-certificates/ech/
Right now it looks like, that this extension is on its way. Unfortunately to my research (please proof me beeing wrong) it is not yet supported by core applications like OpenSSL, Apache, Nginx and others.
There are several forks out there, where this is build in, like:

• https://github.com/defo-project
  with forks of Apache, Nginx,OpenSSL, Curl, lighty, ...
• https://github.com/sftcd/
  with a fork of OpenSSL

The current RFC draft is here: https://datatracker.ietf.org/doc/draft-ietf-tls-esni/ expiring 19 March 2025. As Cloudflare is driving this in own interest, I expect another update or approval within this time.

Right now Firefox and Chrome/Chromium are supporting ECH, all the Linux tools like wget, curl, OpenSSL, GnuSSL, Apache2, Nginx, Lighty etc. don't support it without patch. However I think, that we will see this coming in 2025.

So lets collect informations and ideas here, how to deal with this feature. This is very welcome for all of us, who are hiding ssh access or admin interfaces behind sslh, as todays configurations are more obfuscating than hiding, as SNI is visible to everyone on the network path.

Current discussion for OpenSSL

[yrutschle]

Funny you should post about this, I just recently started reading the RFC draft with the idea to support this eventually.
Current SNI support was imported from another project called sniproxy, which apparently is no longer supported.

Clearly I want to import as little crypto as possible, so the idea would be to rely on any implementation that is widely used (I guess Apache or Nginx would make good starting points) under the form of an external library.
Ideally, we'll want to integrate that in a way that is abstract enough that we can also integrate QUICK in a similar way.

Thanks for the resources. I'll start looking into it slowly, but don't expect much action on my side before January, with Christmas season almost upon us :-)

[ftasnetamot]

My current expectation is, that it will last some month, before something can be coded. Cloudflare seems to be currently ways in front, as they do all inside their own software and they don't need to be compatible to other applications. The pilot projects, forking nginx, OpenSSL etc. are using some quick and dirty workarounds to get things working.
From my understanding, watching the OpenSSL and perhaps GnuSSL discussions, as those two libraries will be the core functionality, other applications like apache, nginx, curl etc. have to use for proper implementation.
Right now -also my current understanding- the DNS implementation is the key, and we will see next year new DNS RRs, which will deliver additional information to a hidden hostname: The public hostname, behind this host is reachable an additional key, which is needed, to encrypt the Client Hello to this host and additional information, like the key type, length, used curves, ...
This information needs to be pulled either via DoH or DoT from the nameserver, and lastly the authoritative nameserver needs to be DNSSEC secured, to avoid MITM attacks, by feeding the DoH server with false dns informations.
I expect, that once there is a SSL-library with ECH available, all the needed applications will followshortly.
But I don't expect, that we get stable SSL-libraries as long, as the RFC is a draft, and as long, as the clean and secure dns definitions are there.

What I see for sslh is the following: SNI will disappear very soon, after ECH is available in most standard applications. For WWW this will happen first, as two browsers are supporting current draft implementations. Other applications like mail, messaging, telephony etc. will follow. SNI will be rated as deprecated by the usual security rating tools, as a better standard is available.
That means, that we need some basic crypto-code in sslh to decrypt the Client Hello packet, before beeing able to decide about routing.
Sslh needs access to the private key, matching the public key, which was distributed by the webserver along with the name of the public facing hostname. So the ECH part can be decrypted, and sslh can continue with the current code for decisions how to handle the packet.
You will need one additional function, which will lookup a new configuration table for the private key associated to the public name found in the TLS packet. After that, the ECH needs to be decrypted, to do the decisions, but the encrypted package will be forwarded to the TLS destination, where the application also owns the private key, to establish the connection.

For sslh the dns-part of ECH is unimportant, as sslh only needs to temporarily decrypt the ECH for the needed decisions, while the encrypted ECH is securely stored. When the decision is made, the stored encrypted ECH needs to be forwarded to the right destination.

So sslh doesn't need to to establish TLS handshakes or whatsoever, it only needs a good library for decrypting the header, before it can continue with routing decisions like today.

But all that is my current understanding, which may change, when I dig deeper into this.

[ftasnetamot]

Did some more reading and came to the conclusion, that sslh could be a motor for rolling out ECH.
Chapter 3.1 of the ECH RFC explains the following:

3.1.  Topologies

                   +---------------------+
                   |                     |
                   |   2001:DB8::1111    |
                   |                     |
   Client <----->  | private.example.org |
                   |                     |
                   | public.example.com  |
                   |                     |
                   +---------------------+
                           Server
             (Client-Facing and Backend Combined)

                       Figure 1: Shared Mode Topology

   In Shared Mode, the provider is the origin server for all the domains
   whose DNS records point to it.  In this mode, the TLS connection is
   terminated by the provider.

              +--------------------+     +---------------------+
              |                    |     |                     |
              |   2001:DB8::1111   |     |   2001:DB8::EEEE    |
   Client <----------------------------->|                     |
              | public.example.com |     | private.example.com |
              |                    |     |                     |
              +--------------------+     +---------------------+
               Client-Facing Server            Backend Server

                       Figure 2: Split Mode Topology

   In Split Mode, the provider is not the origin server for private
   domains.  Rather, the DNS records for private domains point to the
   provider, and the provider's server relays the connection back to the
   origin server, who terminates the TLS connection with the client.
   Importantly, the service provider does not have access to the
   plaintext of the connection beyond the unencrypted portions of the
   handshake.

which looks very interesting towards sslh. My conclusion is, that sslh could be easily the client-facing server, as the role of this is, only decrypting the Client Hello Inner and forward the connection to a second server, which handles this connection as a classic TLS connection with SNI.

So sslh has two possibilities dealing with ECH:

1. Similar to todays functionality, sslh can

• accept the connection
• save the encrypted Client Hello Inner for later use
• decrypt the Client Hello Inner for routing decisions
• forward the connection to the target, using the previous saved Client Hello Inner
• start shoveling

2. In addition sslh could also:

• accept the connection
• decrypt the Client Hello Inner for routing decisions
• construct a traditional Client Hello packet with the SNI name, derived from the previous decryption
• forward the connection to the target, using this backwards compatible Client Hello packet.
• start shoveling

This second approach could be very interesting in rolling out ECH, because this would turn sslh in an ECH enabler, for many TLS based applications. This means, if sslh would have this functionality, you can start using ECH with a lot of applications, while they are not ECH capable by themself.

For this to happen, (at least) three functions would be necessary:

1. A DNS-resolving function, retrieving the needed decryption information
2. A decrypt-function, which uses this information, to decrypt the Client Hello Inner
   for writing this, it is needed to learn about HPKE, see also, or finally the RFC. As HPKE is only a framework, there is also a decent crypt library needed, supporting the used cyphers.
3. A function, which can construct an classic Client Hello packet with embedded SNI based on the information collected by the first two functions.

One important finding is, that Bind9 is already supporting DoH along with DoT, so this could be used with own test-domains as DNS source, independent from cloudflare or other companies.

Another finding: HPKE will be supported by the upcoming OpenSSL 3.2. There is a quiet mature branch at github, the 3.3 and 3.4 branches are already available and what I see is, that in 3.2 only bugfixing is done. October, 26th beta 2 was published. This also includes client support for QUIC.

HPKE is supported by OpenSSL starting with version 3.2. See. I looked yesterday at the wrong end, I should have looked to releases and not to branches. When I look into the debian world, they still stuck to 3.0(.15), all higher versions are however available in the testing/unstable and experimental versions. That means, it should be possible to work in a testing environment, where all third party libraries are coming from an distro repository and not from other forks.