Data in transit encryption with TLS in MySQLi/PDO MySQL/PDO PostgreSQL drivers

[andrepereiradasilva]

Summary of Changes

Add option to enable data in transit encryption with TLS in MySQLi/PDO MySQL/PDO PostgreSQL drivers.

Testing Instructions

0. Code review (not sure i miss something)
1. Test database connection (Mysqli and PDO Mysql and PDO PostgreSQL) in business as usual mode , make sure it works
2. Prepare MariaDB/MySQL/PostgreSQL server to accept TLS connections (examples only for MariaDB/MySQL)
   2.1. Add in /etc/my.cnf (replace for instance, with your public https certificates/private key)

; [...] mysql/mariadb configuration options

[mysqld]
; [...] more mysql/mariadb server configuration options
ssl        = 1
ssl_cert   = /etc/pki/tls/certs/fullchain.cer ; certificate + intermediates
ssl_ca     = /etc/pki/tls/certs/ca-bundle.crt
ssl_key    = /etc/pki/tls/private/yourdomain.key
ssl_cipher = kECDHE+aECDSA+AESGCM+AES128:kECDHE+aECDSA+AESGCM+AES256:kECDHE+aECDSA+AES128+SHA:kECDHE+aECDSA+AES256+SHA:kDHE+aECDSA+AES128:kDHE+aECDSA+AES256:kECDHE+aRSA+AESGCM+AES128:kECDHE+aRSA+AESGCM+AES256:kECDHE+aRSA+AES128+SHA:kECDHE+aRSA+AES256+SHA:kDHE+aRSA+AESGCM+AES128:kDHE+aRSA+AESGCM+AES256:kDHE+aRSA+AES128:kDHE+aRSA+AES256:kRSA+aRSA+AESGCM+AES128:kRSA+aRSA+AESGCM+AES256:kRSA+aRSA+AES128+SHA:kRSA+aRSA+AES256+SHA

2.2. restart the server service mariadb restart

3. For each one of the drivers (Mysqli and PDO Mysql):
   3.1. Test database connection enabling TLS and no certificate verfication, make sure it work (read the notes below)

$options['ssl'] = ['enable' => true, 'verify_server_cert' => false];

3.2. Test database connection enabling TLS and certificate verfication, make sure it work (read the notes below)

$options['ssl'] = ['enable' => true, 'verify_server_cert' => true];

3.3 Make more tests with ca, cipher, key, etc , example:

$options['ssl'] = [
  'enable'             => true,
  'verify_server_cert' => true,
  'cipher'             => 'AES256-GCM-SHA384:DES-CBC3-SHA',
];

Note: with client cert and key you can test two-way encryption also (in that case you need client certificates and acess to it from php), example:

$options['ssl'] = [
  'enable'             => true,
  'verify_server_cert' => true,
  'cipher'             => 'AES256-GCM-SHA384:DES-CBC3-SHA'
  'key'                => '/path/to/client-certficate.key', // must be accessible by php process
  'cert'               => '/path/to/client-certficate.cer', // must be accessible by php process
  'ca'                 => '/path/to/client-certficate-ca.cer', // must be accessible by php process
];

Documentation Changes Required

Would be nice to have a tutorial.

Notes

• Only tested in Joomla 4.0 staging | CentOS 7.x | MariaDB 10.4.x (mariadb repository) | PHP 7.3.x (ius repository)

• IMPORTANT: the server certificate verification verifies the certificate Common Name (CN)/Subject Alternate Names (SAN) so you have to have the proper host in the database connection to match the certificate. So make sure you can conneect to your database using yourdomain.tld hostname and yourdomain.tld is in the certificate.
  To be simply add 127.0.0.1 yourdomain.tld in /etc/hosts and use yourdomain.tld as database host. Make sure yourdomain.tld is in the certificate
  Note: MariaDB 10.1.23+, 10.2.6+ and 10.3.1+reference accepts Subject Alternate Names (SAN) in teh certificates, in previous version only the certificate Common Name (CN) is checked.

• Use TCP connections (ex: 127.0.0.1:3306 | yourdomain.tld:3306), not unix sockets or you will have an error when enabling TLS

• To make sure you have an encrypted connection use the new method in both drivers (it should return the TLS protocol and cipher used in the connection) echo $db->getConnectionEncryption();

• Since PDO does not allow to start a one-way TLS connection with server default vars (Mysqli driver as MYSQLI_CLIENT_SSL for this) we need to force something to start the TLS connection, my choice was the cipher (cipher suit) since is less problematic and it's still overrridable with $options['ssl']['cipher']

• I know nowadays we only use TLS, not SSL, but all config in all systens seems to still use ssl so the var is ssl, not tls

Reference

• https://mariadb.com/kb/en/library/data-in-transit-encryption/
• https://mariadb.com/kb/en/library/securing-connections-for-client-and-server/

Future WishList

• Someone to review/test postrgreSQL TLS connections
• Someone to make tests for this
• PHP be more consistent across database drivers ...

@HLeithner i think this is more consistent

[andrepereiradasilva]

so in the last 3 commits added TLS data in transit encryption to postgresql too, from the docs but don't have a way to test, maybe someone qith postgreSQL skills and environment can give a help with code review/test.
anyone ... @alikon

[richard67]

At the end if that works I would like to have a configuration option in global config server section in the CMS.

[andrepereiradasilva]

At the end if that works I would like to have a configuration option in global config server section in the CMS.

Hi @richard67
Yeah, That would be rather easy to implement latter if Database framework suports it

[alikon]

i've tryied to use a pre-configured docker image like this https://hub.docker.com/r/nimbustech/postgres-ssl/ but surely for my mistakes i wasn't able to setup it properly.....
so i can only state that from a code review perspective seems ok to me on the postgresql side

[andrepereiradasilva]

to simplify the test i made a joomla 4.0 branch with most of the changes required to use this in joomla 4
https://github.com/andrepereiradasilva/joomla-cms/pull/138/files

So

1. prepare your database server to allow encrypted connections (see above instructions and internet)
2. Use joomla 4
3. Apply this PR changes in joomla database (/libraries/vendor/joomla/database)
4. apply PR https://github.com/andrepereiradasilva/joomla-cms/pull/138/files in joomla 4
5. Then go to global config and in database section you have the new options
6. Then you can check in sysinfo the "Database Connection Encryption"

[richard67]

@andrepereiradasilva Thanks for your work and the testing instructions in your comment above.
I plan to test this on weekend, maybe Friday night if my surgery was not too bad. I hope this is sufficient.
By the way, nice to see you contributing from time to time to the db framework here. I will never forget the time some 5 years ago when we were doing much together for the CMS. Pity you don't have the time anymore.

[richard67]

Hmm, I just see I only have PostgreSQL server with version 10.9, but minimum requirement for J4 is 11 meanwhile. It seems that requirement is not really tested yet, because all the time I was able to test with 10.9. Would that be sufficient, or should I update to 11?

[alikon]

10.9 should be enough i cannot see from 11 changelog https://www.postgresql.org/docs/release/11.0/ impacting changes on this specifc matter

[mbabker]

Hmm, I just see I only have PostgreSQL server with version 10.9, but minimum requirement for J4 is 11 meanwhile.

The minimum requirement for the actual driver code does not coincide with the CMS' minimum support. So the driver here still works with PostgreSQL 9.4 and newer (because those are still actively supported versions).

And yes, this means someone will need to adjust the CMS if it imposes tighter restrictions than what the database driver itself does (which, right now, there is no sane reason to flat out blacklist the driver from not working with supported PostgreSQL versions, most of the hacks that were needed anyway were to deal with versions older than 9.4).

[richard67]

@andrepereiradasilva of course I could not wait and have started testing, but now have to interrupt. Stay tuned. Question: the sysinfo, shall it show what is actually used for the connection, or shall it show what is supported by the db server? I saw TLSv1.3 (TLS_AES_256_GCM_SHA384) when I had encryption switched off in global configuration.

Update: As far as I understand code, it should show what is currently used for the connection. Can it be that there is some encryption used even when we don't initiate that? My PostgreSQL 10.9 is a default setup, so maybe it already is ready for that and uses it when possible?

[richard67]

@andrepereiradasilva Find my first fix here: andrepereiradasilva#4.

[richard67]

What I've tested up to now with PostgreSQL is to use no encryption or server side without cert verification. For the rest (cert verification and client side encryption) I have to set up a certification chain which can be trusted and so on. The PostgreSQL doc I've linked above has very detailed desciptions on that with openssl on Linux.

See e.g. https://www.postgresql.org/docs/10/ssl-tcp.html#SSL-CERTIFICATE-CREATION.

Could be helpful for other testers, too, also with other database types.

Will continue to test tomorrow.

[andrepereiradasilva]

I plan to test this on weekend, maybe Friday night if my surgery was not too bad. I hope this is sufficient.

sure it will all go well

By the way, nice to see you contributing from time to time to the db framework here. I will never forget the time some 5 years ago when we were doing much together for the CMS. Pity you don't have the time anymore.

yeah, thanks for that too!
unfortunately i don't have the time to contribute more to this project. I hope someday i can change that. i don't forget this is a project, where, besides the very occasional drama (we are all humans afterall right?), i found that the large majority of people really try (a lot!) to help others and also the global community without expecting nothing in return which is rather rare to see! Also i can tell that in one way or another, with some less and some more, i learned from almost everyone i have been in contact in this community - so thanks for that!

@andrepereiradasilva of course I could not wait and have started testing, but now have to interrupt. Stay tuned. Question: the sysinfo, shall it show what is actually used for the connection, or shall it show what is supported by the db server? I saw TLSv1.3 (TLS_AES_256_GCM_SHA384) when I had encryption switched off in global configuration.
Update: As far as I understand code, it should show what is currently used for the connection. Can it be that there is some encryption used even when we don't initiate that? My PostgreSQL 10.9 is a default setup, so maybe it already is ready for that and uses it when possible?

being perfectly honest postgreSQL is not in my knowledge domains
but AFAIK for a one-way TLS connection to work you need to at least haver a certificate and a private key.
More problably the postgresql method query is not correct

database/src/Pgsql/PgsqlDriver.php

Lines 147 to 149 in 2f5242a

	->select($this->quoteName(['version', 'cipher']))
	->from($this->quoteName('pg_stat_ssl'))
	->where($this->quoteName('pid') . ' = pg_backend_pid()');

@andrepereiradasilva Find my first fix here: andrepereiradasilva#4.

thanks. merged

[richard67]

@andrepereiradasilva

More problably the postgresql method query is not correct

I've just checked PostgreSQL docs and it seems to be correct and return the info for the current session.

[andrepereiradasilva]

could postgresql be by default "sslmode=prefer"? but in that case i'm curious of what postgreSQL uses as server cert/key, a default custom selfsigned cert/key?

See https://ankane.org/postgres-sslmode-explained

If is that then "No Encryption" option should be something like "Auto (server preference)"

[richard67]

Here an extract of my postgresql.conf like it came with the installation, i.e. I have done nothing special to set up SSL/TLS:

- Security and Authentication -

#authentication_timeout = 1min # 1s-600s
ssl = on
#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed SSL ciphers
#ssl_prefer_server_ciphers = on
#ssl_ecdh_curve = 'prime256v1'
#ssl_dh_params_file = ''
ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem'
ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key'
#ssl_ca_file = ''
#ssl_crl_file = ''
#password_encryption = md5 # md5 or scram-sha-256
#db_user_namespace = off
#row_security = on

So you see it is switched on by default and the snakeoil pem and key files exist. This seems to be enough for a basic mode. I googled and found something telling that the way how you return the current ssl mode is right, when the session would have been opened without ssl, the values would be null in db, and also the function you call to get server PID for current connection is right.

In the link you posted is a link to a PostgreSQL doc, and there is below the table written:

The default value for sslmode is prefer.

So all in all PostgreSQL 10.9 seems to start up with encryption support after installation if all is right, i.e. openssl on which they rely is installed and so on.

[richard67]

Another thing I already tested is that when I switch on cert verification and I don't have the ca cert applied on the client, then connetion fails with an error message telling the reason, so setting sslmode=require works with PostgreSQL (PDO). If using that mode works I will test when I have set up my certificates chain.

[richard67]

Meanwhile I've tested one-way encryption with key verification and two way encryption, too, on PostgreSQL. I can verify that the connection is encrypted, but I don't know yet how I can verify that it is two way encrypted. Changing the cipher I also haven't managed yet, but that can be due to the limitations of what can be changed with the connection string on PostgreSQL PDO regarding SSL.

[richard67]

Ah I just see it is simply not possible to change cipher suite via the connection string. It is a server setting only. And so the code here does not try to do that.

[richard67]

Well what I could test with 2 way is that when I rename the client key file, I get an error message from the db client, telling that the cert file could be found but not the key. When changing to one way (clear 2 way parameters before) in global config, this error message does not come. So it seems to work that 2 way encryption is initiated (or tried to be initiated) when the necessary parameters are specified. If it is really used or not I can't really see.

I think this is sufficient as test with PostgreSQL. The PR seems to do what it shall do.

I assume with MySQL it is already well tested. @andrepereiradasilva is that correct?

[richard67]

To be really sure that it encrypts 2 way, I could of course use Wireshark, but to be honest I don't really want to do that.

[wilsonge]

I think that's good enough for me :) thankyou very much!

[andrepereiradasilva]

thansk all

[wilsonge]

Good to see you around again @andrepereiradasilva - hope all is well with you