Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aborted connection to db: 'unconnected' user: 'unauthenticated' host: 'xxx' (This connection closed normally without authentication) #560

Closed
MyNameIsRatchet opened this issue Feb 21, 2024 · 42 comments · Fixed by docker-library/official-images#16293
Labels
bug Workaround Available Bug that can be worked around with a fix. Check issue comments.

Comments

@MyNameIsRatchet
Copy link

MyNameIsRatchet commented Feb 21, 2024

I'm running mariadb in docker with "latest"-tag. A nextcloud-container accesses the db via docker internal network.
Everything worked fine, but after I pull the latest (dae04d35ddd4ca837c1d330962eab9b5577325d1751e1877203b3fafe67b5daa) mariadb-image, Nextcloud can't connect anymore and mariadb throws the following:
Aborted connection to db: 'unconnected' user: 'unauthenticated' host: 'IP' (This connection closed normally without authentication)
When I start the container with the previous image I used before, everything works fine.

Anyone else?

@PhilippHandle
Copy link

Yeah and if i want to connect an application to it, i get the error SQLSTATE[HY000] [2054] Server sent charset (0) unknown to the client. Please, report to the developers the update killed all my applications.

@WAdama
Copy link

WAdama commented Feb 21, 2024

Can second that:
Message from MariaDB: 2024-02-21 8:03:53 259 [Warning] Aborted connection 259 to db: 'unconnected' user: 'unauthenticated' host: '172.23.0.3' (This connection closed normally without authentication)

Message from Nextcloud: Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2054] Server sent charset (0) unknown to the client. Please, report to the developers in /var/www/html/lib/private/DB/Connection.php:139

Going back to last image solved the problem for the moment.

P.S. Bookstack container seems to have the same problem...

@thomase1234
Copy link

I rolled back to 11.2 because none of my applications could connect to the mariadb.

@WAdama
Copy link

WAdama commented Feb 21, 2024

Found out, not all of my containers using MariaDB have this problem:
Bookstack > don't work
Firefox Sync RS > works
Nextcloud > don't work
Paperless-ngx > works

@cvicentiu
Copy link
Member

cvicentiu commented Feb 21, 2024

We're looking into this right now.

If anyone can provide an easy to reproduce test case, it would help out in diagnosing this.

Relevant info:

  1. The user authentication configuration the server had. (Which authentication plugins)
    SHOW CREATE USER (and remove any password hashes from there for privacy)
  2. The client version that was used to connect to the server. Language and version of the connector if it's an application connector.

@PhilippHandle
Copy link

PhilippHandle commented Feb 21, 2024

Here are my configurations, all from a local installation so passwords are dummy
export of show create user:

+---------------------------------------------------------------------------------------------------+
| CREATE USER for root@localhost                                                                    |
+---------------------------------------------------------------------------------------------------+
| CREATE USER `root`@`localhost` IDENTIFIED BY PASSWORD 'HASH' |
+---------------------------------------------------------------------------------------------------+
1 row in set (0.002 sec)

docker settings
image

it's an php (8.3.2) application connection with doctrine 3.0.0
image

@kimbtech
Copy link

I would like to add that the issue does not affect:

  • Roundcube Mail (Docker-Image roundcube/roundcubemail:latest-apache)
  • GiTea (Docker-Image gitea/gitea:1)

@LinuxJedi
Copy link

I believe this is the cause: https://jira.mariadb.org/browse/MDEV-32975

@LinuxJedi
Copy link

LinuxJedi commented Feb 21, 2024

I'm not a Docker expert, so I'm trying to figure a workaround now, but basically the file /etc/mysql/mariadb.conf.d/50-server.cnf has an issue that needs fixing. The line character-set-collations = utf8mb4=uca1400_ai_ci should be changed to:

  • character-set-collations = utf8mb4_general_ci

I'll see if I can investigate how to do this now, but if anyone has more Docker knowledge than me, please feel free to post it here.

@martadinata666
Copy link

assuming if this only charset issue, docker wise can override command part with the charset.

 db:
    image: mariadb:lts
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin 
    environment:
      #- MARIADB_AUTO_UPGRADE=1
      - MARIADB_ROOT_PASSWORD=mariadbnc
      - TZ=Asia/Jakarta
      - MARIADB_DATABASE=nextcloud
      - MARIADB_USER=nextcloud
      - MARIADB_PASSWORD=nextcloud
    volumes:
      - ./db:/var/lib/mysql
    restart: unless-stopped

@LinuxJedi
Copy link

That looks like it should work, I'm going to set up a test environment here shortly.

@PhilippHandle
Copy link

assuming if this only charset issue, docker wise can override command part with the charset.

 db:
    image: mariadb:lts
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin 
    environment:
      #- MARIADB_AUTO_UPGRADE=1
      - MARIADB_ROOT_PASSWORD=mariadbnc
      - TZ=Asia/Jakarta
      - MARIADB_DATABASE=nextcloud
      - MARIADB_USER=nextcloud
      - MARIADB_PASSWORD=nextcloud
    volumes:
      - ./db:/var/lib/mysql
    restart: unless-stopped

it works for me!

@an3l
Copy link
Contributor

an3l commented Feb 21, 2024

Yes without changes of collations for mariadb:latest :

$ docker exec -it mariadb-1 mariadb -uroot -psecret -e "select @@version; select @@character_set_collations;"
+---------------------------------------+
| @@version                             |
+---------------------------------------+
| 11.3.2-MariaDB-1:11.3.2+maria~ubu2204 |
+---------------------------------------+
+-------------------------------+
| @@character_set_collations    |
+-------------------------------+
| utf8mb4=utf8mb4_uca1400_ai_ci |
+-------------------------------+

I have tested suggested changes for character-set-collations on the latest image, variable is reset, but doesn't work change.

  mariadb-new-collation:
    <<: *common-attributes # aliases
    container_name: mariadb-cont
    command: --character-set-collations=utf8mb4_general_ci
$ docker exec -it mariadb-cont  -uroot -psecret -e "select @@version; set character_set_collations=utf8mb4_general_ci; select @@character_set_collations"
+---------------------------------------+
| @@version                             |
+---------------------------------------+
| 11.3.2-MariaDB-1:11.3.2+maria~ubu2204 |
+---------------------------------------+
ERROR 1231 (42000) at line 1: Variable 'character_set_collations' can't be set to the value of 'utf8mb4_general_ci'

$ docker exec -it mariadb-cont -uroot -psecret -e "select @@version; select @@character_set_collations;"
+---------------------------------------+
| @@version                             |
+---------------------------------------+
| 11.3.2-MariaDB-1:11.3.2+maria~ubu2204 |
+---------------------------------------+
+----------------------------+
| @@character_set_collations |
+----------------------------+
|                            |
+----------------------------+

Same happens for other collation ERROR 1231 (42000) at line 1: Variable 'character_set_collations' can't be set to the value of 'utf8mb4_uca1400_ai_ci'
Same happens on startup - tested with docker compose

mariadb-cont      | 2024-02-21 12:01:38+00:00 [Note] [Entrypoint]: Initializing database files
mariadb-cont      | 2024-02-21 12:01:38 0 [ERROR] Variable 'character_set_collations' can't be set to the value of 'utf8mb4_uca1400_ai_ci'

@cvicentiu
Copy link
Member

cvicentiu commented Feb 21, 2024

assuming if this only charset issue, docker wise can override command part with the charset.

 db:
    image: mariadb:lts
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin 
    environment:
      #- MARIADB_AUTO_UPGRADE=1
      - MARIADB_ROOT_PASSWORD=mariadbnc
      - TZ=Asia/Jakarta
      - MARIADB_DATABASE=nextcloud
      - MARIADB_USER=nextcloud
      - MARIADB_PASSWORD=nextcloud
    volumes:
      - ./db:/var/lib/mysql
    restart: unless-stopped

it works for me!

Just to be clear: Are you running the :lts MariaDB or the :latest?

@PhilippHandle
Copy link

the latest version -> 11.3.2

@cvicentiu cvicentiu added Workaround Available Bug that can be worked around with a fix. Check issue comments. bug labels Feb 21, 2024
@WAdama
Copy link

WAdama commented Feb 21, 2024

Hi All,
can second this, too...
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin did the trick in my case also.

Both, Netxcloud and Bookstack are running now with 11.3.2. And the other ones, which were not affected, are still running...

@SHOzturk-PV
Copy link

SHOzturk-PV commented Feb 21, 2024

First of all thank you @LinuxJedi for hitting the nail on the head.

Many of us are here because our nextcloud setups went down and putting the collation into the docker command serves as a workaround for now. Nonetheless I would advise having a glance at the nextcloud documentations for docker compose examples and database config.

The example already gives the following line:
command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
And the typical use case collation seems to be "utf8mb4_general_ci"
So the following line would probably serve nextcloud users the best:
command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci

@stanthewizzard
Copy link

command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci

IT works for me

@martadinata666
Copy link

martadinata666 commented Feb 21, 2024

So far I can tell, if the database created by latest 11.3.2, then it can't be read/connect mostly "PHP module" I assume. If the database created before latest then force mariadb to use old charset. *cmiiw

Check on dbtest
Database created from :latest no command override

docker exec -it db-test-db-1 mariadb -uroot -ppassword -e "SELECT SCHEMA_NAME 'database', default_character_set_name 'charset', DEFAULT_COLLATION_NAME 'collation' FROM information_schema.SCHEMATA;"
+--------------------+---------+-----------------------+
| database           | charset | collation             |
+--------------------+---------+-----------------------+
| information_schema | utf8mb3 | utf8mb3_general_ci    |
| performance_schema | utf8mb3 | utf8mb3_general_ci    |
| mysql              | utf8mb4 | utf8mb4_uca1400_ai_ci |
| dbtest             | utf8mb4 | utf8mb4_uca1400_ai_ci |
| sys                | utf8mb3 | utf8mb3_general_ci    |
+--------------------+---------+-----------------------+

Database created from :latest override command

docker exec -it db-test-db-1 mariadb -uroot -ppassword -e "SELECT SCHEMA_NAME 'database', default_character_set_name 'charset', DEFAULT_COLLATION_NAME 'collation' FROM information_schema.SCHEMATA;"
+--------------------+---------+--------------------+
| database           | charset | collation          |
+--------------------+---------+--------------------+
| information_schema | utf8mb3 | utf8mb3_general_ci |
| performance_schema | utf8mb3 | utf8mb3_general_ci |
| mysql              | utf8mb4 | utf8mb4_general_ci |
| dbtest             | utf8mb4 | utf8mb4_general_ci |
| sys                | utf8mb3 | utf8mb3_general_ci |
+--------------------+---------+--------------------+

Database from old version to latest no override

docker exec -it db-test-db-1 mariadb -uroot -ppassword -e "SELECT SCHEMA_NAME 'database', default_character_set_name 'charset', DEFAULT_COLLATION_NAME 'collation' FROM information_schema.SCHEMATA;"
+--------------------+---------+--------------------+
| database           | charset | collation          |
+--------------------+---------+--------------------+
| information_schema | utf8mb3 | utf8mb3_general_ci |
| performance_schema | utf8mb3 | utf8mb3_general_ci |
| mysql              | utf8mb4 | utf8mb4_general_ci |
| dbtest             | utf8mb4 | utf8mb4_general_ci |
| sys                | utf8mb3 | utf8mb3_general_ci |
+--------------------+---------+--------------------+

Stay same

docker exec -it db-test-db-1 mariadb -uroot -ppassword -e "SELECT SCHEMA_NAME 'database', default_character_set_name 'charset', DEFAULT_COLLATION_NAME 'collation' FROM information_schema.SCHEMATA;"
+--------------------+---------+--------------------+
| database           | charset | collation          |
+--------------------+---------+--------------------+
| information_schema | utf8mb3 | utf8mb3_general_ci |
| performance_schema | utf8mb3 | utf8mb3_general_ci |
| mysql              | utf8mb4 | utf8mb4_general_ci |
| dbtest             | utf8mb4 | utf8mb4_general_ci |
| sys                | utf8mb3 | utf8mb3_general_ci |
+--------------------+---------+--------------------+

@SHOzturk-PV
Copy link

SHOzturk-PV commented Feb 21, 2024

50-server.cnf commit

It seems the dev wanted to modernize utf8mb4_general_ci to utf8mb4_uca1400_ai_cias as default but made both an executive decision and typo for php reasons.

CATCHPOLEBHC added a commit to bhccwebmaster/drupal-container that referenced this issue Feb 21, 2024
@LinuxJedi
Copy link

50-server.cnf commit

It seems the dev wanted to modernize utf8mb4_general_ci to utf8mb4_uca1400_ai_cias as default but made both an executive decision and typo.

I too thought that it was a typo too, but it is in fact correct (see https://mariadb.com/kb/en/server-system-variables/#character_set_collations). The problem stems from compatibility with the PHP MySQL connector and that default change for our Debian packages (which affects Docker images too).

We already have a patch to fix the compatibility, but it hasn't been merged yet. This patch is now blocker level priority for us. I suspect something will be done on the Docker image as well in the mean time.

@agarov
Copy link

agarov commented Feb 21, 2024

I encountered the same problem when using this new image with wordpress docker image.

I can connect using the mysql client from one container to the other, but wordpress can't.

Rollbacking to the previous version resolved the problem.

tianon pushed a commit to docker-library/official-images that referenced this issue Feb 22, 2024
Unfortunately the last latest release broken PHP and nodejs
because of a protocol issue. The configuration item
character-set-collations added in the default 11.3/11.4
image generated this.

Closes: MariaDB/mariadb-docker#560
@jcrossbdn
Copy link

Can confirm that all my containers running php scripts that connect to mariadb databases using mysqli also have this issue resulting in error "Aborted connection to db: 'unconnected' user: 'unauthenticated' host: 'IP' (This connection closed normally without authentication)". Connecting from the command line from inside the container using "mysql -h x -u y -p" did work properly as well as externally using dBeaver.

Adding "command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin" to the docker-compose file on each container has "corrected" the issue.

@grooverdan
Copy link
Member

grooverdan commented Feb 22, 2024

Docker hub contains new image as of ~15 mins ago. No work arounds should be required.

@hosseinitabar
Copy link

Man, we had mardiadb docker from 34 hours ago and we never even realized that this madness went on. Today I wanted to create a database and encountered this error. Kudos to people who solved it that fast. I got the newest version, from 15 minutes ago. I'm just so happy.

@LinuxJedi
Copy link

We apologise again for the inconvenience, I'm working an internal post-mortem right now (may make public too if there is interest) so that we can learn from it and prevent anything like this happening again for PHP / Docker users.

@ckuethe
Copy link

ckuethe commented Feb 22, 2024

I'm working an internal post-mortem right now (may make public too if there is interest) so that we can learn from it ...

I think it's fair to say there would be interest. We're all interested in building more robust systems, right?

@stanthewizzard
Copy link

I keep the commabd even with the new version :)
thanks for post mortem

@LinuxJedi
Copy link

I think it's fair to say there would be interest. We're all interested in building more robust systems, right?

Had clearance to do that. I'll write it up in a more publicly consumable form in the next couple of days.

@mattdale77
Copy link

I just deployed the latest and still received the same error. I've switched back to lts and it's fine but I believed that this issue was resolved already?

@danieljweinberg
Copy link

I just deployed the latest and still received the same error. I've switched back to lts and it's fine but I believed that this issue was resolved already?

I had the same problem in docker-compose, "latest" simply reused the most recent image, docker did not think the image had changed. I had to delete all mariadb images and pull latest again, then it worked. @mattdale77

@mattdale77
Copy link

I just deployed the latest and still received the same error. I've switched back to lts and it's fine but I believed that this issue was resolved already?

I had the same problem in docker-compose, "latest" simply reused the most recent image, docker did not think the image had changed. I had to delete all mariadb images and pull latest again, then it worked. @mattdale77

Thanks. I've got a few so I'll need to track them all down. Thanks for the tip

@LinuxJedi
Copy link

For those who requested it, post-mortem here: https://mariadb.org/post-mortem-php-and-mariadb-docker-issue/

martin-g pushed a commit to martin-g/docker-official-images that referenced this issue Apr 3, 2024
Unfortunately the last latest release broken PHP and nodejs
because of a protocol issue. The configuration item
character-set-collations added in the default 11.3/11.4
image generated this.

Closes: MariaDB/mariadb-docker#560
grooverdan added a commit to grooverdan/mariadb-docker that referenced this issue Apr 11, 2024
grooverdan added a commit that referenced this issue Apr 15, 2024
grooverdan added a commit that referenced this issue May 17, 2024
grooverdan added a commit that referenced this issue May 23, 2024
This reverts commit 1e14bca.
but only for 11.3/11.4 which haven't been updated.
grooverdan added a commit to grooverdan/mariadb-docker that referenced this issue May 23, 2024
grooverdan added a commit that referenced this issue May 30, 2024
@Nefcanto
Copy link

Nefcanto commented Jun 5, 2024

Guys, I read this thread, but I could not figure out if the new collation utf8mb4_uca1400_ai_ci is here to stay as the new default collation or if it will be reverted back to utf8mb4_general_ci? Can anyone please guide me regarding that?

Because if it's the new default, then we should create a migration script to update all of our databases on our production server.

@Simbiat
Copy link

Simbiat commented Jun 23, 2024

I just recently started using Docker, and I am using MariaDB's 11.4.2-noble. I have

[client]
default-character-set=utf8mb4

[mysqld]
character-set-server=utf8mb4
collation-server=utf8mb4_uca1400_nopad_as_cs

and I am getting these errors every like 10 seconds:

2024-06-23 13:11:52 19351 [Warning] Aborted connection 19351 to db: 'unconnected' user: 'unauthenticated' host: '::1' (This connection closed normally without authentication)

But as far as I know I do not have anything on the same container that would connect over ::1. Connections from another container (frankenphp) work absolutely fine, too, no errors. Is this somehow related to this issue or is this something completely different?

@grooverdan
Copy link
Member

@Simbiat,

The original cause of this one was fixed with 11.4.2.

The way I managed to generate that error was podman exec containername socat STDIO TCP6:[::1]:3306. It looks like a custom healthcheck or liveliness test performing tcp connection without authentication.

I also reproduced this with:

$ podman exec  cool_solomon rm /var/lib/mysql/.my-healthcheck.cnf
$ podman exec  cool_solomon healthcheck.sh --connect

If your .my-healthcheck.sh is also has all permissions like #573 that could also be a causing factor if and unauthenticated tcp checks.

@Simbiat
Copy link

Simbiat commented Jun 24, 2024

My case is slightly different, than tat, but I'll post in that issue, since it's close to that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Workaround Available Bug that can be worked around with a fix. Check issue comments.
Development

Successfully merging a pull request may close this issue.