MySQL connection times out after pt-online-schema-change run

[hseffler]

When using pt-online-schema-change from liquibase via liquibase-percona the following happens for long migrations:

1. liquibase locks the database
2. pt-online-schema-change runs and executes sucessfully
3. liquibase tries to write the changelog entry and remove the lock

However the jdbc database connection that liquibase is using has gone idle in the meantime and the liquibase run fails and requires manual intervention.

db: mysql
liquibase: 3.5.1
liquibase-percona: 1.6.0

[adangel]

Do you use some specific connection properties? How long took pt-online-schema-change to run?

You are maybe running into the timeout, that the configurable server-side as wait_timeout. This is by default 8 hours.
You could try to play with autoReconnect. But according to the description, the driver will still throw a exception, so liquibase's query to write the changelog entry probably still doesn't work.

We probably need a small background thread to keep liquibase's connection alive while pt-online-schema-change uses its own connection.

[hseffler]

In my scenario pt-online-schema-change runs for more than 1h, its quite a large production database. I already tried the autoReconnect-flag of the mysql-driver, but as you noted it still throws an exception before eventually reconnecting, so this does not help in this case.

wait_timeout on the mysql-server-side would possibly help but its set to a much lower value due to other constraints on this database by the mysql-admins.