Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail gracefully if no JDBC driver found #1434

Merged
merged 7 commits into from
Jul 21, 2019
Merged

Fail gracefully if no JDBC driver found #1434

merged 7 commits into from
Jul 21, 2019

Conversation

rnorth
Copy link
Member

@rnorth rnorth commented Apr 28, 2019

Fixes #678

@rnorth rnorth changed the title WIP: Fail gracefully if no JDBC driver found Fail gracefully if no JDBC driver found May 15, 2019
@rnorth rnorth marked this pull request as ready for review May 15, 2019 09:22
kiview
kiview previously approved these changes Jun 12, 2019
core/build.gradle Outdated Show resolved Hide resolved
@bsideup bsideup mentioned this pull request Jul 6, 2019
@rnorth
Copy link
Member Author

rnorth commented Jul 6, 2019

Oh boy, awaitility has a dependency on hamcrest, which seems to break spectacularly when shaded. Having Hamcrest as an unshaded dependency doesn't seem good.

Will pause work on this for today.

@rnorth rnorth changed the title Fail gracefully if no JDBC driver found WIP: Fail gracefully if no JDBC driver found Jul 16, 2019
@rnorth rnorth self-assigned this Jul 16, 2019
@rnorth rnorth dismissed kiview’s stale review July 16, 2019 17:20

Taking this back to WIP state - I want to remove the awaitility dependency

(Auto-selected driver on classpath seems to be MariaDB; seemingly not compatible with 8.0.16 of MySQL)
to avoid reconnection attempts to dead containers after test has already completed
@@ -26,7 +31,16 @@ DataSource getDataSource(JdbcDatabaseContainer container) {
hikariConfig.setJdbcUrl(container.getJdbcUrl());
hikariConfig.setUsername(container.getUsername());
hikariConfig.setPassword(container.getPassword());
hikariConfig.setDriverClassName(container.getDriverClassName());
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found that, seemingly dependent on test order, the MariaDB driver was incorrectly being used for communicating with MySQL. This was OK normally but failed with MySQL 8.0.16. This was a strange issue that only affected my local Gradle builds, not IDE runs of the affected test, which is why I think it's likely to be test order related.

Adding this line ensures that Hikari definitely uses the MySQL driver for talking to MySQL, and resolves the issue.

return new HikariDataSource(hikariConfig);
@After
public void teardown() {
datasourcesForCleanup.forEach(HikariDataSource::close);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was noticing a lot of Creating connection logs from the Clickhouse driver during unrelated tests, long after the Clickhouse tests completed.

I realised that the Hikari connection pools were not being closed after use, and were hanging around trying to reconnect to containers that were long dead.

I've added this cleanup code to the parent test class to ensure that the connection pools are always closed, regardless of how they are used. We could alternatively use try-with-resources, but we'd have to make sure we use it in all the tests that are based on this abstract class.

logger().info("Waiting for database connection to become available at {} using query '{}'", getJdbcUrl(), getTestQueryString());
Unreliables.retryUntilSuccess(getStartupTimeoutSeconds(), TimeUnit.SECONDS, () -> {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this class are the core of this PR:

I've removed use of duct-tape Unreliables as it isn't a great fit for the fail-fast behaviour we want. Also, Unreliables' currently can't properly enforce timeout on code that would otherwise infinitely loop (TLDR, it doesn't cancel the Future, which is a bug. I'd like to fix this separately).

I originally dropped in Awaitility as a replacement, but realised this was a mistake when shading it; it has a dependency on Hamcrest which seems to break when shaded, and I generally became quite uncomfortable with having Hamcrest in our dependency tree at all.

Despite not using a library any more, I hope that the code here is actually fairly clear in its intent.

@rnorth rnorth changed the title WIP: Fail gracefully if no JDBC driver found Fail gracefully if no JDBC driver found Jul 17, 2019
@rnorth
Copy link
Member Author

rnorth commented Jul 17, 2019

Ready for re-review. Making the logback-test.xml files consistent has made this PR a lot bigger than it would otherwise have been; please let me know if it's a PITA to review and I could separate out into a separate PR.

@rnorth rnorth requested review from bsideup and kiview July 17, 2019 19:38
@rnorth rnorth added this to the next milestone Jul 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MySQLContainer junit Rule hangs if there is no jdbc-driver on classpath
3 participants