Embedded database connection closed by shutdown() method [SPR-11372]

[spring-projects-issues]

Phil Webb opened SPR-11372 and commented

Spring Batch recently hit a very hard to track-down problem when upgrading from Spring 3.2.0 to 3.2.7.

When running the entire test suite some tests fail with the following exception:

java.sql.SQLNonTransientConnectionException: connection exception: closed

Running tests in isolation causes no problems.

The minimum suite that replicates the issue is:

@RunWith(Suite.class)
@SuiteClasses({
SimpleJobRepositoryIntegrationTests.class,
JobBuilderConfigurationTests.class,
ChunkOrientedStepIntegrationTests.class })
public class ReproTests {
}

After a lot of digging it appears that the root cause of the failure is #14347 [1].

Here is what happens:

SimpleJobRepositoryIntegrationTests uses spring-test and includes a commons-dbcp data-source. The data-source is part of a config that is cached.

JobBuilderConfigurationTests creates a direct EmbeddedDataSource. Tests here do not use spring-test. When the context is closed the shutdown method is called on the EmbeddedDataSource. This in turn issues some SQL that actually closes the HSQLDB session.

ChunkOrientedStepIntegrationTests runs, and reuses the previously created commons-dbcp pool. The pool appears to be open but because the entire session has been shutdown interaction cause exceptions.

[1] 4c823a3

---

Affects: 3.2 GA, 4.0 GA

Issue Links:

• De-register JDBC drivers on EmbeddedDatabase shutdown [SPR-7978] #12633 De-register JDBC drivers on EmbeddedDatabase shutdown
• Introduce support for generating unique names for embedded databases [SPR-8849] #13491 Introduce support for generating unique names for embedded databases

1 votes, 7 watchers

[spring-projects-issues]

Juergen Hoeller commented

I'll have a another look at this one, although I'm not sure we should be doing something about it... After all, an EmbeddedDatabase is usually meant to be shut down along with the closing of an application context, so this is arguably a feature... I guess a pre-existing EmbeddedDatabase could be decorated as a plain DataSource so that its shutdown method isn't exposed to the context... Or we could internally track whether a specific EmbeddedDatabase instance has been initialized by the current application context and decide whether the shutdown method is to be called that way.

Juergen

[spring-projects-issues]

Sam Brannen commented

Out of curiosity, are you assigning each embedded database a unique name?

Or are you relying on the default name (i.e., "testdb") which gets silently reused for every embedded database instance by default?

[spring-projects-issues]

Phil Webb commented

Sam Brannen, I can't remember now. I think it was the standard Spring Batch tests. spring-projects/spring-batch@b278239 might help.

[spring-projects-issues]

Michael Minella commented

Sam Brannen, is what you're proposing that each test in an entire project that happens to use an embedded database be required to come up with a unique name? On a large project, managing those names could get unwieldily. If that is the recommended approach (I'm not 100% sure if that would have addressed the issue, I'd have to go back myself), it would be nice at a minimum to have a flag of some kind that handles the name management (random names for example) instead of having to manage unique names across an entire project.

[spring-projects-issues]

Sam Brannen commented

Phil and Michael, I'll have a look in the near future, but as a quick reference...

#13491 is what I had in mind. See my comments regarding a work-around.

[spring-projects-issues]

Sam Brannen commented

Michael Minella, I am not suggesting that each test create its own unique embedded database instance; however, I am suggesting that groups of related tests ensure that they are using a unique embedded database instance. This can be achieved by assigning the embedded DB a unique name (as suggested in #13491).

In any case... with the use case described in the Description of this JIRA issue, the application context configuration for JobBuilderConfigurationTests should not attempt to reuse the embedded database created for SimpleJobRepositoryIntegrationTests.

As a general rule, development teams should ensure that one embedded DB is created for each application context used within the test suite. Otherwise, problems are likely to arise that are difficult to debug.

Of course, most developers are not aware of the pitfalls of accidentally reusing an embedded DB within a test suite. Thus, my recommendation is that Spring address the issue raised in #13491, perhaps along the lines that you mentioned (i.e., automatic unique DB name generation, potentially controlled by a boolean flag).

[spring-projects-issues]

Sam Brannen commented

Update: due to recent changes introduced in conjunction with #17432, there is now a robust and viable solution for setting a unique name for each embedded database configured via <jdbc:embedded-database>:

<jdbc:embedded-database id="dataSource" database-name="#{T(java.util.UUID).randomUUID().toString()}">
    <jdbc:script location="..." />
</jdbc:embedded-database>

See #13491 for further details in the comments section.

[spring-projects-issues]

Sam Brannen commented

Due to completion of #13491, there is now first-class support for auto-generating embedded database names configured via:

XML

<jdbc:embedded-database id="dataSource" generate-name="true">
    <jdbc:script location="..." />
</jdbc:embedded-database>

Java Config

DataSource dataSource = new EmbeddedDatabaseBuilder()
    .addScripts("my-schema.sql", "my-test-data.sql")
    .generateUniqueName(true)
    .build();