Batch MetaData Collection Hangs When Using HSQLDB

[geiz123]

As the title says. I've tried the same code base with H2 and the job doesn't hang. When it hangs, pausing the debugger shows Thread 1 is waiting for Thread 2 at HsqlMaxValueIncrementer and Thread 2 is waiting for Thread 1. Another case of thread deadlock.

Here is the entire example: https://github.com/geiz123/async-batch-test

It has two classes.

AppDataSourceConfiguration just setup a datasource bean for the batch job to use.
AppConfiguration sets up everything else, step, reader, main method, and etc.

Did I setup my batch job incorrectly?

[fmbenhassine]

I think this is because the multi-version concurrency control is not active by default in HSQLDB.

Can you add hsqldb.tx=mvcc to the connection string and try again?

Here is an example:

spring-batch/spring-batch-core/src/test/resources/org/springframework/batch/core/repository/dao/sql-dao-test.xml

Line 8 in 018fcf4

<property name="url" value="jdbc:hsqldb:mem:test;sql.enforce_strict_size=true;hsqldb.tx=mvcc" />

[geiz123]

I added the parameter to the connection string, but the program still encountered the same deadlock.

Here is the new change branch: https://github.com/geiz123/async-batch-test/tree/spring-suggestion

I created a new DataSource bean with the new connection URL and populated it with the metadata tables:
https://github.com/geiz123/async-batch-test/blob/f30f81c052163b0c59e004b079b1d7f333176c61/src/main/java/com/test/AppDataSourceConfiguration.java#L45C2-L64C3

Then let DefaultBatchConfiguration#getDataSource() return:
https://github.com/geiz123/async-batch-test/blob/f30f81c052163b0c59e004b079b1d7f333176c61/src/main/java/com/test/AppConfiguration.java#L219C2-L222C3

[fmbenhassine]

Thank you for the update. I tried your sample and I noticed that it works when there is a single flow in the split. (by commenting this line) or by replacing one of the flows with a simple tasklet. So this might be related to the way threads from the pool are scheduled to run and how they acquire connections from the database pool.

Another thing I noticed is that the issue happens even without opening/closing a connection in the reader. Any reason for doing that? FTR, I never had this issue with a regular jdbc reader getting data from a table.

[geiz123]

No particular reason for the open/close connection in the reader. I've tried the code with a simple reader that count with an AtomicInteger and the job would still hang.

I'll try replacing one of the flows with a simple tasklet per your suggestion.

[fmbenhassine]

yes please, try and let me know.

We use hsqldb in our entire test suite and we did not encounter any issues with it.