[jdbc] support JDBC option to rewrite batch statement to multi-row in…

[robert-s-lee]

batch statement int[] results = insertStatement.executeBatch() returns 1 for successful completion. when the batch statement is re-written to multi-row insert, SUCCESS_NO_INFO (-2) is returned instead. This change expands allowable results values to be 1 AND -2

Comparison of CockroachDB, MariaDB, Postgres was run to validate batchrewrite does provide performance improvements. NOTE: No attempts have been made to get optimal performance of each databases. Relative performance gains from each technique -- no batch, batch, batch rewritten to multi-row inserts -- are demonstrated.

All tests were performed on the same single MacBook

• CockroachDB v2.1.0-alpha.20180730 runs default serializable isolation with 3 way replicas.
• MariaDB 10.3.8-MariaDB Homebrew runs default REPEATABLE-READ
• Postgres 10.5 runs default read committed

Below summarizes Throughput(ops/sec) loading 100,000 rows.

database	nobatch	batch	batchrewrite
cockraochdb	339	495	3859
mariadb	998	563	4154
postgres	2137	8140	9254

• CockroachDB

cockroach sql --insecure -e "truncate usertable"

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1:26257/defaultdb?autoReconnect=true&sslmode=disable&ssl=false" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=false -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar >  cockraochdb.nobatch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1:26257/defaultdb?autoReconnect=true&sslmode=disable&ssl=false" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar > cockraochdb.batch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1:26257/defaultdb?autoReconnect=true&sslmode=disable&ssl=false&reWriteBatchedInserts=true" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar  > cockraochdb.batchrewrite.log

• MariaDB

mysql -u root -D ycsb -e "truncate usertable"

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=com.mysql.jdbc.Driver -p db.url="jdbc:mysql://localhost/ycsb" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=false -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/mysql-connector-java-5.1.47.jar > mariadb.nobatch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=com.mysql.jdbc.Driver -p db.url="jdbc:mysql://localhost/ycsb" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/mysql-connector-java-5.1.47.jar > mariadb.batch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=com.mysql.jdbc.Driver -p db.url="jdbc:mysql://localhost/ycsb?rewriteBatchedStatements=true" -p db.user=root -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/mysql-connector-java-5.1.47.jar > mariadb.batchrewrite.log 

• Postgres

psql -U postgres -c "truncate usertable"

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1/?autoReconnect=true&sslmode=disable&ssl=false" -p db.user=postgres -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=false -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar > postgres.nobatch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1/?autoReconnect=true&sslmode=disable&ssl=false" -p db.user=postgres -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar > postgres.batch.log

bin/ycsb load jdbc -s -P workloads/workloada -p db.driver=org.postgresql.Driver -p db.url="jdbc:postgresql://127.0.0.1/?autoReconnect=true&sslmode=disable&ssl=false&reWriteBatchedInserts=true" -p db.user=postgres -p db.passwd="" -p db.batchsize=1000  -p jdbc.fetchsize=10 -p jdbc.autocommit=true -p jdbc.batchupdateapi=true -p db.batchsize=1000 -p recordcount=100000 -cp ~/Downloads/postgresql-42.2.4.jar > postgres.batchrewrite.log

Logs of each runs below:

cockraochdb.batch.log
cockraochdb.batchrewrite.log
cockraochdb.nobatch.log
mariadb.batch.log
mariadb.batchrewrite.log
mariadb.nobatch.log
postgres.batch.log
postgres.batchrewrite.log
postgres.nobatch.log

[busbey]

this is great! thanks for all the up front testing work.

would you mind adding a note to the JDBC README.md about setting the needed connection parameter to allow multi-row insert?

[busbey]

thanks again, this is great. I will probably ping you once the next release gets going to make sure everything works as expected for the JDBC driver then.