Single round trip commit on BatchExecute

[rafael]

Description

• When auto-commit is on, passDML is on and ExecuteBatch is in a transaction,
  there is no need to explicitly create a transaction. We can forward the DML
  directly to the database.
• This optimization yielded significant more throughput in vttablets. We got
  around 25-30% improvement. Most of our queries are single point
  inserts/updates that already use auto commit when coming from vtgates, so this
  improvement is something that we've been wanting to do for a long time.

Benchmark

Setup

• Two vtgates generating traffic to a keyspace with a single shard master. Note: Using more vtgates than this, did not yield more throughput. After this point, latency just increase with more concurrency.
• Benchmark was run using vtbench.
• Tests were running using a table with/without a BLACKHOLE. Results below.

Results

TL;DR

• 20% Improvement in throughput capacity for our tablet configuration. Max write throughput goes from around 5.5K to 6.5K. If you also turn of query logging, this goes up close to 30%.
• Results are not much different with or without the blackhole engine. This I don't fully understand why.

Prior to this change BLACKHOLE ENGINE:

The following parameters where passed to vtbench:

vtbench -host xxxx -protocol mysql -port xxx -user xxx -db-credentials-file ./vtbench.creds -db @master --count 30000 --threads 25 -sql INSERT INTO blackhole_engine_test_table (i,c)  VALUES(1,'record one') /* vt_bench:thread */
Initializing test with mysql protocol / 25 threads / 30000 iterations

Initializing test with mysql protocol / 25 threads / 30000 iterations
Starting test threads
Average Rows Returned: 1
Average Query Time: 8.66276ms
Total Test Time: 4m20.365234606s
QPS (Per Thread): 115.22275639218026
QPS (Total): 2880.5689098045063
Query Timings:
1ms-5ms: 37891
5ms-10ms: 520573
10ms-50ms: 191315
50ms-100ms: 46
100ms-500ms: 175

Prior to this change REAL ENGINE:

The following parameters where passed to vtbench:

 vtbench -host xxxx -protocol mysql -port xxx -user xxx -db-credentials-file xxxx -db @master --count 30000 --threads 25 -sql INSERT INTO test_table (i,c)  VALUES(1,'record one') /* vt_bench:thread */

Initializing test with mysql protocol / 25 threads / 30000 iterations
Starting test threads
Average Rows Returned: 1
Average Query Time: 8.960426ms
Total Test Time: 4m29.269666645s
QPS (Per Thread): 111.41247498757976
QPS (Total): 2785.311874689494
Query Timings:
1ms-5ms: 35800
5ms-10ms: 483292
10ms-50ms: 230707
50ms-100ms: 1
100ms-500ms: 200

New Build BLACKHOLE ENGINE:

The following parameters where passed to vtbench:

vtbench -host xxxx -protocol mysql -port xxx -user xxx -db-credentials-file xxx -db @master --count 30000 --threads 25 -sql INSERT INTO blackhole_engine_test_table (i,c)  VALUES(1,'record one') /* vt_bench:thread */

Initializing test with mysql protocol / 25 threads / 30000 iterations
Starting test threads
Average Rows Returned: 1
Average Query Time: 7.516542ms
Total Test Time: 3m45.847952647s
QPS (Per Thread): 132.8327294907559
QPS (Total): 3320.8182372688975
Query Timings:
1ms-5ms: 75114
5ms-10ms: 596264
10ms-50ms: 78470
50ms-100ms: 2
100ms-500ms: 150

New Build REAL ENGINE:

The following parameters where passed to vtbench:

+ vtbench -hostxxx -protocol mysql -port xxx -user xxx -db-credentials-file xxxx -db @master --count 30000 --threads 25 -sql INSERT INTO blackhole_engine_test_table (i,c)  VALUES(1,'record one') /* vt_bench:thread */

Initializing test with mysql protocol / 25 threads / 30000 iterations
Starting test threads
Average Rows Returned: 1
Average Query Time: 7.638981ms
Total Test Time: 3m49.606864281s
QPS (Per Thread): 130.65811466021793
QPS (Total): 3266.4528665054486
Query Timings:
1ms-5ms: 74750
5ms-10ms: 585007
10ms-50ms: 90068
50ms-100ms: 21
100ms-500ms: 154```

[sougou]

An alternate approach I was thinking is to add a new txIsolations entry called AUTOCOMMIT, and it will just not issue a begin (and skip commit).

And then you can use a LocalBegin-LocalCommit in that mode, which will end up doing exactly what you want.

Advantage is that it's a minor code change and less duplication. But I'm not sure if it will be construed as too much magic.

[sougou]

TabletServer has a SetPassthroughDML call. Were you not able to just switch it for this endtoend test?

[demmer]

changes look good except for a couple small typos we should clean up and some kind of better testing.