Skip to content

Conversation

cb1kenobi
Copy link
Contributor

This sets the timestamp in RocksDB when the commit happens. The NativeTransaction's commit() and commitSync() will return this timestamp. This timestamp is currently internal only (e.g. db.transaction() does not surface this timestamp). Only consumers of the Transaction class can get this timestamp.

Copy link

📊 Benchmark Results

encoding.bench.ts

Key encoding > ordered-binary keys - strings (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 24.78K ops/sec 40.35 35.37 673.29 0.762 12,392
🥈 rocksdb 2 4.58K ops/sec 218.377 197.466 2,393.45 1.33 2,290

Key encoding > ordered-binary keys - numbers (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 27.84K ops/sec 35.92 32.68 552.937 0.897 13,920
🥈 rocksdb 2 4.66K ops/sec 214.426 190.692 667.218 0.650 2,332

Key encoding > ordered-binary keys - mixed types (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 24.18K ops/sec 41.36 35.83 523.565 1.00 12,090
🥈 rocksdb 2 4.58K ops/sec 218.554 199.23 3,023.496 1.20 2,288

Value encoding > msgpack values - strings (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 15.75K ops/sec 63.48 55.21 724.339 0.791 7,877
🥈 rocksdb 2 4.24K ops/sec 236.078 218.043 643.553 0.476 2,118

Value encoding > msgpack values - numbers (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 26.30K ops/sec 38.03 34.74 510.235 0.567 13,150
🥈 rocksdb 2 4.78K ops/sec 209.028 188.609 3,404.698 1.40 2,393

Value encoding > msgpack values - arrays (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 9.14K ops/sec 109.354 97.06 1,104.158 1.08 4,573
🥈 rocksdb 2 3.41K ops/sec 293.605 269.652 935.48 0.669 1,703

Value encoding > msgpack values - small objects (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 5.91K ops/sec 169.121 153.501 714.969 0.796 2,957
🥈 rocksdb 2 2.62K ops/sec 382.011 346.309 2,033.407 0.928 1,309

Value encoding > msgpack values - large objects (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 634.73 ops/sec 1,575.482 1,437.574 4,956.639 2.90 318
🥈 rocksdb 2 623.87 ops/sec 1,602.893 1,434.291 3,269.87 1.98 312

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 20.99K ops/sec 47.65 41.78 1,803.721 0.932 10,494
🥈 rocksdb 2 4.21K ops/sec 237.334 203.364 11,274.527 6.74 2,107

get.bench.ts

get() > rocksdb - random vs sequential keys (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 sequential 1 3.98K ops/sec 251.05 227.181 1,138.619 0.747 1,992
🥈 random 2 3.88K ops/sec 257.515 232.393 1,064.393 0.909 1,942

get() > random keys - max 1978 lmdb key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 3.17K ops/sec 315.789 281.1 2,285.532 1.90 1,584

get() > rocksdb - async vs sync

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 sync 1 4.16K ops/sec 240.636 214.073 3,705.502 2.03 2,078
🥈 async 2 3.64K ops/sec 274.814 249.983 966.482 0.547 1,820

put-sync.bench.ts

putSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 2.18K ops/sec 459.413 431.324 702.31 0.370 1,089
🥈 lmdb 2 4.37 ops/sec 228,637.475 217,746.419 251,071.304 2.86 10.00

putSync() > random keys - max 1978 lmdb key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 977.23 ops/sec 1,023.295 917.216 1,338.984 0.486 489
🥈 lmdb 2 4.21 ops/sec 237,704.138 227,586.943 258,684.863 2.85 10.00

putSync() > sequential keys (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 981.12 ops/sec 1,019.243 446.574 268,354.148 104.917 491
🥈 lmdb 2 4.37 ops/sec 228,641.713 222,292.631 242,926.153 2.37 10.00

putSync() > put 100KB value (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 50.47 ops/sec 19,815.303 18,147.219 21,302.795 2.35 26.00
🥈 lmdb 2 3.28 ops/sec 305,170.756 280,403.62 328,276.108 3.41 10.00

putSync() > put 1MB value (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 5.23 ops/sec 191,276.273 183,395.918 208,629.602 2.66 10.00
🥈 lmdb 2 1.82 ops/sec 548,838.401 496,666.551 591,442.236 4.54 10.00

putSync() > get 10MB value (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 0.36 ops/sec 2,777,337.849 2,092,950.441 3,449,779.1 9.67 10.00
🥈 lmdb 2 0.26 ops/sec 3,844,695.204 3,163,733.024 4,176,710.672 6.98 10.00

put.bench.ts

put > small dataset (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 1.85K ops/sec 539.22 495.999 924.454 0.545 928
🥈 lmdb 2 4.03 ops/sec 248,412.072 234,673.59 267,685.304 3.81 10.00

put > async vs sync overhead

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 sync 1 2.12K ops/sec 470.906 446.52 635.504 0.263 1,062
🥈 async 2 1.91K ops/sec 523.91 479.584 979.496 0.407 955

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 23.24K ops/sec 43.02 36.64 895.812 1.18 11,623
🥈 rocksdb 2 2.75K ops/sec 363.909 289.149 2,542.558 2.74 1,374

getRange() > full scan vs range scan

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb range scan 1 21.48K ops/sec 46.55 39.53 1,017.251 1.30 10,742
🥈 lmdb full scan 2 12.06K ops/sec 82.89 75.01 773.589 1.05 6,033
🥉 rocksdb range scan 3 3.05K ops/sec 328.352 281.278 1,640.241 1.90 1,524
rocksdb full scan 4 1.64K ops/sec 609.532 523.52 3,462.473 2.79 821

getKeys() > keys only (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 50.08K ops/sec 19.97 16.55 3,363.161 1.96 25,038
🥈 rocksdb 2 5.01K ops/sec 199.458 177.808 880.06 1.13 2,507

Reverse iteration > reverse range (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 552.76K ops/sec 1.81 1.16 6,423.266 7.27 276,378
🥈 rocksdb 2 2.98K ops/sec 335.408 297.723 1,217.309 1.40 1,491

Reverse iteration > rocksdb - reverse vs forward

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 forward 1 3.12K ops/sec 320.926 281.762 5,173.433 2.63 1,558
🥈 reverse 2 2.90K ops/sec 344.265 307.403 2,286.312 2.09 1,453

Range query patterns > prefix scan performance

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 31.35K ops/sec 31.90 27.72 527.701 0.895 15,675
🥈 rocksdb 2 3.35K ops/sec 298.642 268.648 1,476.731 1.55 1,675

Sparse data patterns > sparse - range over gaps

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 61.89K ops/sec 16.16 14.20 565.412 0.617 30,944
🥈 rocksdb 2 6.22K ops/sec 160.887 144.553 1,176.813 1.11 3,108

Sparse data patterns > sparse - prefix with gaps

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 133.16K ops/sec 7.51 6.47 582.337 0.587 66,581
🥈 rocksdb 2 12.47K ops/sec 80.21 70.91 1,513.137 0.778 6,234

remove-sync.bench.ts

removeSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 12.39K ops/sec 80.69 69.90 692.072 1.08 6,197
🥈 rocksdb 2 2.66K ops/sec 375.241 347.664 529.725 0.305 1,333

removeSync() > sequential keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 13.07K ops/sec 76.49 70.49 720.944 0.867 6,538
🥈 rocksdb 2 2.65K ops/sec 377.74 360.465 632.075 0.253 1,324

removeSync() > rocksdb - random vs sequential keys (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 random 1 2.66K ops/sec 375.743 354.112 576.143 0.287 1,331
🥈 sequential 2 2.66K ops/sec 376.372 344.815 881.052 0.355 1,329

removeSync() > random keys - max 1978 lmdb key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 10.33K ops/sec 96.76 88.37 628.755 0.852 5,168
🥈 rocksdb 2 1.09K ops/sec 915.281 822.703 1,235.314 0.500 547

removeSync() > random access pattern (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 11.93K ops/sec 83.84 74.95 663.297 0.986 5,964
🥈 rocksdb 2 2.64K ops/sec 379.276 357.596 537.38 0.275 1,319

removeSync() > non-existent keys (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 12.27K ops/sec 81.47 75.94 381.824 0.429 6,138
🥈 rocksdb 2 2.72K ops/sec 367.734 346.81 555.634 0.268 1,360

transaction-sync.bench.ts

transaction sync > optimistic > simple put operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 669.72 ops/sec 1,493.168 1,383.824 5,221.072 1.98 335
🥈 lmdb 2 4.38 ops/sec 228,119.22 220,680.144 233,727.548 1.32 10.00

transaction sync > optimistic > batch operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 2.82K ops/sec 354.107 306 1,875.957 0.685 1,413
🥈 lmdb 2 350.20 ops/sec 2,855.522 2,068.79 8,623.827 5.85 176

transaction sync > optimistic > read-write operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 377.00 ops/sec 2,652.554 2,320.977 5,318.994 2.21 189
🥈 lmdb 2 4.39 ops/sec 227,899.02 216,335.011 249,135.328 2.88 10.00

transaction sync > optimistic > concurrent non-conflicting operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 643.24 ops/sec 1,554.619 1,463.337 4,121.015 1.39 322
🥈 lmdb 2 4.43 ops/sec 225,639.899 217,034.09 250,951.26 3.08 10.00

transaction sync > optimistic > rollback operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 16.09K ops/sec 62.17 59.07 313.761 0.360 8,043
🥈 rocksdb 2 5.49K ops/sec 182.291 172.28 695.218 0.345 2,743

transaction sync > optimistic > rocksdb - large transaction vs many small

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 2.94K ops/sec 340.267 297.73 522.932 0.336 1,470
🥈 rocksdb 2 705.68 ops/sec 1,417.071 1,352.915 4,535.452 1.73 354

transaction sync > optimistic > lmdb - large transaction vs many small

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 384.24 ops/sec 2,602.537 2,152.194 4,972.766 1.50 193
🥈 lmdb 2 4.56 ops/sec 219,195.24 213,709.373 226,547.907 1.52 10.00

transaction sync > optimistic > empty transaction overhead

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 782.40K ops/sec 1.28 1.10 1,693.868 0.727 391,199
🥈 rocksdb 2 113.99K ops/sec 8.77 7.52 1,619.101 1.21 56,994

transaction sync > optimistic > transaction with only reads (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 1.63K ops/sec 612.454 594.155 1,054.886 0.390 817
🥈 rocksdb 2 280.91 ops/sec 3,559.918 3,347.536 11,455.199 3.67 141

transaction sync > pessimistic > simple put operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 630.42 ops/sec 1,586.245 1,524.476 1,891.468 0.376 316
🥈 lmdb 2 4.38 ops/sec 228,087.204 220,874.55 247,222.759 2.44 10.00

transaction.bench.ts

transaction > optimistic > simple put operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 338.86 ops/sec 2,951.048 2,444.197 5,874.852 3.36 170
🥈 lmdb 2 3.78 ops/sec 264,733.959 245,969.365 300,643.493 4.32 10.00

transaction > optimistic > batch operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 2.25K ops/sec 443.731 351.086 595.978 0.393 1,127
🥈 lmdb 2 347.07 ops/sec 2,881.256 2,273 3,688.447 2.06 174

transaction > optimistic > read-write operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 137.36 ops/sec 7,279.95 6,240.07 15,186.185 3.87 69.00
🥈 lmdb 2 3.63 ops/sec 275,462.94 261,235.319 292,764.877 3.02 10.00

transaction > optimistic > concurrent non-conflicting operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 947.39 ops/sec 1,055.527 879.367 4,544.681 3.63 474
🥈 lmdb 2 329.25 ops/sec 3,037.201 2,255.928 6,576.614 2.51 165

transaction > optimistic > rollback operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 4.82K ops/sec 207.438 191.855 865.086 0.566 2,411
🥈 lmdb 2 234.28 ops/sec 4,268.397 2,104.752 43,760.353 27.40 122

transaction > optimistic > rocksdb - large transaction vs many small

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 2.21K ops/sec 451.686 337.174 913.907 0.507 1,107
🥈 rocksdb 2 307.48 ops/sec 3,252.209 2,477.742 7,754.838 3.74 154

transaction > optimistic > lmdb - large transaction vs many small

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 324.49 ops/sec 3,081.796 2,410.793 8,094.767 2.55 163
🥈 lmdb 2 3.84 ops/sec 260,745.039 245,697.588 283,256.673 3.15 10.00

transaction > optimistic > empty transaction overhead

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 43.63K ops/sec 22.92 15.45 517.06 0.416 21,816
🥈 rocksdb 2 35.53K ops/sec 28.15 16.72 4,345.284 3.07 17,764

transaction > optimistic > transaction with only reads (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 265.32 ops/sec 3,769.098 2,867.374 13,086.113 10.18 133
🥈 rocksdb 2 130.48 ops/sec 7,663.919 6,202.354 15,819.935 6.39 66.00

transaction > pessimistic > simple put operations (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 265.51 ops/sec 3,766.352 2,863.415 13,953.058 9.16 133
🥈 lmdb 2 3.81 ops/sec 262,609.654 244,738.986 308,751.502 5.16 10.00

worker-get-sync.bench.ts

Worker > random keys - small key size (100 records, 1 worker)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 13.59K ops/sec 73.60 59.34 6,821.131 3.30 6,794
🥈 rocksdb 2 3.71K ops/sec 269.719 222.987 1,386.933 1.06 1,854

Worker > random keys - small key size (100 records, 2 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 14.79K ops/sec 67.59 61.33 4,085.429 1.77 7,398
🥈 rocksdb 2 3.52K ops/sec 283.958 232.058 2,737.511 1.40 1,761

Worker > random keys - small key size (100 records, 10 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 7.54K ops/sec 132.598 116.777 5,955.646 2.34 3,771
🥈 rocksdb 2 1.68K ops/sec 596.894 493.024 2,897.823 2.48 838

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 1 worker)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 1.83K ops/sec 546.547 502.865 2,281.351 1.04 915
🥈 lmdb 2 4.19 ops/sec 238,872.54 228,582.438 263,257.487 3.08 10.00

putSync() > random keys - small key size (100 records, 2 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 1.37K ops/sec 730.443 667.939 2,312.386 0.830 685
🥈 lmdb 2 2.07 ops/sec 483,967.477 466,342.467 520,507.587 2.41 10.00

putSync() > random keys - small key size (100 records, 10 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 781.25 ops/sec 1,280 1,090.217 4,324.858 2.06 391
🥈 lmdb 2 1.02 ops/sec 984,821.577 893,525.367 1,086,775.882 5.88 10.00

Results from commit 1da8edc

Copy link
Member

@kriszyp kriszyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to be able to access the commit timestamp on gets as well, but that also may be a different PR/effort.
Anyway, this looks like great progress, and maybe you already have this stuff planned.

state->status = rocksdb::Status::Aborted("Database closed during transaction commit operation");
} else {
// set current timestamp before committing
auto now = std::chrono::system_clock::now();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this should be part of this PR or a subsequent PR for transaction log, but:

  • The timestamp should be unique; we need to track the last commit timestamp used, and if it is identical to (or greater than) now, we need to increment it by the smallest increment possible in a double representation (which will require a mutex).
  • There should be an option to specify the commit timestamp, presumably from the TransactionOptions that are passed in. This is important for situations like replication where we are replicated the state of transaction that has already been committed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants