Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sqlite: Initial benchmark for putTxHistory #341

Merged
merged 15 commits into from
Jun 4, 2019
Merged

Sqlite: Initial benchmark for putTxHistory #341

merged 15 commits into from
Jun 4, 2019

Conversation

rvl
Copy link
Contributor

@rvl rvl commented May 29, 2019

Relates to issue #154.

Overview

  • This starts off the benchmarking of the Sqlite DBLayer.

Comments

  • It turns out putTxHistory performance sucks when running it in batches of 10 (no surprise).
  • This benchmark has uncovered a bug in the removeWallet code ... our old friend (?,?,...,?): too many SQL variables.

@rvl rvl self-assigned this May 29, 2019
@rvl rvl force-pushed the rvl/154/db-bench branch 3 times, most recently from 86ed114 to 1520539 Compare June 2, 2019 23:42
@KtorZ KtorZ force-pushed the rvl/154/db-bench branch from 1520539 to dbde910 Compare June 3, 2019 15:46
@rvl rvl force-pushed the rvl/154/db-bench branch from 301e360 to c576c8c Compare June 4, 2019 05:56
@KtorZ KtorZ force-pushed the rvl/154/db-bench branch from 8bdf24e to 97c88df Compare June 4, 2019 15:55
rvl and others added 14 commits June 4, 2019 18:57
This lays out benchmarks in a table, with comments describing the
parameters in that column.

Using a record type doesn't enhance readability at all.
The function repsertMany does what I thought putMany did. When there
is a conflict it updates the rows. This means that we don't need to
delete before inserting.
It would have exploded if there were more than 999 TxMeta or
PendingTx.

I tried Esqueleto and it can't do this query. Actually, it could
express the query, but the generated SQL was invalid. And the code
looked just as bad as raw SQL.
@KtorZ KtorZ force-pushed the rvl/154/db-bench branch from 97c88df to 34c23eb Compare June 4, 2019 16:58
Copy link
Member

@KtorZ KtorZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few top-level comments + some extra dbChunked on insertMany. Merging as it is so we can better iterate on it.

cc @rvl

@KtorZ KtorZ merged commit d7e6fd6 into master Jun 4, 2019
@iohk-bors iohk-bors bot deleted the rvl/154/db-bench branch June 4, 2019 17:28
@KtorZ KtorZ restored the rvl/154/db-bench branch June 4, 2019 17:45
@KtorZ KtorZ removed this from the SQLite implementation for the DB Layer milestone Jun 5, 2019
@KtorZ KtorZ deleted the rvl/154/db-bench branch June 7, 2019 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants