Skip to content

Batch reuse#15989

Merged
karalabe merged 2 commits into
ethereum:masterfrom
holiman:batch_reuse
Jan 30, 2018
Merged

Batch reuse#15989
karalabe merged 2 commits into
ethereum:masterfrom
holiman:batch_reuse

Conversation

@holiman
Copy link
Copy Markdown
Contributor

@holiman holiman commented Jan 29, 2018

This PR goes on top of #15988. This PR exposes the Reset() interface from leveldb.Batch, implements it for the relevant types, and uses it during block import.

This should decrease the number of allocations and make the gc work less. I'll add charts to this PR to see if it makes any difference.

@holiman holiman requested a review from karalabe as a code owner January 29, 2018 14:05
@holiman
Copy link
Copy Markdown
Contributor Author

holiman commented Jan 29, 2018

So, some graphs. Here's geth master:
geth_master
And here's this PR:
geth_pr_with_reset

No big differences, biggest seems to be that the latter reaches the 'silent area' (when nothing is imported for a few minutes, due to db compaction (?)) around 1750s, and finishes it in ~300s, whereas geth-master for some reason didn't reach it until ~1800s, and didn't resume until around 1000s later.

Can't say for sure it's due to the code changes though.

@holiman holiman requested a review from fjl January 30, 2018 12:35
Copy link
Copy Markdown
Member

@karalabe karalabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@karalabe karalabe added this to the 1.8.0 milestone Jan 30, 2018
@karalabe karalabe merged commit 017b9f7 into ethereum:master Jan 30, 2018
@holiman
Copy link
Copy Markdown
Contributor Author

holiman commented Jan 30, 2018

With the added patch to set the size of the batch correctly:

image

And a bit zoomed in:

image

prestonvanloon pushed a commit to prestonvanloon/go-ethereum that referenced this pull request Apr 2, 2018
* leveldb: Update leveldb to 211f780 (poolfix)

* core, ethdb: reuse database batches
mariameda pushed a commit to NiluPlatform/go-nilu that referenced this pull request Aug 23, 2018
* leveldb: Update leveldb to 211f780 (poolfix)

* core, ethdb: reuse database batches
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants