core/rawdb: implement sequential reads in freezer_table#23117
Merged
karalabe merged 6 commits intoethereum:masterfrom Aug 13, 2021
Merged
core/rawdb: implement sequential reads in freezer_table#23117karalabe merged 6 commits intoethereum:masterfrom
karalabe merged 6 commits intoethereum:masterfrom
Conversation
Contributor
Author
|
I made a little benchmark tool for this, in this branch. Using the old reads, where each header is read separately, and causing 3 syscalls per header (so It speeds up continously (it's on a NUC with pretty hefty RAM, so the disk reads will be better cached) but after three and a half minute, it's reached If the sequential-read mechanism in this PR is used instead, which does |
karalabe
suggested changes
Aug 3, 2021
Member
karalabe
left a comment
There was a problem hiding this comment.
Mostly nitpicks, but there's one corner case that I believe is broken.
8f1453f to
821c696
Compare
atif-konasl
pushed a commit
to frozeman/pandora-execution-engine
that referenced
this pull request
Oct 15, 2021
* core/rawdb: implement sequential reads in freezer_table * core/rawdb, ethdb: add sequential reader to db interface * core/rawdb: lint nitpicks * core/rawdb: fix some nitpicks * core/rawdb: fix flaw with deferred reads not being performed * core/rawdb: better documentation
This was referenced Sep 23, 2022
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a bit of a successor to #20308, which implemented sequential access to freezer data via an iterator interface.
This PR instead returns a slice of data. It does not defer reads until
Next(), so instead does all the reads while holding the readlock.It also switches the implementation of
Retrieveto be a just a short-cut for doing a sequential read with max items set to1. This PR doesn't really add any uses of the new method (except theRetrieve), but does actually improve one thing as-is: previously, two syscalls were performed to read two index items from the index-file. This PR instead reads all (both) index items using only one syscall.Later on, this can be used to serve
eth64data such as headers directly from ancients with a minimum number of syscalls. If that is done, we can get by with2syscalls per "192-header-delivery" instead of576.