Skip to content

feat: share mmap handle inbetween snapshot providers & cursors#5162

Merged
joshieDo merged 59 commits intomainfrom
joshie/snapshot-prov-mod
Oct 27, 2023
Merged

feat: share mmap handle inbetween snapshot providers & cursors#5162
joshieDo merged 59 commits intomainfrom
joshie/snapshot-prov-mod

Conversation

@joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Oct 24, 2023

PR into #5139

  • adds a few helper types: MmapHandle (struct over file and mmap arc) and LoadedJar (struct over mmap handle and jar)
  • use above to share file/mmap handles across snapshot/cursor providers
  • smol refactor: split snapshot provider into multiple files

@joshieDo joshieDo requested a review from Rjected as a code owner October 24, 2023 15:04
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only style nits,
lgtm otherwise


// SAFETY: File is read-only and its descriptor is kept alive as long as the mmap handle.
let mmap = unsafe { Mmap::map(&file)? };
pub fn new(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs some docs about when to pass an option, but I'd prefer a separate function with_handle that directly accepts the mmapHandle

Comment on lines +48 to +51
let header = Header::decompress(
cursor.row_by_key_with_cols::<0b01, 2>(&block_hash.0).unwrap().unwrap()[0],
)
.unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these should be errors, but fine with cleaning this up separately

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, next PR is actually implementing the HeaderProvider, and will clean this up

Base automatically changed from joshie/sprov to main October 27, 2023 10:16
@codecov
Copy link

codecov bot commented Oct 27, 2023

Codecov Report

Merging #5162 (6bf07fd) into main (4dc15c3) will increase coverage by 35.61%.
Report is 25 commits behind head on main.
The diff coverage is 24.53%.

❗ Current head 6bf07fd differs from pull request most recent head ad289c8. Consider uploading reports for the commit ad289c8 to get more accurate results

Impacted file tree graph

Files Coverage Δ
crates/interfaces/src/provider.rs 100.00% <ø> (+100.00%) ⬆️
crates/primitives/src/transaction/mod.rs 73.62% <ø> (+36.86%) ⬆️
crates/stages/src/stages/tx_lookup.rs 96.28% <100.00%> (+94.10%) ⬆️
crates/storage/codecs/src/lib.rs 93.91% <ø> (+57.14%) ⬆️
crates/storage/db/src/snapshot.rs 84.31% <100.00%> (+84.31%) ⬆️
crates/storage/nippy-jar/src/error.rs 0.00% <ø> (ø)
...torage/provider/src/providers/database/provider.rs 77.32% <100.00%> (+49.84%) ⬆️
crates/storage/provider/src/providers/mod.rs 21.42% <ø> (-5.50%) ⬇️
crates/storage/nippy-jar/src/cursor.rs 85.71% <96.96%> (+85.71%) ⬆️
crates/storage/nippy-jar/src/lib.rs 93.55% <97.29%> (+93.55%) ⬆️
... and 15 more

... and 470 files with indirect coverage changes

Flag Coverage Δ
integration-tests ?
unit-tests 61.69% <24.53%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 28.64% <0.00%> (+2.86%) ⬆️
blockchain tree 80.82% <ø> (+52.36%) ⬆️
pipeline 88.20% <100.00%> (+83.15%) ⬆️
storage (db) 69.22% <46.18%> (+39.24%) ⬆️
trie 94.93% <ø> (+72.39%) ⬆️
txpool 45.41% <ø> (+4.03%) ⬆️
networking 71.27% <ø> (+40.37%) ⬆️
rpc 40.90% <ø> (+14.42%) ⬆️
consensus 62.95% <ø> (+37.88%) ⬆️
revm 23.36% <ø> (+13.51%) ⬆️
payload builder 3.39% <ø> (-10.77%) ⬇️
primitives 81.03% <11.86%> (+51.86%) ⬆️

@joshieDo joshieDo enabled auto-merge October 27, 2023 10:26
@joshieDo joshieDo added this pull request to the merge queue Oct 27, 2023
Merged via the queue into main with commit a2323c9 Oct 27, 2023
@joshieDo joshieDo deleted the joshie/snapshot-prov-mod branch October 27, 2023 10:36
mattsse pushed a commit that referenced this pull request Nov 8, 2023
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants