Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth, trie/triedb/pathdb: pbss patches #1955

Merged
merged 4 commits into from
Nov 6, 2023
Merged

eth, trie/triedb/pathdb: pbss patches #1955

merged 4 commits into from
Nov 6, 2023

Conversation

joeylichang
Copy link
Contributor

@joeylichang joeylichang commented Oct 31, 2023

Description

This PR fixes the stability and performance of PBSS, opt the add asyncnodebuffer option for flushing disk(it non-block state commit), fix rewind root hash mismatch.

Rationale

Example

--pathdb.sync use the origin nodebuffer, the defaulte use the asyncnodebuffer.

Changes

Notable changes:

  • trie/triedb/pathdb
  • core

@joeylichang joeylichang added the wip work in process label Oct 31, 2023
@joeylichang joeylichang changed the title Pbss fix [Testing] Oct 31, 2023
@joeylichang joeylichang force-pushed the pbss_fix branch 4 times, most recently from d8a631f to cea8fa4 Compare October 31, 2023 10:20
@joeylichang joeylichang changed the title [Testing] eth, trie/triedb/pathdb: pbss patches Oct 31, 2023
@joeylichang joeylichang added r4r ready for review and removed wip work in process labels Oct 31, 2023
eth/backend.go Show resolved Hide resolved
trie/triedb/pathdb/disklayer.go Outdated Show resolved Hide resolved
trie/triedb/pathdb/disklayer.go Show resolved Hide resolved
trie/triedb/pathdb/nodebuffer.go Show resolved Hide resolved
@brilliant-lx
Copy link
Collaborator

1.DefaultDirtyBufferSize 64MB, it is conflicted with the current cache allocation, which allocate 25% cache to dirty cache by default.

cmd/geth/snapshot.go Outdated Show resolved Hide resolved
@joeylichang
Copy link
Contributor Author

1.DefaultDirtyBufferSize 64MB, it is conflicted with the current cache allocation, which allocate 25% cache to dirty cache by default.

PBSS nodebuffer will aggregate the nodes of multiple difflayers together, so the nodebuffer will be written in one batch, so it should not be too large. MaxDirtyBufferSize and DefaultDirtyBufferSize are both verified and considered values.

@joeylichang joeylichang force-pushed the pbss_fix branch 3 times, most recently from 4c80518 to becbe8f Compare November 6, 2023 03:03
Copy link

@RenRick RenRick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@buddh0 buddh0 added duplicate This issue or pull request already exists and removed duplicate This issue or pull request already exists labels Nov 6, 2023
@brilliant-lx brilliant-lx merged commit 53559fc into develop Nov 6, 2023
5 checks passed
@joeylichang joeylichang deleted the pbss_fix branch November 7, 2023 03:34
whw188 pushed a commit to tylerteea/mev-bsc that referenced this pull request Dec 6, 2023
* fix: use the top root hash for rewinding under path schema

* feat: add async flush nodebuffer in path schema

* chore: add prun-block param suffix check

* fix: code review comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
r4r ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants