-
Notifications
You must be signed in to change notification settings - Fork 848
feat(evm/sync): implement dynamic state sync orchestration #5051
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
powerslider
wants to merge
59
commits into
master
Choose a base branch
from
powerslider/dynamic-state-sync-poc
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
525903a
feat(evm/sync): implement dynamic state sync orchestration
powerslider 7b62f83
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 9246b21
feat(evm/sync): add sessioned code queue for pivot-anytime dynamic st…
powerslider 23af203
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider a15da5a
fix: lint error
powerslider cfce189
fix(evm/sync): prevent re-entrancy infinite loop during batch replay
powerslider a51375a
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 94d7cf8
feat(evm/sync): harden coordinator state transitions and update seria…
powerslider 74c1d27
feat(sync): add block syncer UpdateTarget with bounded catch-up
powerslider 2847a9a
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 944845b
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 97b1107
refactor(evm/sync): implement dynamic sync as a lifecycle wrapper aro…
powerslider 6789c82
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider e2915dc
feat(sync): enable atomic syncer to operate within dynamic sync
powerslider c1ed252
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 266e826
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider f4d4da4
feat(sync): enable firewood syncer for dynamic state sync
powerslider 4df5cfe
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider af5c78d
chore: regenerate Bazel metadata
powerslider eb4e98e
test(sync): add pivot and fault-injection coordinator tests
powerslider d139492
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider e1d85b8
build(session) add BUILD.bazel
powerslider 2d20520
build: re-generate bazel metadata
powerslider 09d1dab
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 9d4049e
fix(sync): prevent double-accept of commit-target block during batch …
powerslider df38656
test(sync): add VM-level block injection test for dynamic state sync
powerslider aaf03b5
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 8621572
test(sync): add dynamic sync block injection test for subnet-evm and …
powerslider a1a72f0
fix(sync): use test deadline for gated AppRequests
powerslider 9c63664
feat(sync): add dynamic pivot support for atomic syncer
powerslider 0dea4f5
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 7a046aa
debug(sync): log sync client error on block injection failure
powerslider e4dcf65
fix(sync): inject blocks inside response interceptor to prevent sched…
powerslider 4d849cc
fix(sync): reconcile chain.State after batch replay for correct engin…
powerslider e1b15aa
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 11e04ff
refactor(sync): remove TargetReporter since all syncers now pivot tog…
powerslider 503e5c2
refactor(sync): simplify dynamic code syncer by removing session infr…
powerslider 2148d63
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 2a66ff8
fix(sync): prevent commit target from advancing after syncers complete
powerslider 7f60ee1
refactor(sync): reorder coordinator methods and tighten comments
powerslider 8663962
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider 6e4b514
fix(bootstrap): enable dynamic state sync pivots
powerslider 771fb40
fix(bootstrap): pass sync target height via ConsensusContext
powerslider cfaf433
fix(bootstrap): skip BootstrapTracker wait during dynamic state sync
powerslider 485035e
feat(bootstrap): skip bootstrapper during dynamic state sync
powerslider 78ad231
fix(bootstrap): bound ancestor walk at sync target height
powerslider 89946bf
fix(bootstrap): revert premature NormalOp transition
powerslider ccaa60d
refactor(proposervm): move sync target awareness to VM level
powerslider 2fd45d3
chore(sync): add coordinator state transition logging
powerslider 3d63829
fix(sync): more diagnostic logging
powerslider 54d04a6
fix(bootstrap): defer NormalOp until dynamic state sync completes
powerslider cad23ff
feat(sync): incremental storage trie pivoting for dynamic state sync
powerslider 4d6e8ea
chore(sync): add diagnostic logging for storage trie filter
powerslider 231ddbd
chore(sync): add more diagnostic logging
powerslider b4a79d1
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider fd49583
chore(sync): more diagnostics
powerslider 15a292e
fix(sync): clear only main trie segments on pivot, keep storage trie …
powerslider 9622af3
fix(sync): clear stale main trie segments on pivot in dynamic syncer
powerslider 2b5d1da
Merge branch 'master' into powerslider/dynamic-state-sync-poc
powerslider File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| // Copyright (C) 2019, Ava Labs, Inc. All rights reserved. | ||
| // See the file LICENSE for licensing terms. | ||
|
|
||
| package sync | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/ava-labs/libevm/common" | ||
| "github.com/ava-labs/libevm/log" | ||
|
|
||
| "github.com/ava-labs/avalanchego/database/versiondb" | ||
| "github.com/ava-labs/avalanchego/graft/evm/sync/types" | ||
|
|
||
| atomicstate "github.com/ava-labs/avalanchego/graft/coreth/plugin/evm/atomic/state" | ||
| ) | ||
|
|
||
| const ( | ||
| atomicDynamicSyncerName = "Atomic State Syncer (dynamic)" | ||
| atomicDynamicSyncerID = "state_atomic_sync" | ||
| ) | ||
|
|
||
| var _ types.PivotSession = (*atomicPivotSession)(nil) | ||
|
|
||
| // atomicPivotSession implements types.PivotSession for atomic trie sync. | ||
| type atomicPivotSession struct { | ||
| inner *Syncer | ||
| client types.LeafClient | ||
| db *versiondb.Database | ||
| atomicTrie *atomicstate.AtomicTrie | ||
| opts []SyncerOption | ||
| } | ||
|
|
||
| func (s *atomicPivotSession) Run(ctx context.Context) error { | ||
| return s.inner.Sync(ctx) | ||
| } | ||
|
|
||
| func (*atomicPivotSession) ShouldPivot(common.Hash) bool { | ||
| // Always returns true because the incoming root is a block root | ||
| // (proxy), not the actual atomic root, so comparing it is meaningless. | ||
| return true | ||
| } | ||
|
|
||
| func (s *atomicPivotSession) Rebuild(newRoot common.Hash, newHeight uint64) (types.PivotSession, error) { | ||
| log.Info("atomic syncer pivoting", "newHeight", newHeight, "newRoot", newRoot) | ||
|
|
||
| if err := s.inner.Finalize(); err != nil { | ||
| log.Error("failed to flush atomic syncer during pivot", "err", err) | ||
| } | ||
| s.atomicTrie.ResetToLastCommitted() | ||
|
|
||
| newInner, err := NewSyncer(s.client, s.db, s.atomicTrie, newRoot, newHeight, s.opts...) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &atomicPivotSession{ | ||
| inner: newInner, | ||
| client: s.client, | ||
| db: s.db, | ||
| atomicTrie: s.atomicTrie, | ||
| opts: s.opts, | ||
| }, nil | ||
| } | ||
|
|
||
| func (*atomicPivotSession) OnSessionComplete() error { | ||
| return nil | ||
| } | ||
|
|
||
| // Finalize flushes the inner syncer's in-progress work. | ||
| func (s *atomicPivotSession) Finalize() error { | ||
| return s.inner.Finalize() | ||
| } | ||
|
|
||
| // NewAtomicDynamicSyncer creates a DynamicSyncer backed by an | ||
| // atomicPivotSession. The returned syncer supports pivoting to new | ||
| // targets during sync. | ||
| func NewAtomicDynamicSyncer( | ||
| inner *Syncer, | ||
| client types.LeafClient, | ||
| db *versiondb.Database, | ||
| atomicTrie *atomicstate.AtomicTrie, | ||
| initialRoot common.Hash, | ||
| initialHeight uint64, | ||
| opts ...SyncerOption, | ||
| ) *types.DynamicSyncer { | ||
| session := &atomicPivotSession{ | ||
| inner: inner, | ||
| client: client, | ||
| db: db, | ||
| atomicTrie: atomicTrie, | ||
| opts: opts, | ||
| } | ||
| return types.NewDynamicSyncer( | ||
| atomicDynamicSyncerName, | ||
| atomicDynamicSyncerID, | ||
| session, | ||
| initialRoot, | ||
| initialHeight, | ||
| ) | ||
| } |
51 changes: 51 additions & 0 deletions
51
graft/coreth/plugin/evm/atomic/sync/dynamic_syncer_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| // Copyright (C) 2019, Ava Labs, Inc. All rights reserved. | ||
| // See the file LICENSE for licensing terms. | ||
|
|
||
| package sync | ||
|
|
||
| import ( | ||
| "math/rand" | ||
| "testing" | ||
|
|
||
| "github.com/ava-labs/libevm/common" | ||
| "github.com/ava-labs/libevm/core/rawdb" | ||
| "github.com/ava-labs/libevm/triedb" | ||
| "github.com/stretchr/testify/require" | ||
|
|
||
| "github.com/ava-labs/avalanchego/graft/coreth/plugin/evm/atomic/state" | ||
| "github.com/ava-labs/avalanchego/graft/evm/sync/synctest" | ||
| "github.com/ava-labs/avalanchego/graft/evm/sync/types" | ||
| ) | ||
|
|
||
| // TestAtomicDynamicSyncer_CompletesWithoutPivot verifies that the dynamic | ||
| // wrapper completes a full sync when no pivot is triggered. | ||
| func TestAtomicDynamicSyncer_CompletesWithoutPivot(t *testing.T) { | ||
| targetHeight := 2 * uint64(testCommitInterval) | ||
| r := rand.New(rand.NewSource(1)) | ||
| serverTrieDB := triedb.NewDatabase(rawdb.NewMemoryDatabase(), nil) | ||
| root, _, _ := synctest.GenerateIndependentTrie(t, r, serverTrieDB, int(targetHeight), state.TrieKeyLength) | ||
|
|
||
| ctx, mockClient, atomicBackend, clientDB := setupTestInfrastructure(t, serverTrieDB) | ||
| atomicTrie := atomicBackend.AtomicTrie() | ||
|
|
||
| inner, err := NewSyncer(mockClient, clientDB, atomicTrie, root, targetHeight) | ||
| require.NoError(t, err) | ||
|
|
||
| ds := NewAtomicDynamicSyncer(inner, mockClient, clientDB, atomicTrie, root, targetHeight) | ||
|
|
||
| require.NoError(t, ds.Sync(ctx)) | ||
| require.Equal(t, targetHeight, ds.TargetHeight()) | ||
| } | ||
|
|
||
| // TestAtomicDynamicSyncer_UpdateTarget_StaleIgnored verifies that UpdateTarget | ||
| // with a height at or below the current desired height is a no-op. | ||
| func TestAtomicDynamicSyncer_UpdateTarget_StaleIgnored(t *testing.T) { | ||
| ds := types.NewDynamicSyncer("test", "test", &synctest.PivotSession{}, common.Hash{1}, 100) | ||
|
|
||
| require.NoError(t, ds.UpdateTarget(&synctest.SyncTarget{BlockRoot: common.Hash{2}, BlockHeight: 100})) | ||
| require.Equal(t, common.Hash{1}, ds.DesiredRoot()) | ||
|
|
||
| require.NoError(t, ds.UpdateTarget(&synctest.SyncTarget{BlockRoot: common.Hash{3}, BlockHeight: 50})) | ||
| require.Equal(t, common.Hash{1}, ds.DesiredRoot()) | ||
| require.Equal(t, uint64(100), ds.TargetHeight()) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting edge case. I do think this should be handled by the block syncer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The block syncer is the right layer conceptually, but there's a timing gap that makes it insufficient on its own.
The block syncer runs concurrently with the state syncer and finishes independently. Its
UpdateTargetrecords the new height, but the catch-up pass only triggers when drift exceedsblocksToFetch(256). With small advances during dynamic sync, no catch-up fires. You can envision it the following way:UpdateTargetrecords height 265blocksToFetch(256), no catch-upSync, returns nilUpdateTarget(268)AcceptSync->ResetToStateSyncedBlock(block 268)-> needs block 268 in DBAt step 9, the block syncer is long gone. We could add a targeted fetch of the latest target block before the syncer returns (between steps 3 and 4), but that only covers the target at the time the syncer finishes, not later pivots like step 6. The commit target can advance after the block syncer finishes from late block injections while the state syncer is still running.
ResetToStateSyncedBlockis the one place that has the correct block object and runs at the exact moment the block is needed. The write is two lines (WriteBlock+WriteCanonicalHash) and is idempotent if the block syncer did fetch it. I'd rather have a reliable two-line write in the right place at the right time than a more complex block syncer change that still needs a fallback for the timing gap. I hope this makes it clear. I am open to other suggestions, but at the time being I don't see a better practical solution.