perf(l1): run "engine_newPayload" block execution in a worker thread #5051
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extract the block execution for payloads to an OS thread, make all its callees regular blocking functions, and removes the reliance on Tokio. Caveats: 1. The worker is in the RPC, mostly because the Blockchain is not Clone; 2. The L2 is broken from the removal of async, fee config mostly becomes a mess; 3. The channel is provided by Tokio because the RPC is still async and shouldn't block waiting.
Lines of code reportTotal lines added: Detailed view |
MegaRedHand
changed the title
MegaRedHand
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR refactors block execution in Layer 1 to run synchronously in a dedicated worker thread instead of using async operations. The main change moves engine_newPayload block execution off Tokio's runtime by converting async methods to blocking functions and introducing a thread-based worker with channel communication.
Key changes:
- Converted
add_block,store_block,execute_block, and related storage methods from async to synchronous - Introduced
start_block_executorfunction that spawns an OS thread with a channel-based communication pattern - Removed
.awaitcalls throughout the codebase where async block operations were invoked
Reviewed Changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/blockchain/blockchain.rs | Converted core blockchain methods (add_block, store_block, execute_block) from async to sync |
| crates/storage/store.rs | Made storage methods synchronous (store_block_updates, add_pending_block, apply_account_updates_batch) |
| crates/storage/store_db/rocksdb.rs | Removed spawn_blocking wrapper and made apply_updates synchronous |
| crates/storage/store_db/in_memory.rs | Updated in-memory store implementation to match synchronous trait |
| crates/storage/api.rs | Updated StoreEngine trait to define synchronous method signatures |
| crates/networking/rpc/rpc.rs | Added start_block_executor worker thread and channel infrastructure |
| crates/networking/rpc/engine/payload.rs | Introduced add_block helper to communicate with worker thread via channel |
| crates/networking/rpc/utils.rs | Updated test utilities to initialize block worker channel |
| crates/networking/rpc/lib.rs | Exported start_block_executor function |
| crates/networking/p2p/sync.rs | Removed .await from block sync operations |
| crates/l2/sequencer/block_producer.rs | Updated L2 block production to use synchronous storage methods |
| crates/l2/sequencer/l1_committer.rs | Updated L1 committer to use synchronous blockchain operations |
| crates/l2/networking/rpc/rpc.rs | Initialized block worker channel for L2 RPC |
| crates/l2/based/block_fetcher.rs | Updated block fetcher to use synchronous add_block |
| tooling/migrations/src/cli.rs | Removed .await from migration block operations |
| tooling/ef_tests/state_v2/src/modules/result_check.rs | Updated test to use synchronous account updates |
| tooling/ef_tests/state_v2/src/modules/block_runner.rs | Updated test runner to use synchronous block operations |
| tooling/ef_tests/blockchain/test_runner.rs | Updated blockchain tests to use synchronous methods |
| cmd/ethrex/cli.rs | Updated block import to use synchronous operations |
| cmd/ethrex/initializers.rs | Updated head state regeneration to use synchronous methods |
| cmd/ethrex/l2/command.rs | Removed .await from account updates batch application |
| crates/blockchain/smoke_test.rs | Updated smoke tests to use synchronous block operations |
| crates/blockchain/payload.rs | Updated payload processing to use synchronous account updates |
| crates/networking/p2p/rlpx/l2/l2_connection.rs | Updated L2 connection to use synchronous block operations |
| benches/benches/build_block_benchmark.rs | Updated benchmark to use synchronous block execution |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Benchmark Block Execution Results Comparison Against Main
|
MegaRedHand
reviewed
Oct 27, 2025
Comment on lines
+684
to
+693
| ctx.block_worker_channel | ||
| .send((notify_send, block)) | ||
| .map_err(|e| { | ||
| ChainError::Custom(format!( | ||
| "failed to send block execution request to worker: {e}" | ||
| )) | ||
| })?; | ||
| notify_recv | ||
| .await | ||
| .map_err(|e| ChainError::Custom(format!("failed to receive block execution result: {e}")))? |
There was a problem hiding this comment.
We'll need to check for edge cases like two competing blocks being received at the same time, in the future. That could lead to increased latency.
jrchatruc
approved these changes
Oct 27, 2025
MegaRedHand
approved these changes
Oct 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Extract the block execution for payloads to an OS thread, make all its
callees regular blocking functions, and removes the reliance on Tokio.
Caveats:
a mess;
shouldn't block waiting.
Motivation
Description
Closes #issue_number