perf(tree): worker pooling for storage in multiproof generation

[yongkangc]

This PR add worker pooling to our existing proof generation. This PR starts by adding worker pooling to storage proofs, which allows us to reduce overhead from spawning the worker.

Core logic:

1. Determine workers based on parallelism
2. Spawn x workers each with its own tx
3. Queue storage proofs via crossbeam channels to the workers

Issues closed:

• Maintain a thread pool in ProofTaskManager #18735
• Parent: Tracking: Multiproof Thread Pool Optimization #18800

References:

• Overall POC: perf(trie): proofmanager optimisation WIP #18829
• Previously we had a PR up by Rhovian. However the issue was that this is a larger refactors with significant back and forth, and therefore we had to bring this refactor in internally.

Next Steps:

• Moving Account and Blinded nodes to adopt the same infrastructure

Benches:

Reth4 Mainnet

[Copilot]

Pull Request Overview

This PR introduces worker pooling for storage proof generation to improve multiproof performance through parallel processing. The implementation adds dedicated storage workers with long-lived database transactions while maintaining backward compatibility.

• Introduces worker pool architecture with pre-spawned workers for storage proof computation
• Replaces ProofTaskManager internal task handling with mixed worker pool and on-demand execution
• Adds crossbeam channels for efficient worker communication and transaction budget allocation

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
crates/trie/parallel/src/proof_task_metrics.rs	Adds metrics tracking for storage proof workers and fallback scenarios
crates/trie/parallel/src/proof_task.rs	Major refactor introducing worker pool architecture and new task routing logic
crates/trie/parallel/src/proof.rs	Updates test to use new constructor signature with storage worker count
crates/trie/parallel/Cargo.toml	Adds crossbeam-channel dependency for worker communication
crates/engine/tree/src/tree/payload_processor/multiproof.rs	Updates test configuration for new worker pool parameters
crates/engine/tree/src/tree/payload_processor/mod.rs	Integrates storage worker configuration into payload processor
crates/engine/primitives/src/config.rs	Adds configuration options for storage proof worker pool sizing

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Pull Request Overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[yongkangc]

Benching at 3fb97c6 on reth4 mainnet

[yongkangc]

Bench on Base cc @mattsse