perf(tree): worker pooling for account proofs

[yongkangc]

This PR builds on top of #18887 by adding worker pooling to account and blinded account proofs, which allows us to reduce overhead from spawning the worker.

Changes:

• Move Account to pooling logic
• Move Blinded Account Nodes to pooling logic. Removed underlying structure and functions (pending tasks)

Core logic:

1. Determine workers based on parallelism
2. Spawn x workers each with its own tx
3. Queue storage proofs via crossbeam channels to the workers
4. Await Storage proofs to build account proofs

Caller
    ↓ (requests account multiproof)
ProofTaskManager
    ↓ (routes to account worker pool)
Account Worker
    ↓ (calls collect_storage_proofs)
    ↓ (queues all storage proofs to storage worker pool)
Storage Workers (parallel computation)
    ↓ (return storage proofs)
Account Worker
    ↓ (calls build_account_multiproof_with_storage_roots)
    ↓ (walks account trie, looks up pre-computed storage roots)
Return to Caller (final account multiproof)

Issues closed:

• Use ProofTaskManager for account proofs from MultiProofTask #18734
• Use ProofTaskManager for Blinded Node Migration #18833
• Parent: Tracking: Multiproof Thread Pool Optimization #18800

References:

• Overall POC: perf(trie): proofmanager optimisation WIP #18829

Next Steps:

• Removing ProofTaskManager + other cleanups

[Copilot]

Pull Request Overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Pull Request Overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[yongkangc]

@Rjected @shekhirin @mediocregopher ready for a final review 🙏🏻

[yongkangc]

this isaddressed later on in the pr for cleanup where we use proof_task_handle instead

[mattsse]

haven't looked too close into the actual proof logic

other than that lgtm

pending @Rjected

[shekhirin]

as a small optimization for future PRs, this calculation can be skipped if trace logs below won't be emitted

[yongkangc]

Thanks for the note

[shekhirin]

LGTM, my only suggestion is the same as Dan's — about free functions instead of structs with methods that keep the state.

[Rjected]

lgtm with the one suggestions about putting the worker state into structs

[Rjected]

do you mean that we should keep the ProofTaskManager?

no, I just mean that a struct, and a method on the struct called run or something similar, makes more sense to me than having _loop functions, since they are tasks with state

ie something like

let worker = AccountProofWorker::new(...);
executor.spawn_blocking(move || {
    worker.run();
});

makes more sense to me than how we currently spawn

[yongkangc]

@shekhirin @Rjected will address Ur comments about structs in another pr