Anvil: Fix everything to make it work with real assethub westend

[dimartiro]

Description

This PR enables forking from production chains like Westend Asset Hub by implementing dynamic transaction building and batch storage prefetching to address pallet index mismatches and reduce RPC latency.

Changes

Dynamic Transaction Building

When forking from a chain with a different runtime configuration, the hardcoded pallet index used by subxt_client::tx().revive().eth_transact() may not match the forked chain's pallet ordering, causing transaction failures. This PR introduces dynamic transaction building using subxt's dynamic_tx() which reads pallet indices from the runtime metadata at execution time, ensuring correct encoding regardless of the forked chain's configuration.

Batch Storage Prefetching

Each storage read from a remote chain requires an individual RPC call, causing significant latency during transaction validation. This PR implements batch prefetching using state_queryStorageAt to fetch multiple storage keys in a single RPC call. Before validating an Ethereum transaction, the sender's account info is prefetched to avoid sequential RPC round-trips.

Tests

• Updated test URL from local zombienet to real Westend Asset Hub RPC endpoint
• Added send_transaction_and_wait() and deploy_contract_and_wait() helpers that poll for transaction receipts with configurable timeouts, essential for reliable testing against remote networks with variable block times
• Removed forking-tests feature flag - forking tests now run without special configuration

All forking tests pass against real Westend Asset Hub

[alindima]

I'm still not convinced that the prefetching is very useful at this point.
We only prefetch a couple of key/value pairs for one account, but then the block mining will probably need many more keys than this.

I wouldn't do premature optimisations unless you saw a clear improvement in testing. Do you have some numbers?

[dimartiro]

I'm still not convinced that the prefetching is very useful at this point. We only prefetch a couple of key/value pairs for one account, but then the block mining will probably need many more keys than this.

I wouldn't do premature optimisations unless you saw a clear improvement in testing. Do you have some numbers?

The improvement is only a few milliseconds in our tests, so we can remove it for now and revisit potential optimizations in future PRs

[iulianbarbu]

The forking tests fail locally for me. Are they passing on your end?

failures:

---- forking::test_fork_can_deploy_contract_from_westend stdout ----

thread 'forking::test_fork_can_deploy_contract_from_westend' panicked at crates/anvil-polkadot/tests/it/forking.rs:997:10:
Contract deployment receipt not found within timeout: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- forking::test_fork_impersonate_account_from_westend stdout ----

thread 'forking::test_fork_impersonate_account_from_westend' panicked at crates/anvil-polkadot/tests/it/forking.rs:1070:10:
Impersonated transaction receipt not found within timeout: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }

---- forking::test_fork_state_snapshotting_from_westend stdout ----

thread 'forking::test_fork_state_snapshotting_from_westend' panicked at crates/anvil-polkadot/tests/it/forking.rs:809:10:
Transaction receipt not found within timeout: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }

---- forking::test_fork_eth_get_nonce_from_westend stdout ----

thread 'forking::test_fork_eth_get_nonce_from_westend' panicked at crates/anvil-polkadot/tests/it/forking.rs:740:10:
Transaction receipt not found within timeout: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }

---- forking::test_fork_can_send_tx_from_westend stdout ----

thread 'forking::test_fork_can_send_tx_from_westend' panicked at crates/anvil-polkadot/tests/it/forking.rs:902:10:
Transaction receipt not found within timeout: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }

---- forking::test_fork_eth_get_code_from_westend stdout ----

thread 'forking::test_fork_eth_get_code_from_westend' panicked at crates/anvil-polkadot/tests/it/utils.rs:390:57:
called `Result::unwrap()` on an `Err` value: RpcError { code: InternalError, message: "Internal error: Failed to submit transaction: RPC error: client error: ErrorObject { code: ServerError(1010), message: \"Invalid Transaction\", data: Some(RawValue(\"Inability to pay some fees (e.g. account balance too low)\")) }", data: None }


failures:
    forking::test_fork_can_deploy_contract_from_westend
    forking::test_fork_can_send_tx_from_westend
    forking::test_fork_eth_get_code_from_westend
    forking::test_fork_eth_get_nonce_from_westend
    forking::test_fork_impersonate_account_from_westend
    forking::test_fork_state_snapshotting_from_westend

test result: FAILED. 114 passed; 6 failed; 0 ignored; 0 measured; 0 filtered out; finished in 89.72s

[dimartiro]

The forking tests fail locally for me. Are they passing on your end?

Yeah I noticed the same yesterday, the used to pass but I'm getting some 404 now, I'm working on debugging this, will let u know

[dimartiro]

The forking tests fail locally for me. Are they passing on your end?

I found the issue — it’s related to the latest runtime update on Westend, so I need to update polkadot-sdk to fix it.
To do that, I opened this PR to sync the feature branch with master.
Could you guys please merge it so I can unblock this one?

Thanks!

[dimartiro]

The forking tests fail locally for me. Are they passing on your end?

Fixed

[re-gius]

We are getting close to the finish line - just a reminder to improve on the 120 seconds timeouts and try to use a much lower timeout. Otherwise, in case CI fails or something is broken, forking tests may take too long

[re-gius]

LGTM

[re-gius]

I will merge it by the end of today - in case anyone else has something to point out

[dimartiro]

I will merge it by the end of today - in case anyone else has something to point out

Great! thks!