feat!: Switch to using jsonrpsee for foreign calls; refactor run_test; foreign call layering

[aakoshh]

Description

Problem*

Resolves #6742

Summary*

Instead of jsonrpc we use jsonrpsee to make calls to the Oracle function resolver.

Also includes #6857 and #6858 which effectively make nargo::ops::test receive a generic ForeignCallExecutor (a breaking change in the API) so that we can use it with Wasm, which otherwise could not compile the RpcForeignCallExecutor.

Additional Context

Our codebase is mostly synchronous, but jsonrpsee is async. I created a tokio::runtime::Runtime in the executor where I can block on the calls, however this doesn't work well with the #[tokio::test], runtimes don't nest. Because of this the tests require a bit more supporting machinery than before.

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[github-actions]

Peak Memory Sample

Program	Peak Memory
keccak256	78.48M
workspace	123.28M
regression_4709	422.91M
ram_blowup_regression	1.58G
private-kernel-tail	201.81M
private-kernel-reset	717.07M
private-kernel-inner	291.88M
parity-root	172.14M

[github-actions]

Compilation Sample

Program	Compilation Time	%
sha256_regression	1.381s	-4%
regression_4709	0.803s	1%
ram_blowup_regression	15.375s	-1%
rollup-root	3.744s	0%
rollup-block-merge	3.685s	-12%
rollup-base-public	115.384s	1%
rollup-base-private	96.769s	3%
private-kernel-tail	1.053s	-7%
private-kernel-reset	7.406s	-3%
private-kernel-inner	2.099s	1%
parity-root	0.776s	4%
noir-contracts	88.147s	4%

[aakoshh]

@TomAFrench it looks like it doesn't compile to Wasm because the http-client feature brings in tokio:net which brings mio as a dependency. It doesn't look like noir_wasm uses the foreign_calls module, so I guess I can try putting that behind a feature flag, so we don't have these network related dependencies when we're compiling to Wasm.

[TomAFrench]

Yeah we don't need an RPC server in wasm as we expect that to use callbacks directly. We can put it behind a feature flag then.

[aakoshh]

@TomAFrench the problem is not with the RPC server, which was only used as a dev-dependency, it's the RPC client itself. I wanted rpc to be behind a feature flag, but then realised that ops::test directly depends on being able to make HTTP calls: TestForeignCallExecutor uses RPCForeignCallExecutor. By contrast, ProgramExecutor in ops::execute only uses the ForeignCallExecutor trait, which would be easier to isolate from the network related stuff in foreign_calls module.

I created a test and execute feature flag, so that noir_wasm can depend only on compilation machinery, which is the only one it has #[wasm_bindgen] for right now, but by disabling test in Wasm, #6835 is moot at the moment.

We can refactor the whole module so that a ForeignCallExecutor is passed to the ops::test::run_test as a dependency, similar to how it's done in ProgramExecutor::new, rather than as a URL, which I think is useless within Wasm; at least I don't see any injected methods that would handle network calls. It would be a breaking change in the test API, though. I opened a PR for this: #6858

in wasm as we expect that to use callbacks directly

Perhaps this is related, can you elaborate on what you wanted to do?

[github-actions]

Execution Sample

Program	Execution Time	%
sha256_regression	0.619s	-2%
regression_4709	0.386s	-5%
ram_blowup_regression	4.440s	2%
rollup-root	1.913s	0%
rollup-block-merge	1.913s	1%
rollup-base-public	21.265s	1%
rollup-base-private	19.601s	3%
private-kernel-tail	0.720s	-1%
private-kernel-reset	1.517s	1%
private-kernel-inner	0.967s	2%
parity-root	0.529s	-2%

[TomAFrench]

Looks good, some nits on how we construct the TestForeignCallExecutor.

[TomAFrench]

I'm going to fix the conflicts in this PR caused by #6865

[TomAFrench]

As you're out today I'll just make the above changes so we can get master green again.

[TomAFrench]

LGTM