Add cli option --wasmtime-precompiled and subcommand precompile-wasm

[librelois]

Description

What does this PR do?

This PR adds the possibility of starting the node with a precompiled WASM runtime.
This eliminates the need to compile the runtime at startup, making it possible to launch the node much more quickly.

Why are these changes needed?

A practical example of where this feature is useful is when you need to run a large number of end2end tests quickly: you precompile the runtime once in the same environment, then use the precompiled artifact every time you need to spawn the node (potentially hundreds of times).

How were these changes implemented and what do they affect?

As a security measure, a hash of the wasmtime configuration is inserted into the precompiled artifact and checked by wasmtime at startup, to ensure that, to ensure that the precompiled artifact has been generated under the same conditions.

This check is delegated to wasmtime directly thanks to the module_version function.

How to use

1. Generate the precompiled artifact with the subcommand precompile-wasm:

precompile-wasm --chain <your-chain-spec> path/to/artifacts/folder

This will precompile the on-chain wasm at the last finalized block and save the precompiled artifact in path/to/artifacts/folder.

2. Use the precompiled artifact with the cli option --wasmtime-precompiled path/to/artifacts/folder

[koute]

What's your motivation to add this? Is this only supposed to be used for tests?

I'd be fine to have this to speed up tests, but I'm not convinced we should support this for running normal nodes, as it would add extra complexity and be yet another place where something can go wrong for not much benefit.

There's also the issue that for PVFs in Polkadot we already have caching implemented, so this would essentially duplicate that functionality in an incompatible way. (Although the requirements are different, as for PVFs we need sandboxing, so it's not really obvious how you would unify those.)

How about we fundamentally simplify this and do something like this instead?

• Remove the extra CLI arguments/subcommands you've added.
• Remove all other changes outside of the sc-executor-wasmtime.
• In sc-executor-wasmtime check if a special environment variable is set, let's say, WASMTIME_PRECOMPILED_CACHE_DIR or something like that.
• If it is set then the executor would automatically and transparently use that directory as a cache.
• As the key for the cache use BLAKE3 hash of the semantics struct, the raw .wasm blob and of the current executable (std::fs::read the std::env::current_exe); unlike CARGO_PKG_VERSION this would guarantee that the executable has to be bit-exact for cache to be used.
• If there are errors anywhere in this process they should be warn! printed and ignored (that is, the node should compile the module normally)

This would be, I think, simpler, need less code, be more maintainable, be more foolproof, and be easier to use.

[bkchr]

Yeah I was proposing to go this route. They are running a lot of tests in their CI and that got really slow with the removal of wasmi as they will now need to recompile it all the time.

I'm not sure that a env variable is the way to go here either. No one will find out about this env variable besides the three of us :P I would propose that you extend WasmExecutorBuilder to add a precompiled_path or whatever. The CLI stuff is really not required here in our repo. You can just configure the executor as you like in moonbeam and use a CLI argument or a env variable or whatever. I didn't check your pr in detail, but if you need functionality exposed to precompile a binary, it is also fine to expose this function to be used in moonbeam.

[koute]

I'm not sure that a env variable is the way to go here either. No one will find out about this env variable besides the three of us :P I would propose that you extend WasmExecutorBuilder to add a precompiled_path or whatever.

Yeah, that also sounds good to me.

[crystalin]

The command should output the name of the file also IMO

[crystalin]

The expected wasm and the generated one don't have the same hash:

2023-09-28 09:38:09 artifact_version: 0x3bb7567dc2e545fc246e248f2d0d3e3da0ba14c65bb01948001586448da83e56
2023-09-28 09:38:09 wasm file_name: precompiled_wasm_0x0x91b7bcbc9e993dded10ed74930ca3f2805a470980b47889f4d5cff4aab63e952

[crystalin]

The wasm is actually loaded at multiple places with different hashes:
In resolve_state_version_from_wasm where it takes the :code value but uses a special hash (Which I think is due to not having storage yet as we are building genesis)

The other one is initiated by grandpa where it retrieve the runtime and hash in the call_executor, computed in the state-machine using the storage_hash

This mean that for the same code, we have 2 different hashes and so we won't load the precompiled wasm

[paritytech-cicd-pr]

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3840831

[crystalin]

Something to verify. I replaced the hasher of the genesis code to use the Blake2Hasher to be compatible with the precompiled and on-chain wasm calculated hash:
https://github.com/paritytech/polkadot-sdk/pull/1641/files#diff-680ae468913e48e81b977d13c888b391a21b3254e62ed1cc40bfb0e7310b7389R51-R52

After that change and marking allow_missing_func_imports to false, I was able on Moonbeam to boot the node without any compilation.

Something to note also, by default the "precompile-wasm" command will use the default base-path if none is provided, making this command:
../target/release/moonbeam precompile-wasm -lwasmtime-runtime --chain moonbase-dev wasm not use the moonbase-dev chain spec code but the one from your $HOME/.local/share/moonbeam/chains/moonbase_dev which might not be the same

[crystalin]

The precompile-wasm actually store the chain data in the base-path, like when launching a network. I don't think we should include that

[librelois]

The CLI stuff is really not required here in our repo. You can just configure the executor as you like in moonbeam and use a CLI argument or a env variable

This isn't a problem specific to moonbeam, it concerns all substrate chains that need to spawn a lot of nodes for their end2end tests. For example, Tanssi also has many end2end typescrit tests.

The precompile-wasm actually store the chain data in the base-path, like when launching a network. I don't think we should include that

This is linked to the fact that the command doesn't know if you want to retrieve the runtime code from the DB or not, so it has to be initialized. Adding an explicit flag to retrieve from chain spec should solve the problem.

[girazoki]

Any update on this? as @librelois this would be heplful for other ecosystem projects as well (like Tanssi in this case)