Upgrade wasmtime #8714

bkchr · 2025-05-31T21:45:53Z

This upgrades wasmtime to the latest version and also fixes backtraces for debug builds.

bkchr · 2025-05-31T21:52:58Z

/cmd fmt

koute · 2025-06-02T06:58:23Z

substrate/client/executor/wasmtime/Cargo.toml

+	"gc",
+	"gc-null",
 	"parallel-compilation",
 	"pooling-allocator",
+	"threads",


The garbage collection and threads features should not be necessary.

Yeah I know, but I activated them to deactivate them in the config. Otherwise someone downstream, who maybe imports wasmtime with the default feature activated (which activates the threads and gc features) will silently activate the stuff, because they are true by default.

koute · 2025-06-02T07:01:29Z

substrate/client/executor/wasmtime/src/runtime.rs

 	// they should be introduced here as well.
 	config.wasm_reference_types(semantics.wasm_reference_types);
 	config.wasm_simd(semantics.wasm_simd);
+	config.wasm_relaxed_simd(semantics.wasm_simd);


Looks like the newest wasmtime has some extra new knobs besides this that are now true by default: wasm_extended_const, wasm_tail_call

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

bkchr · 2025-06-02T20:30:31Z

/cmd prdoc --bump major --audience node_dev

…-audience node_dev'

paritytech-workflow-stopper · 2025-06-02T21:02:44Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/15402198357
Failed job name: test-linux-stable

jasl · 2025-07-14T20:58:39Z

Now the latest version is 34.0.1, and the 33.0.0 seems yanked (replaced by 33.0.1) because of panic with host-defined tables/globals and concrete reference

bkchr · 2025-07-30T12:41:23Z

@s0me0ne-unkn0wn @koute do you have an idea how much we can trust the test-syscalls test?

koute · 2025-07-30T12:53:50Z

@s0me0ne-unkn0wn @koute do you have an idea how much we can trust the test-syscalls test?

Should be reliable, as long as there are no warnings generated by the script.

(Of course there's always a chance that there could be a weird bug/corner case, but if it worked so far then it probably still works? :P)

bkchr · 2025-07-30T15:34:13Z

Should be reliable, as long as there are no warnings generated by the script.

I'm just asking because I needed to modify the expected calls :D

s0me0ne-unkn0wn

Should be fine (but if everything explodes, don't quote me). I'd be much more concerned about possible regressions, such as compilation time, execution time, stack usage, etc, given that we're switching versions in a non-deterministic way. I think we had plans to have tooling to at least check all the existing on-chain code for such regressions?

bkchr · 2025-07-30T16:07:58Z

I think we had plans to have tooling to at least check all the existing on-chain code for such regressions?

Yes, but not sure this manifested into anything concrete? 🙈

iulianbarbu · 2025-08-13T12:48:12Z

prdoc/pr_8714.prdoc

+- audience: Node Dev
+  description: This upgrades wasmtime to the latest version and also fixes backtraces
+    for `debug` builds.
+crates:


Should we have majorly bumped all crates depending on wasmtime too here? I'll give you an example. sp-wasm-interface depends on wasmtime. If no bump is introduced here, no publishing will happen for it in next release and it will still depend on older wasmtime.

However, sc-executor-wasmtime has a major bump, meaning it will be released in a new version depending on wasmtime 35.0.0.

But, sc-executor-wasmtime has a dependency on sp-wasmtime-interface, and more specifically, relies on this trait (https://github.com/paritytech/polkadot-sdk/blob/master/substrate/primitives/wasm-interface/src/lib.rs#L344) here (https://github.com/paritytech/polkadot-sdk/blob/master/substrate/client/executor/wasmtime/src/imports.rs#L95). Our parity-publish check fails since some time ago (before this PR), but I think if successful and made mandatory it would've catched this. Later merged PRs show such error in parity-publish failing worklfow.

error[E0053]: method `with_function_context` has an incompatible type for trait --> substrate/client/executor/wasmtime/src/imports.rs:101:11 | 101 | caller: wasmtime::Caller<Self::State>, | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `Caller<'_, StoreData>`, found `wasmtime::Caller<'_, StoreData>` | = note: expected signature `fn(sp_wasm_interface::wasmtime::Caller<'_, StoreData>, _) -> _` found signature `fn(wasmtime::Caller<'_, StoreData>, _) -> _` help: change the parameter type to match the trait | 101 - caller: wasmtime::Caller<Self::State>, 101 + caller: sp_wasm_interface::wasmtime::Caller<'_, StoreData>, |

I think we should consider making parity-publish check mandatory on master, to avoid issues with missing bumps when releasing.

It is more of a problem of parity-publish actually. What I observed is that whenever we bump a crate, all the crates depending on the bumped crates might get a bump too based on how they depend on the initially bumped crates (eg. usually we depend on explicit versions - by virtue of having all crates in a monorepo workspace - so any bump attracts bumps of dependents I think).

But if we bump a crate not part of the monorepo/workspace, I think parity-publish can't tell all dependents (and their dependents and so on) that need to be bumped as a result.

LE: major bumps for crates attract bumps for dependents.

# Description Adds a few crate bumps associated to PRs which missed to bump them, and updates parity-publish version across the board to 0.10.6 (to support rustc 1.88). Additionally, makes it so that parity-publish-check-compile runs first on all unreleased prdocs to bump associated crates, and only after moving those to an `unreleased` directory, runs on the current PR's prdoc. This is so that we first create a "local release" based on the unreleased prdocs, and then we follow with a "patch" release based on the previous local release, considering only the prdoc pushed with the current PR. If the workflow fails at the end it means current PR missed certain bumps. If we don't do the plan/apply twice we risk to miss bumps due to all prdocs being considered (current PR's prdoc + unreleased ones) when running parity-publish plan/apply, which might result in a set of crate bumps which are sufficient, but once some unreleased prdocs will be moved to a stable prodoc directory, because they will be part of a stable release, then the ones left will not be enough from a bump perspective (e.g. like it happened in #9320). That's why it is important to check every PR that adds a prdoc whether it is self-sufficient from a crates bumping perspective. If no prdoc is provided, the parity-publish does not need to be taken into consideration, but it should also pass nonetheless. ## Integration N/A ## Review Notes There seems to a be a corner case parity-publish can not easily catch. All bumps below are a manifestation of it. More details below: * #8714 - a major bump is necessary for `sp-wasm-interface` - context here: #8714 (comment) * `sp-keystore` was bumped during 2506 in #6010 , and the relevant prdoc got moved to stable2506 dir in #9320. This moved prdoc coexisted alongside other unreleased prdocs, and covered a needed patch bump for `sp-keystore`, that is not easily visible, and also required for crates publishing IIUC: 1. `sp-io` is major bumped because its direct dependency, `sp-state-machine`, was major bumped. 2. `sp-io` has a direct dependency on `sp-core` (minor bumped), and `sp-keystore` (not touched, not bumped by now) 3. `sp-io` fails to compile because it pulls same types from different `sp-core` versions (it implements `Keystore` trait from `sp-keystore` with methods signatures referencing types from `sp-core 38.0.0` by using the `sp-core 0.38.1` - unreleased yet - types, which confuses rustc). * `sp-rpc` needs a bump too due to pulling `sp-core 38.0.0`, like `sp-keystore`, and it is an indirect dependency of `polkadot-cli`, which has also a direct dependency on unreleased `sp-core 38.1.0`, so again, if we don't bump `sp-rpc` (historically it has been bumped only with major, but I think we can go with patch on this one), `polkadot-cli` can't compile. * `sc-storage-monitor` is in a similar situation as `sp-rpc`/`sp-keystore` - `polkadot-cli` depends on `sc-storage-monitor` (which is not bumped, and has a dependency on `sp-core 38.0.0`), but it also depends on `sp-core 38.1.0`. And yet again, something is used in `polkadot-cli` from the two different `sp-core` versions, which confuses rustc. --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Alexander Samusev <[email protected]>

@saraswatpuneet

# Goal The goal of this PR is to fix failures in `make lint-audit`. # Discussion - `slab` was yanked, so it was updated to version v0.4.11 - `tracing-subscriber` reported a RUSTSEC vulnerability warning and was updated to v0.3.20 - Additional crates were updated due to dependencies on these crates. - A new `Cargo.lock` file was generated. - Updating `tracing-subscriber` required Rust `edition2024` and therefore the rust toolchain was updated to `1.88` - The github actions container was updated to use `1.88` and that version was updated: `1.5.8` - Rust toolchain `1.88` required code edits to eliminate several warnings that have been mainlined. These are mostly around strings and some clippy warnings. - Most of these changes were grabbed from a recent polkadot SDK update, courtesy of @saraswatpuneet - `fxhash` was flagged as `unmaintained`. It is a `polkadot-sdk` dependency from `wasmtime`. This [Parity PR](paritytech/polkadot-sdk#8714) has updated `wasmtime` which removes the dependency on `fxhash`. - The RUSTSEC unmaintained warning was added to the `deny.toml` ignore list until the above PR is merged into a stable release. - `yamux` updated to 0.13.6, possible solution to some network instability Co-authored-by: Puneet Saraswat <[email protected]>

This upgrades wasmtime to the latest version and also fixes backtraces for `debug` builds. --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <[email protected]>

# Description Adds a few crate bumps associated to PRs which missed to bump them, and updates parity-publish version across the board to 0.10.6 (to support rustc 1.88). Additionally, makes it so that parity-publish-check-compile runs first on all unreleased prdocs to bump associated crates, and only after moving those to an `unreleased` directory, runs on the current PR's prdoc. This is so that we first create a "local release" based on the unreleased prdocs, and then we follow with a "patch" release based on the previous local release, considering only the prdoc pushed with the current PR. If the workflow fails at the end it means current PR missed certain bumps. If we don't do the plan/apply twice we risk to miss bumps due to all prdocs being considered (current PR's prdoc + unreleased ones) when running parity-publish plan/apply, which might result in a set of crate bumps which are sufficient, but once some unreleased prdocs will be moved to a stable prodoc directory, because they will be part of a stable release, then the ones left will not be enough from a bump perspective (e.g. like it happened in #9320). That's why it is important to check every PR that adds a prdoc whether it is self-sufficient from a crates bumping perspective. If no prdoc is provided, the parity-publish does not need to be taken into consideration, but it should also pass nonetheless. ## Integration N/A ## Review Notes There seems to a be a corner case parity-publish can not easily catch. All bumps below are a manifestation of it. More details below: * #8714 - a major bump is necessary for `sp-wasm-interface` - context here: #8714 (comment) * `sp-keystore` was bumped during 2506 in #6010 , and the relevant prdoc got moved to stable2506 dir in #9320. This moved prdoc coexisted alongside other unreleased prdocs, and covered a needed patch bump for `sp-keystore`, that is not easily visible, and also required for crates publishing IIUC: 1. `sp-io` is major bumped because its direct dependency, `sp-state-machine`, was major bumped. 2. `sp-io` has a direct dependency on `sp-core` (minor bumped), and `sp-keystore` (not touched, not bumped by now) 3. `sp-io` fails to compile because it pulls same types from different `sp-core` versions (it implements `Keystore` trait from `sp-keystore` with methods signatures referencing types from `sp-core 38.0.0` by using the `sp-core 0.38.1` - unreleased yet - types, which confuses rustc). * `sp-rpc` needs a bump too due to pulling `sp-core 38.0.0`, like `sp-keystore`, and it is an indirect dependency of `polkadot-cli`, which has also a direct dependency on unreleased `sp-core 38.1.0`, so again, if we don't bump `sp-rpc` (historically it has been bumped only with major, but I think we can go with patch on this one), `polkadot-cli` can't compile. * `sc-storage-monitor` is in a similar situation as `sp-rpc`/`sp-keystore` - `polkadot-cli` depends on `sc-storage-monitor` (which is not bumped, and has a dependency on `sp-core 38.0.0`), but it also depends on `sp-core 38.1.0`. And yet again, something is used in `polkadot-cli` from the two different `sp-core` versions, which confuses rustc. --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Alexander Samusev <[email protected]>

This upgrades wasmtime to the latest version and also fixes backtraces for `debug` builds. --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <[email protected]>

bkchr added 2 commits May 30, 2025 18:26

Upgrade wasmtime

145e616

Some more fixes

4427f65

bkchr requested a review from koute as a code owner May 31, 2025 21:45

bkchr added the R0-no-crate-publish-required The change does not require any crates to be re-published. label May 31, 2025

github-actions bot and others added 3 commits May 31, 2025 21:55

Update from github-actions[bot] running command 'fmt'

a8e2302

Fix mixnet

6e151ee

Hopefully fix the erros

81ef25a

koute reviewed Jun 2, 2025

View reviewed changes

bkchr and others added 3 commits June 2, 2025 13:33

Mighty compiler god, are you happy with this?

71bfd73

Increase instance size

7783a7c

Exclude zepter for wasmtime

16201ef

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

bkchr added T0-node This PR/Issue is related to the topic “node”. T18-zombienet_tests Trigger zombienet CI tests. and removed R0-no-crate-publish-required The change does not require any crates to be re-published. labels Jun 2, 2025

update Zepter in CI

c8f2ba2

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez requested review from a team as code owners June 2, 2025 15:46

alvicsam approved these changes Jun 2, 2025

View reviewed changes

paritytech-review-bot bot requested a review from a team June 2, 2025 15:54

AndWeHaveAPlan approved these changes Jun 2, 2025

View reviewed changes

EgorPopelyaev approved these changes Jun 2, 2025

View reviewed changes

ggwpez and others added 3 commits June 2, 2025 19:01

Also update Zepter in other CI check...

afed46e

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Deduplicate features

f6372cc

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Fix test

d765ab7

Update from github-actions[bot] running command 'prdoc --bump major -…

3f0ccfa

…-audience node_dev'

Fix issues

c8a089f

bkontur mentioned this pull request Jun 9, 2025

Bump subxt from 0.40.0 to 0.40.1 paritytech/parity-bridges-common#3164

Merged

bkchr mentioned this pull request Jun 11, 2025

Benchmark panics in slice::from_raw_parts #8818

Closed

bkchr added 2 commits July 29, 2025 21:52

Merge remote-tracking branch 'origin/master' into bkchr-upgrade-wasmtime

67bf701

Upgrade to latest wasmtime

bada44a

bkchr requested a review from koute July 29, 2025 20:16

bkchr and others added 2 commits July 29, 2025 22:17

Merge branch 'master' into bkchr-upgrade-wasmtime

a6b17c7

Update syscalls

945d0d2

koute approved these changes Jul 30, 2025

View reviewed changes

s0me0ne-unkn0wn approved these changes Jul 30, 2025

View reviewed changes

Also update the prepare-worker

0a0901c

bkchr added this pull request to the merge queue Jul 31, 2025

Merged via the queue into master with commit 13bc266 Jul 31, 2025
247 of 252 checks passed

bkchr deleted the bkchr-upgrade-wasmtime branch July 31, 2025 22:45

iulianbarbu reviewed Aug 13, 2025

View reviewed changes

iulianbarbu mentioned this pull request Aug 14, 2025

fix: add missing crates bumps and upgrade parity-publish #9488

Merged

mattheworris mentioned this pull request Sep 5, 2025

Update crates and toolchain frequency-chain/frequency#2589

Merged

7 tasks

mattheworris mentioned this pull request Sep 8, 2025

[Chore]: Watch for fxhash crate, a wasmtime dependency, to be updated in polkadot-sdk frequency-chain/frequency#2590

Open

3 tasks

Upgrade wasmtime #8714

Upgrade wasmtime #8714

Uh oh!

Conversation

bkchr commented May 31, 2025

Uh oh!

bkchr commented May 31, 2025

Uh oh!

koute Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

bkchr Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

koute Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

bkchr commented Jun 2, 2025

Uh oh!

paritytech-workflow-stopper bot commented Jun 2, 2025

Uh oh!

jasl commented Jul 14, 2025

Uh oh!

bkchr commented Jul 30, 2025

Uh oh!

koute commented Jul 30, 2025

Uh oh!

bkchr commented Jul 30, 2025

Uh oh!

s0me0ne-unkn0wn left a comment

Choose a reason for hiding this comment

Uh oh!

bkchr commented Jul 30, 2025

Uh oh!

Uh oh!

iulianbarbu Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iulianbarbu Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

iulianbarbu Aug 13, 2025 •

edited

Loading

iulianbarbu Aug 15, 2025 •

edited

Loading