feat: support wasm

[CPunisher]

Description

Related: #4550

By following rolldown/rolldown#467, I have made an ugly MVP to run rspack in browser environment.

Since I find that support for wasm is a feature that involves lots of small changes in current codebase, I'd like to list all the known steps in the following tasklist and divide the process into stages:

1. Compile rspack to wasm target, i.e. we can get .wasm files by napi build. In this stage we mostly add #[cfg(not(target_family = "wasm"))] to distinguish platforms.
2. Make rspack wasi available in node.
3. Make rspack wasi available in browser and provide an easy way for users.

Roadmap

• Compile rspack to wasm target: feat(wasm): support wasm compilation #9585
• Run rspack wasi in node
  • fix: pass regex error feat: support rspack wasi with node.js #9872
  • feat: bump napi to support multi-thread chore: bump napi 3.0.0-alpha.24 -> 3.0.0-alpha.33 #9855
  • fix: too much exports: perf(build): remove --export-dynamic flag in wasi compilation napi-rs/napi-rs#2523
  • fix: rayon thread pool stucks workers. fix(wasm): never reference handles of workers #10478
  • fix: version & platform check: fix(wasm): add wasm fallback in checkVersion #10097
  • fix: module_export in napi doesn't support wasm target. refactor: use #[napi(module_exports)] #10437
  • fix: check test cases. chore(wasm): enable wasm test ci #10757

Debugging Hint (Browser)

1. Install https://chromewebstore.google.com/detail/cc++-devtools-support-dwa/pdcpmagijalfljmkmjngeonclgbbannb
2. Set profile.release.debug = true in Cargo.toml and ensure the wasm file is smaller than 1GB, which is the limitation of the browser
3. Add path mappings by following https://users.rust-lang.org/t/getting-raw-wasm-debugging-working-nicely-in-chrome-devtools/94646#getting-debugging-working-4
4. Add another mappings by following https://issues.chromium.org/issues/401522579#comment5 due to the bugs of the extension in current version.

[netlify]

✅ Deploy Preview for rspack canceled.

Built without sensitive environment variables

Name	Link
🔨 Latest commit	019dc0d
🔍 Latest deploy log	https://app.netlify.com/sites/rspack/deploys/679798c8923ee00008517981

[codspeed-hq]

CodSpeed Performance Report

Merging #9134 will not alter performance

_{Comparing CPunisher:feat/wasm-2 (019dc0d) with main (cc5e3f6)}

🎉 Hooray! codspeed-rust just leveled up to 2.7.2!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

✅ 6 untouched benchmarks

[hardfist]

Run @rspack/core in the browser. Since rspack interface exposed by node_binding is missing lots of features, such as option nomalization, essential plugins, loader context, which make it hard to directly use, we also need to make this part avaliable. Actually I have not explored this stage in the MVP.

If Rust API is good enough, maybe we don't need to ship the js binding parts to browser, I'm not sure whether we need a light weight implementation or a full implementation, @h-a-n-a @chenjiahan what do you think?

[hardfist]

tokio: async tokio::fs => sync std::fs

seems unnecessary cause the ntaivefs is used for native OS, we can just ship a browser_fs for browser side which even don't need std::fs

[CPunisher]

tokio: async tokio::fs => sync std::fs

seems unnecessary cause the ntaivefs is used for native OS, we can just ship a browser_fs for browser side which even don't need std::fs

This is necessary because:

1. tokio fs feature is not compilable in wasm: https://docs.rs/tokio_wasi/latest/tokio/#wasm-support.
2. with the help of napi, std::fs is easily linked to an external file system provided in js side. See: https://github.com/rolldown/rolldown/pull/467/files#diff-603a6b705934598e015625888d735ef43ae35ffc4cb19aba185610fd7c9eb0dfR18

So I just call std::fs in wasm with #[cfg] in the MVP.

[hardfist]

I mean tokio::fs should only be used in NativeFileSystem

rspack/crates/rspack_fs/src/native_fs.rs

Line 19 in 5a6ed31

pub struct NativeFileSystem {

which maybe not necessary for browser, so you could choose either ways

• change tokio::fs to std::fs in NativeFileSystem when target to wasm
• or don't use NativeFileSystem and passing fs instance to compiler in browser

  rspack/crates/rspack_core/src/compiler/mod.rs

  Line 80 in 5a6ed31

  pub input_filesystem: Arc<dyn ReadableFileSystem>,

I'm not sure which way is better

[hardfist]

rspack_futures: its dependency async-scoped depends on tokio multi-thread rt.

not sure rspack_futures can be replaced with with tokio_stream and wether tokio_stream can be compatible with wasm

persistence cache: it dependes on inventory, which supports wasm in 0.3. But there are some big breaking changes. We may temporarily disable it.

I agree persistent cache maybe not needed for browser at all if it's hard to be compatible

[hardfist]

@CPunisher do you plan to finish all tasks in this PR or separated into multi PRs

[CPunisher]

rspack_futures: its dependency async-scoped depends on tokio multi-thread rt.

not sure rspack_futures can be replaced with with tokio_stream and wether tokio_stream can be compatible with wasm

I'm not sure whether rspack_futures is used for performance, since rspack_futures uses async-scoped to create a isolated tokio runtime. In the MVP I just replace all the usages with join_all. See: CPunisher@7123f19#diff-5eb759d814281903aa0355174051f134df9444a46e9a1ce332cad27eed896215R1051.

[CPunisher]

@CPunisher do you plan to finish all tasks in this PR or separated into multi PRs

I'm also considering, but I think both are ok. Because this feature consist of many small changes, it's very easy to separate it. But it seems that incomplete changes are useless. For example, incomplete stage 1 is not compilable and incomplete the whole feature is not avaliable for users. So what's your suggestion?

[hardfist]

that depends on how long would it take, I suggest when you can successfully compile rspack to wasm target(no need to actually run in browser) we can merge it first and introduce wasm target in CI so we wouldn't introduce wasm incompatible crate in the future.

[chenjiahan]

I'm not sure whether we need a light weight implementation or a full implementation

It would be great if a full implementation could be provided, which allows us to run Rsbuild on Stackblitz.

[CPunisher]

Close as the wasm target is avaliable in Rspack 1.4