asm.js -> wasm migration path

[buu700]

Is it planned for the migration path for existing asm.js projects to wasm to be as simple as adding -s WASM=1 to an existing Makefile?

That doesn't seem to be the case right now, based on a quick test with one of the asm.js libraries that I maintain, but I understand that wasm support is still early right now. It would be very convenient for everything to eventually work exactly as it does now for asm.js, without any extra required tooling or rewriting of existing code.

Specific concerns that come to mind:

• --pre-js/--post-js support — my implementations use pre.js to start a closure and post.js to call into Module; does (or will) this already work as-is for wasm?

• Module API — does this change at all when using wasm?

• wasm bundling, as in the webpack wasm-loader which simply converts wasm ES6 imports into Uint8Arrays (we have a constraint that requires our asm libraries to each compile to a single JavaScript file)

• asm.js bundling — I'd be fine waiting a year or two for non-wasm-supporting browser versions to lose market share before flipping the wasm switch, but for my purposes it would really be ideal if I could just start using it today in newer browsers for the performance benefit, even at the cost of inflating the bundle size

I see in #5104 that the last two aren't currently implemented, but as a whole this seems like a separate issue since that ticket looks to be more focused on the XHR-based implementation (which explicitly won't apply to my use case).

[kripken]

In general the plan is for migration to be just -s WASM=1, yes. But wasm has some current limitations that cause issues. The main one is that chrome does not support sync compilation yet, so we must do async compilation, and that means that our startup is async, which requires refactoring in some projects. Another is that wasm is a binary file on the side, so there is another file to deal with in distributing the code.

For your points:

• pre/post-js should be the same. However, async startup may alter when the post-js code is called. (Code that waits for main() or the ready callbacks is ok in both asm.js and wasm, and that's always been the proper way to write, but wasm exposes issues that happened to work before.)
• The Module API should be exactly the same. No known bugs there AFAIK.
• I'm not familiar with wasm-loader, and not sure what you mean by asm.js bundling?

[buu700]

Awesome, thanks for all the clarification. I wasn't aware of the async compilation requirement, but all things considered it sounds like a net positive if it's offloading work that otherwise would've blocked the main JS thread.

As far as bundling, in my use case it isn't an option for these libraries to request external files at run-time, so they'd need to be distributed as single JS files (as they are now). At a minimum, the wasm would need to be embedded into the JS bundle in some way (wasm-loader just inserts a really big Uint8Array, similarly to how emscripten already handles --memory-init-file 0), but what I meant to suggest was an additional option to bundle in the asm.js fallback for environments where wasm isn't supported. I know this would double the resulting JS size, but at least in my case the tradeoff would be worth it.

[kripken]

An option could be added to bundle the wasm as a typed array, yeah, like memory-init-file 0. Happy to help guide someone interested to add that. We would need to add a clear warning about the downsides of using that option, but I see how it can help in cases where it simply must be one file.

Bundling the asm.js as a fallback is basically already supported, see here. However the asm.js is kept as a side file, so for your use case, that would require another feature to bundle the asm.js as well. I suppose it could be bundled as a string perhaps.

[buu700]

Cool, I wouldn't mind submitting a PR for that if you can point me in the right direction of where to start and any preferences you have on what the new command line flag(s) should look like (name(s), behaviour, arguments, etc.).

As far as asm.js fallback bundling, how about something like this?

if (typeof WebAssembly === 'undefined') {
    Module['asm'] = function () { ... };
}
else {
    Module['wasmBinary'] = new Uint8Array([ ... ]).buffer;
}

[kripken]

How about EMBED_WASM_BINARY as a new option name? Can add it alongside LEGALIZE_JS_FFI in settings.js. Applying it should be done in emcc.py, basically in the wasm section (around where we call asm2wasm) can add the code. Might need to add something in src/preamble.js or postamble.js, that ifdefs on that new option, and emcc.py writes to/replaces.

About the fallback, just doing Module['asm'] = function () { ... }; will still trigger AOT compilation in at least firefox and edge, maybe also chrome. So it should be either a side file, or a string, probably.

[buu700]

Thanks, sounds good. I should be able to take care of this some time in the next week or so. As far as embedding the asm.js fallback, should that be a separate option, or should -s EMBED_WASM_BINARY on its own trigger asm.js embedding as well when -s "BINARYEN_METHOD='native-wasm,asmjs'" (or similar) is used?

re: asm.js and AOT compilation, got it, that makes sense. In that case, how about we just set wasmTextFile, wasmBinaryFile, and asmjsCodeFile to base64 data URIs? I'd wanted to avoid doing anything that would potentially require changes to Content Security Policies, but if there's no practical way around it then this seems like the best solution to me for space efficiency (for the wasm, anyway) and probably simplicity of implementation.

[kripken]

Good point about asm.js too. Perhaps it makes more sense to have an option SINGLE_FILE which would work to embed everything - asm.js, wasm, mem init, file data, etc?

Someone tried something like base64 encoding for the mem init file, and got pretty far but it stalled at some point. I don't remember the details offhand, see MEM_INIT_METHOD option 2, some details in settings.js.

[buu700]

Ah yeah, SINGLE_FILE would directly capture what I'm actually trying to accomplish, so that makes more sense to me. Can I just add this on its own, or would including EMBED_WASM_BINARY as well be preferred? If the former, will --memory-init-file and any other similar existing options ultimately be deprecated?

As far as --memory-init-file 2, assuming it was meant to do something a bit more complicated than replacing the file name with a data URI, should whatever has already been implemented for it be removed after SINGLE_FILE is finished?

[kripken]

Let's just add SINGLE_FILE for now I think, no need for separate options for the wasm binary. Maybe we'll want to add them later. (And yeah, maybe in the long term we'd deprecate separate options, but I'm not sure, that's a separate discussion I guess.)

Yeah, memory init file 2 was more complicated I guess. It tried to compress the size too. We should probably deprecate it eventually, since it was never fully finished anyhow.

[buu700]

Perfect, sounds good. In that case, I'll just add the one SINGLE_FILE option that turns all subresource paths into base64 data URIs without depending on any similar pre-existing options, and whatever happens next with those other options can be handled separately.