Better documentation for WASI/WASM

[brunoluiz]

I am developing a toy project using TinyGo and it works quite well when using as a drop-in replacement for Golang. But, if I try to use only WASI, replacing syscall/js with only what WASI provides, it is not exactly clear how to deal with some aspects of it. TinyGo is short of documentation in this aspect and, from what I searched in the issues here, there are many things that are not as simple to infer.

My suggestions are:

1. Better WASM/WASI documentation: currently it only has a very basic tutorial, without any mentions about the specifics around the data model and limitations of WASI in TinyGo (strings, structs, multi-returns). I found some of these things in Github issues or in some other parts of the documentation, but developers would benefit to have a revamped WASM section, with all these details
2. More WASI examples: there are no examples that touch the memory model, required for things like strings and slices (multi-return as well?). Most examples are quite simple but don't show the real powers of TinyGo and don't show common caveats developers will get into when developing some real apps.
3. More details about optimisation: if people are going to use TinyGo for web projects, optimising the output is a must. There is some material in some blogs, but it would be great if there was some official guidance around it.

Some references I used:

• https://wasmbyexample.dev/examples/webassembly-linear-memory/webassembly-linear-memory.go.en-us.html
• https://elewis.dev/are-we-wasm-yet-part-1
• https://elewis.dev/are-we-wasm-yet-part-2
• https://www.fermyon.com/blog/optimizing-tinygo-wasm

EDIT: As suggested on the PR, we could have some more elaborate WASI + JS examples, such as:

1. Manipulating big blobs of data (eg: image)
2. Manipulating streamed content (io.Reader doesn't seem to be straight forward to use with JS)
3. Receiving complex objects
4. Returning complex objects
5. Returning multiple outputs (simple or complex)
6. Handling errors correctly (returning _, error is not standard in WASI)

[codefromthecrypt]

wazero also maintains this and having written almost all of it, I'm happy to paste it somewhere else also https://wazero.io/languages/tinygo

[brunoluiz]

I guess it would be super helpful to have these closer to the TinyGo documentation. But, especially for developers using JavaScript (as in the current example), there is still no good docs on more complex scenarios for WASI. wazero seems to cover Golang runtime, isn't it the case?

[dgryski]

Easy enough to add some links to the tinygo wasm docs page or equivalent. If it doesn't end up on the tinygo site proper I'm sure we can find some place for it to live.

[codefromthecrypt]

@brunoluiz ack. the wazero page won't get into details of browser use much as it is vast, and wazero embeds in go apps which aren't likely to include a javascript interpreter. the only reason we cover GOOS=js is because Go has no other option at the moment. I've been told that how the js operating system works in TinyGo is different than go, but it is possible that they are similar enough to harvest some of the content, validate if it is relevant, then start a TinyGo page out of it.

[codefromthecrypt]

last thing on wazero's page:

https://wazero.io/languages/tinygo/#system-calls only applies to the wasi target. It isn't used in the wasm target, so it should be irrelevant for js. The system call approach for JS is different enough to probably be better to not say WASI when discussing it.

wazero has a little bit of content on WASI here and I'll add more later, as a top-level page. WASI weirdness applies to all languages that use it and so there's some required reading I think in general before trying to get complex into one language https://wazero.io/specs/#wasi

one "more complex scenarios for WASI" One thing I noticed in writing docs, is that I've never been able to guess what people think. I would recommend that you mention examples of complex scenarios you are interested in. Ex without guessing, what I would probably add examples to the internals part, how to run TinyGo tests for the wasi code, and any other things needed to help develop them. That's because in my world view, that's what I notice developers struggling with.. how to fix a gap, how to know where to start, how to know what tool to use to fix a gap. TL;DR; is maybe add to the description some bullet points of key things you personally would like to see covered, and be very specific.

my 2p!

[brunoluiz]

@codefromthecrypt Hey, I am using WASM just as a side thing, so I wouldn't have plenty of examples to give. I will add some to the description, but probably someone that works daily with this might be better suited for giving "real-life" scenarios/examples.

[mar1n3r0]

Is it safe to assume that those targeting the web are left with the only option to wait for WASI to land in Go so that we have access to both the host operating system and the DOM at the same time without using JS?

Also is it true that TinyGo looks at using WASI for its non-web functionality rather?

[codefromthecrypt]

@mar1n3r0 I'll try to answer..

golang/go and tinygo-org/tinygo are alternative compilers of go source to WebAssembly. Both support GOOS=js, but at the moment only tinygo supports WASI for access to sys calls like opening files.

For example, GOOS=js has custom host functions for that, which aren't compatible with WASI, but support similar sys calls. The versions of the javascript support needed for those are slightly different due to forks

https://github.com/tinygo-org/tinygo/blob/release/targets/wasm_exec.js
https://github.com/golang/go/blob/master/misc/wasm/wasm_exec.js

Note that the js operating system also works in node
https://github.com/golang/go/blob/master/misc/wasm/wasm_exec_node.js

So, if you compile go source via GOOS=js to %.wasm with either, you need to use the correct wasm_exec for your JS runtime, such as a browser or node.

Currently only tinygo can compile to wasi instead, which isn't GOOS=js, rather a fake operating system. To let wasi code work in a browser or node, you need something like wasmer-js. If you are running outside the browser, like embedded in a Go app, you can use something like wazero

Hope this helps!

[mar1n3r0]

Thanks @codefromthecrypt already aware of the js glue. The question is more about wasm and web without js. There was a spec in the past about interface types which would allow for wasm to access the DOM. And wasi for accessing the host os. It would be great if both can be combined for a complete alternative to js in the browser.

From what I understand though wasi is aiming to rather be an alternative to containers outside the browser. What I am missing is if wasm and wasi can be combined for the web making js completely obsolete.

In the grand scheme of things my main problem is that we can't access the DOM with anything else but js. It's become a dangerous monopoly leading to no choice for frontend dev.

Perhaps I should paraphrase to: can tinygo target wasi compile to wasm which is for the web?

References: golang/go#31105
#2910

[codefromthecrypt]

I think I need to defer to others, but last hint is that the people who maintain WASI are using interface types. wit-bindgen could have a plugin to generate tinygo stubs alongside other languages. This might be a breadcrumb helpful to figure out the DOM story (which would need host imports I think and in that case would be WASI+something else, but I could be wrong on that)

[mar1n3r0]

Thanks. Seems to be not applicable for web as it works directly with the WebAssembly language and .wit and .wat rather than Go to wasm compilation.

I suppose the solution will come from Go rather than from bytecodealliance for the web use case.

[deadprogram]

tinygo-org/tinygo-site#352 addresses some of the specific points in this issue. I am marking this one to be closed on next release. Thank you everyone!

[deadprogram]

This is part of the v0.28 release so now closing this issue. Thanks!