research: rust and webasm

[quinn-dougherty]

Do we want to move into Rust's webasm compilation target?

We want to devote at least a couple days of research into grokking the value prop of this approach, and I would recommend:

• In worlds where integrating squiggle with more number-crunchy applications has GPU payout if the compilation target is webasm rather than js, that's a substantial point in favor of this approach
• In worlds where we expect JS to continue dominating the web over the next 15-30 years, there may not be a huge value prop to this approach
• In worlds where we expect webasm to become more fundamental than JS over time, that's a tremendous point in favor of this approach

Let's recall that we're interested in developer ecosystems and web infrastructure in the long term.

[Hazelfire]

I think that moving to Rust has high value, and I believe Ozzie agrees. However, it's not a priority as it currently stands, there are still documentation, testing and cleaning to do before getting to that stage

[quinn-dougherty]

My takeaway from speaking with @OAGr is that there's a research task to determine if it's actually the correct move. I'd like to know what criteria we're looking for before doing carrying out the research.

But what isn't there a concern about investing testing and cleaning into code that ultimately gets wholesale replaced? I might be misunderstanding a thing that makes you not worry about this.

[OAGr]

"about investing testing and cleaning into code that ultimately gets wholesale replaced"

There is, but I'm not too worried.

1. This whole project is very experimental. I wouldn't be too surprised if, after 3-10 months, we realize we could use a heavy refactor/rewrite.

2. The Squiggle language is only one part of the bigger thing. I imagine the frontend, research, and syntax innovations, could all carry over.

3. Even if we do change it, my guess is that having a js/rescript implementation would still be useful for some situations. Like, much of it is just the math library, which is valuable.

4. I really don't want to let the perfect be the enemy of the good. We could really use quick progress in the short-term still. I bet we're not too far away from some showcase demonstrations of value.

5. At this point I think it's kinda unlikely we'll want to choose rust in the next year (60% probability?).

6. Rescript/ocaml are higher-level than Rust, and seem like they'd be quite a bit faster for us to make the first several steps in. (I could be wrong here)

[OAGr]

Just thinking about this;

If this is moved to Rust, I think it could be done incrementally. Begin with the most compute-intensive work (monte carlo, I assume), then try to integrate that into the existing codebase. Expand incrementally.

[quinn-dougherty]

Ok so there are a couple of ways to go about this (epistemic status: worked on some rust/wasm/js interop)

• mc-wasm package in monorepo that squiggle-lang depends on. This would look like putting "mc-wasm": "file:../mc-wasm/pkg" in "dependencies" and cd ../mc-wasm && wasm-pack build prefixing the build step in lang. Then, we can say import "*" as mcWasm from "mc-wasm"; in javascript wherever we like.
  • This sort of implies that as more functionality gets refactored to wasm, we just add more packages to the monorepo, and make them dependencies.
• squiggle-lang/src/rust subdirectory, with some build config that points to src/rust/pkg/output_file.js as a resolve for mc-wasm, and the yarn build command appropriately cding and wasm-pack building (but cd src/rust instead of cd ../subpackage-name.

I think one of these options is the correct way to do montecarlo with caching. In either case, we define an API where rescript ends and rust begins / where js ends and wasm begins.