Yielding from host calls

[bkolobara]

I'm currently embedding wasmer into a Rust project. So far I'm really happy with it and I made great progress.

One feature I need though is being able to call from WASM back to Rust and then suspend the executing of WASM until some IO finishes. I'm basically trying to embed wasmer into an async/await environment. From the perspective of wasm it would be a blocking call (runtime suspended). Lucet exposed an API to do this, but I couldn't find anything similar in wasmer.

What would be the best approach to implement something like this? I would also be happy to contribute some code if someone pointed me in the right direction. Thanks!

[syrusakbary]

That would be a great addition. We would love to support yielding from host calls.

I think first, we need to figure out a good API to use it (we can use this issue to make proposals) and then just create a PR to implement it.

There are a few ways we can get it working with the following green-threads/fibers approaches:

• Via libfringe. Cons: only support in *Nix platforms
• Via mioco. Cons: unmantained
• Via genawaiter.

After reviewing Lucet API and all the different yielding libraries implementations, it seems the simplest way to achieve it is via async generators (genawaiter), as it will support any platform and since it relies on native async/await and it's implementation is close to zero-cost.

Here's an example API that I have in mind. Thoughts? @bkolobara @MarkMcCaskey

pub enum Factorial {
    Multiply(u64, u64),
    Result(u64),
}

#![wasmer_generator]
pub unsafe extern "C" fn factorial(
    &mut vmctx,
    n: u64,
) -> u64 {
    let result = if n <= 1 {
        1
    } else {
        let n_rec = factorial(vmctx, n - 1);
        vmctx.yield(Factorial::Multiply(n, n_rec))
    };
    vmctx.yield(Factorial::Result(result))
    result
}

let import_object = imports! {
  factorial => factorial
};

// Run the Wasm
let instance = instantiate(WASM, &import_object)?;

let result = instance
        .dyn_func("run")?
        .call(&[Value::I32(42)])?;

let mut factorials = vec![];

while let Yield(val) = result.resume() {
    match k {
        Factorial::Multiply(n, n_rec) => {
            // guest wants us to multiply for it
            res = inst.resume_with(n * n_rec);
        }
        Factorial::Result(n) => {
            // guest is returning an answer
            factorials.push(*n);
            res = inst.resume();
        }
    }
}

[MarkMcCaskey]

So I haven't had time to really dig into async/await in Rust yet but here are my initial thoughts,

• I think we don't need the unsafe extern "C" part in Wasmer due to the way our imports work
• if we're using procedural macros then these don't have to be methods on Ctx, we can probably just use the yield keyword directly. Alternatively, we can do this without macros.
• we'd be updating the return result of every function with an API like this, which seems hard to avoid in general (though probably possible with a different API, i.e. have a separate case for when no imports have this) just because we can't know the full CFG of the program in general before executing it.
• We should consider integrating with the standard async/await and Future trait now that it's stable (I haven't looked at Lucet's implementation in detail, perhaps they're doing this as well); though admittedly I probably don't understand the proposed feature enough to have a good sense of this
• The way I understand how green-thread like things are implemented would make something like exceptions potentially very complicated when talking about unwinding through host and guest functions. Though perhaps this a larger architectural thing, like maybe we need to virtualize all our stacks and have Wasmer manage everything itself (I think we'll end up here eventually if we want to offer fine-grained control over our system)

[satrobit]

Any update on this?
It looks like a great addition indeed.

[bkolobara]

@satrobit After a few attempts, I did not manage to make it work with the current wasmer architecture. I ended up writing my own WASM runtime with virtual stacks (as @MarkMcCaskey suggested in his comment), that's also how Lucet does it.

I couldn't come up with a proof of concept without virtual stacks. This would probably be a necessary addition to wasmer, before yielding becomes possible

On a side note, implementing my own (pretty limited) async wasm runtime, using Cranelift + Tokio.rs + Lucet inspired virtual stacks, was not as difficult as I anticipated it to be. It could be a viable route to go.

[MarkMcCaskey]

This is something that's still very much on the table for us, we just haven't had the spare resources to focus on it recently! Sorry if this is a blocking issue for you

[slinkydeveloper]

Any updates on this? I heard there is a huge refactoring incoming, will it include this feature?

[kaimast]

I am interested in working on this.

@bkolobara was there a specific thing that blocked you from implementing it or was the codebase just too complicated to get it to work quickly?

[bkolobara]

@bkolobara was there a specific thing that blocked you from implementing it or was the codebase just too complicated to get it to work quickly?

I ended up implementing this as part of the Lunatic project. I wrote something that pretends to be a rust Future and is compatible with Rust's async runtimes, but uses separate stacks to execute Wasmer instances. So I can suspend the instance at any point.

This way I can use async code in Wasmer/Wasmtime host functions with almost zero-cost abstractions, solving my initial problem.

I spent one year thinking about this problem, implementing different solutions and looking what others are doing. My conclusion would be that just running the current Wasmer implementation on top of async-wormhole solves the problem quite elegantly.

[kaimast]

I spent one year thinking about this problem, implementing different solutions and looking what others are doing. My conclusion would be that just running the current Wasmer implementation on top of async-wormhole solves the problem quite elegantly.

I actually ended up doing this yesterday and it seems to work fine indeed. Not sure if we should keep this issue open?

Kind of off-topic:
The WasmerEnv trait (in the 1.0 API) is Sync and Send for some reason, which makes it a little hard to pass the AsyncYielder around. I ended up using unsafe code to get it to work; not sure if you have this issue with wasmtime too.

[bkolobara]

I didn't decide yet what a safe API around the Async Yielder would look like. Would definitely like some feedback and suggestions on this.

[bkolobara]

I actually ended up doing this yesterday and it seems to work fine indeed. Not sure if we should keep this issue open?

There is one thing that I would like to have resolved before closing this issue. Wasmer Trap handling depends on a private thread local variable:
https://github.com/wasmerio/wasmer/blob/master/lib/vm/src/trap/traphandlers.rs#L697

When running in an async context the execution can be moved between threads, invalidating this thread local. I have solved this in a bit of hacky way. To summaries, async-wormhole is moving the TLS when it's moved between threads by the async executor. For this to work I use a fork of Wasmer where this variable is exposed as public.

If there was an API in Wasmer to get/set this TLS I could just directly depend on Wasmer and didn't need to maintain a fork. My question would be how reasonable is it to expect such an safe/unsafe API to be added to Wasmer?

[MarkMcCaskey]

The WasmerEnv trait (in the 1.0 API) is Sync and Send for some reason, which makes it a little hard to pass the AsyncYielder around. I ended up using unsafe code to get it to work

The reason WasmerEnv has to be Send and Sync is the result of the way our API works, but it also future-proofs it for using threads in Wasm, so it seems like a reasonable constraint. For example you can share an Instance between threads and then access a host function on both and call it on both, meaning that the Env is aliased.

It may be possible to have some API support for non-thread safe things but we'd need to internally synchronize it.

---

If there was an API in Wasmer to get/set this TLS I could just directly depend on Wasmer and didn't need to maintain a fork. My question would be how reasonable is it to expect such an safe/unsafe API to be added to Wasmer?

Well we'd definitely rather have this functionality upstream, the issue is just in the implementation: it'd be better if we could keep implementation details internal so we don't break the API when changing them. I don't have a lot of context on this part of the code but I'll ping the team about it

[pmuens]

Hey everyone, just chiming in here since I'm trying to solve the same problem right now.

I looked into Lucet and their implementation and @bkolobara async-wormhole lib which looks really promising. Currently I'm trying to get Wasmer to work with async-wormhole but I'm hitting a roadblock.

I looked into the Lunatic source code but I couldn't figure out how it works exactly. Apparently @kaimast was able to get this working with Wasmer too, so it would be super awesome if one of you could guide me into the right direction.

Here's what I got so far.

The WebAssembly file has 2 functions. compute which is exported to the host and heavy_computation which is imported from the host. heavy_computation is where the async code will be executed.

#[no_mangle]
pub extern "C" fn compute() -> i32 {
    let result = 100;
    result += heavy_computation(200);
    result += 300;
    result += heavy_computation(400);
    result
}

extern "C" {
    fn heavy_computation(a: i32) -> i32;
}

And here's the lib.rs file (the host) where I use Wasmer together with async-wormhole.

use async_wormhole::{stack::{EightMbStack, Stack}, AsyncWormhole, AsyncYielder};
use wasmer::{imports, Function, Instance, Module, Store, Val, Value, WasmerEnv};

#[derive(WasmerEnv, Clone)]
struct Env {
    yielder: AsyncYielder<i32>, // TODO --> This barks right now. How can we securely share it with the guest?
}

// The host function we call in WebAssembly
fn heavy_computation(env: &Env, num: i32) -> i32 {
    let result = env.yielder.async_suspend(async { 42 });
    result + num
}

pub fn core() -> Result<Vec<i32>, Box<dyn std::error::Error>> {
    let env = &Env {
        // TODO --> We somehow have to inject the yielder here...
    };

    let wasm_path = concat!(
        env!("CARGO_MANIFEST_DIR"),
        "/wasm/target/wasm32-unknown-unknown/debug/wasm.wasm"
    );
    let wasm_bytes = std::fs::read(wasm_path)?;

    let store = Store::default();
    let module = Module::new(&store, wasm_bytes)?;

    let import_object = imports! {
        "env" => {
            "heavy_computation" => Function::new_native_with_env(&store, env.clone(), heavy_computation),
        }
    };
    let instance = Instance::new(&module, &import_object)?;

    let mut results: Vec<i32> = Vec::with_capacity(1);

    let stack = EightMbStack::new().unwrap();
    let task = AsyncWormhole::<_, _, fn()>::new(stack, |yielder| {
        // TODO --> Only now do we have access to the yielder
        let func = instance.exports.get_native_function("compute")?;
        func.call().unwrap()
    })
    .unwrap();
    let result = futures::executor::block_on(task);
    assert_eq!(result, 1084);

    results.push(result);

    Ok(results)
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_core() {
        let results = compute().unwrap();

        assert_eq!(results[0], 1084);
    }
}

The main issue I'm facing right now is that I need to pass the yielder into the Wasm environment but I cannot do that since I need to import the functions (which in turn depend on the environment) to create an instance so that I can execute the guest function which kicks-off the whole async flow.

What am I missing here? Is there an easier way to run async code in Wasm?

Thanks in advance for taking the time to look into this!

[kaimast]

As far as I understand, there is not pretty way to do this. You need to use some unsafe code (or at least a bunch of mutexes).

The way I did this basically is that Env holds and Arc<Mutex<Option<AsyncYielder>>> which is initialized as None .
You keep a copy to that arc around and after you initialize the wormhole you set it to hold the actual yielder.

Older versions of Wasmer seemed to have code built-in for creating an execution stack and even storing that stack on disk (see #489 for example).
It seems that most of this stuff has been removed during the "big refactor" last year. I am hoping to eventually open a pull request to re-add those features. However, Wasmer now has support for multiple compilers and engines, which makes this much more complicated.

In the mean time, it might be more straightforward to use wasmtime instead. It seems like @bkolobara tailored async-wormhole towards that library.

[pmuens]

Thanks a lot for getting back and providing the missing pieces @kaimast 👍

Using an Option is a pretty clever hack. I started to implement it this way but ran into some other hiccups so I switched my attention to wastime (as you proposed).

While doing that I stumbled upon bytecodealliance/wasmtime#2434 which looks really promising (also /ccing @bkolobara here in case he missed it).

Would be nice if Wasmer adds support for async functions as well at some point since I really love the Wasmer APIs and ergonomics. Also more than happy to help once this gets reprioritized.

Thanks again for your help.

[bkolobara]

Hi @pmuens,

As @kaimast mentioned already, it's not straight forward to wrap the AsyncYielder inside of a WasmerEnv struct. Lunatic does an unsafe pointer cast to an usize that is stored inside the instance state, and when using an async host function it's casted back.

I also create the instance inside of the wormhole closure to make sure that the instance never outlives the AsyncYielder. Wrapping Stores, Linkers and Instances inside of the closure that is passed to AsyncWormhole::new is especially important in Wasmtime where the types are !Send and !Sync, but it's ok to move all of them at once from thread to thread (what AsyncWormhole) does.

I think that nowadays AsyncWormhole works a bit better with Wasmer, mostly because all the types are Send so that less unsafe wrappers are needed, but I checked out the Wasmtime native async support you linked above and it looks really promising.

For the brave ones :), Lunatic can be also used as a library. If you look at the entry point of Lunatic, it's just a small wrapper around the library. Instead of spawning a process with Process::create and the default API (WASI, networking, etc.) you can just provide your own host functions with Process::create_with_api. We used this to create https://lunatic.run/, where we need to redirect stdin/out to HTTP requests.

One big benefit of using Lunatic is that you get the nice interface of uptown_funk to define regular async functions as host functions. You can also switch between Wasmer or Wasmtime, but only need to provide one implementation for host functions. Lunatic's host functions follow the WASI convention and know how to accept higher level types from pointers. On the other hand, a big drawback is that there is almost no documentation for it now and you will need to find your way through the code.

[pmuens]

Hey @bkolobara,
thanks for getting back and providing such an in-depth explanation. Really appreciate it!

Also looked more into Lunatic which is pretty impressive. I'm working on a project which is based on the Actor-Model, hence the comment on this issue.

I hope that this issue might be picked-up and re-prioritized again in the future. As I said above, I'm more than happy to help once this gets more traction.

[supercmmetry]

Can you please solve this issue? This seems to be a blocker for me.

[syrusakbary]

Hi @supercmmetry we have been working on steps to enable this, I'd say that we are halfway (@Amanieu could probably explain much better than me!)

In any case, we'd love to learn about your use case if you are up for adding more details here :)

[supercmmetry]

@heyjdp @syrusakbary Anything that's pending to resolve this issue? I would be happy to help!

[kaimast]

It seems like most of the heavy lifting is indeed done already.

What seems to be missing is something like Yielder::on_parent_stack_async in corosensei and then adding all the required macros to wasmer to expose async host calls.

Is someone actively working on this right now?
I would love to move my fork to a more recent version of wasmer and can also potentially help.

[supercmmetry]

@syrusakbary I can help too. I need this feature for my project to allow WASM to make HTTP calls.

[cbrzn]

hey guys! just wondering if there are updates regarding this issue 😄 currently need it for my use case 😛

edit: I was able to achieve this behavior by using block_on from futures, you can see the implementation here in case someone is interested :)

[AdamJSoftware]

Any news on this? I see it was pushed back

[kaimast]

I am also curious what the current status of this is? Seems like some stuff landed over a year ago and then work towards async support stopped.

I still use my super outdated fork (from #2219) to provide async support. Rebasing this on the most recent Wasmer has become considerably harder due to the introduction of coresensei.
At this point, I am contemplating a move to a different wasm runtime, but I will hold off if there is hope for async support in wasmer. I would also be more than happy to help with adding such support as mentioned in my last post from June '22.

[thedavidmeister]

@kaimast i think async exists in wasix https://wasix.org/ but i haven't tested it myself

[FranklinWaller]

@thedavidmeister It does have a spawn_await but the import functions are still sync so we cannot await on the result of that spawn_await