Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasmer 3.0.0 and fast data transferral #3274

Closed
Icedude907 opened this issue Nov 3, 2022 · 2 comments
Closed

Wasmer 3.0.0 and fast data transferral #3274

Icedude907 opened this issue Nov 3, 2022 · 2 comments
Labels
priority-high High priority issue ❓ question I've a question! 🏚 stale Inactive issues or PR
Milestone

Comments

@Icedude907
Copy link

Icedude907 commented Nov 3, 2022

Hi there,

I've been experimenting with Wasmer as a hobby, and one of the things I've been doing is moving a large amount of data back and forth.
Previously, I had a function like this which dumped ~100mb of data into the guest.

// Wasmer 2.3.0
#[derive(Copy, Clone)] pub struct Arr<T, const N: usize>(pub [T; N]);
unsafe impl<T: Copy, const N: usize> ValueType for Arr<T, N>{}

fn send_big_buffer(env: &EnvWithMemory, outbuf: WasmPtr<Arr<u8, 104857600>>)
    let mem = unsafe{outbuf.deref(env.memory_ref_unchecked())}.expect("Pointer provided was invalid");
    let mem = unsafe{ mem.get_mut() };

    mem.0.copy_from_slice(&data[..]);
}

Given large api changes (and my utilisation of a depreciated function) this is no longer possible.
As per the comment on #1249, it seems that the correct way to do this is with WasmSlice, so my function becomes:

// Wasmer 3.0.0-rc.2
pub fn send_big_buffer(fnenv: FunctionEnvMut<EnvMem>, outbuf: WasmPtr<[u8;104857600]>){
    let mem = fnenv.data().memory.view(&fnenv);

    let outbuf: WasmPtr<u8> = unsafe{ std::mem::transmute(outbuf) };
    outbuf.slice(&mem, 104857600).unwrap().write_slice(&data[..]).expect("pointer flows out of bounds");
}

I understand these two methods aren't comparable in function (memory safety) - but I can't do the old way anymore and its a non-insugnificant slowdown. Wasm 3.0 on my machine took (dbg)0.38s/(rel)0.021s, whilst 2.3 just took a flat 0.01s.

As it stands, I am unsure if there's a better way to go about this.
Documentation is still in its infancy reguarding 3.0 and lots of it is out of date, so I've had trouble finding info.

Guest code is this for both versions:

fn receive_big_buffer() -> Box::<[u8;104857600]>{
    unsafe{
        let mut buf = Box::<[u8;104857600]>::new([0;104857600]); // Should use new_uninit (nightly)
        super::externs::receive_big_buffer(buf.as_mut());
        return buf;
    }
}
mod externs { extern "C" {
    #[cfg_attr(target_arch = "wasm32", link_name = "receive_big_buffer(ptr)")]
    pub fn receive_big_buffer(outbuf: &mut [u8;104857600]);
}}

Thanks for listening.

@Icedude907 Icedude907 added the ❓ question I've a question! label Nov 3, 2022
@fschutt fschutt added the priority-high High priority issue label Nov 8, 2022
@fschutt fschutt added this to the v3.x milestone Nov 8, 2022
@ptitSeb ptitSeb modified the milestones: v3.x, v4.x May 3, 2023
Copy link

stale bot commented May 3, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the 🏚 stale Inactive issues or PR label May 3, 2024
Copy link

stale bot commented Jun 5, 2024

Feel free to reopen the issue if it has been closed by mistake.

@stale stale bot closed this as completed Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority-high High priority issue ❓ question I've a question! 🏚 stale Inactive issues or PR
Projects
None yet
Development

No branches or pull requests

3 participants