Evolving the API of the wasmtime crate
#708
Comments
|
One of the things I'm interested in doing is creating a custom sandbox, where I provide some custom functions for a wasm module to import, and some wasi functions (but maybe not all of the wasi functions). I think your One of the things that confused me about the current version of fn call(&self, _params: &[Val], results: &mut [Val]) -> Result<(), HostRef<Trap>> {
if let Some(Val::I32(v)) = results.get_mut(0) {
*v = 42;
} // else return trap
}The runtime error you get here is fairly inscrutable. I don't have a concrete proposal here, but I wanted to mention this minor papercut. |
|
As I was hiding HostRef in the Module API in my PR #696 I noticed that once I removed the |
|
I've been looking for a scripting language to embed into some of my Rust game development projects, to facilitate scripting (with hot reloading!) custom logic and events for levels. Along the way, I've tried a bunch of different languages and have developed opinions about their APIs. So without further ado: A review of some Rusty embeddable scripting language APIs. Dyonhttps://github.com/PistonDevelopers/dyon/blob/master/examples/call.rs Forgehttps://github.com/zesterer/forge/blob/master/examples/callbacks.rs Rhaihttps://github.com/jonathandturner/rhai PyO3https://pyo3.rs/master/python_from_rust.html The ultimate solution to this problem, of course, is wasmtime, which is why I'm excited to see the Interface Types proposal implemented; that should remove most of the barriers that prevent its use in my (and the rest of the Rust gamedev community's) projects. |
I would love to see functions provided from the host/embedder in the style of module exports, with interface types and names, and then have wasmtime handle name resolution. Once we have linking, what if we have a new type for a "HostModule", with a module name and a set of named exported functions with types? |
|
@eminence good point! I think that the details of @cedric-h I don't think we'll want to add @joshtriplett I've started a dedicated discussion for that at #727 since I think it'll be a big topic, want to take a look at the proposal there and see if it sounds like it'll meet your needs? |
This commit removes the need to use `HostRef<Engine>` in the Rust API. Usage is retained in the C API in one location, but otherwise `Engine` can always be used directly. This is the first step of progress on bytecodealliance#708 for the `Engine` type. Changes here include: * `Engine` is now `Clone`, and is documented as being cheap. It's not intended that cloning an engine creates a deep copy. * `Engine` is now both `Send` and `Sync`, and asserted to be so. * Usage of `Engine` in APIs no longer requires or uses `HostRef`.
This commit removes the need to use `HostRef<Engine>` in the Rust API. Usage is retained in the C API in one location, but otherwise `Engine` can always be used directly. This is the first step of progress on #708 for the `Engine` type. Changes here include: * `Engine` is now `Clone`, and is documented as being cheap. It's not intended that cloning an engine creates a deep copy. * `Engine` is now both `Send` and `Sync`, and asserted to be so. * Usage of `Engine` in APIs no longer requires or uses `HostRef`.
This commit removes the public API usage of the internal `CompilationStrategy` enumeration from the `Config` type in the `wasmtime` crate. To do this the `enum` was copied locally into the crate and renamed `Strategy`. The high-level description of this change is: * The `Config::strategy` method now takes a locally-defined `Strategy` enumeration instead of an internal type. * The contents of `Strategy` are always the same, not relying on Cargo features to indicate which variants are present. This avoids unnecessary downstream `#[cfg]`. * A `lightbeam` feature was added to the `wasmtime` crate itself to lightbeam compilation support. * The `Config::strategy` method is now fallible. It returns a runtime error if support for the selected strategy wasn't compiled in. * The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely add variants over time to it. This reduces the public crate dependencies of the `wasmtime` crate itself, removing the need to reach into internal crates even more! cc bytecodealliance#708
This commit removes the public API usage of the internal `CompilationStrategy` enumeration from the `Config` type in the `wasmtime` crate. To do this the `enum` was copied locally into the crate and renamed `Strategy`. The high-level description of this change is: * The `Config::strategy` method now takes a locally-defined `Strategy` enumeration instead of an internal type. * The contents of `Strategy` are always the same, not relying on Cargo features to indicate which variants are present. This avoids unnecessary downstream `#[cfg]`. * A `lightbeam` feature was added to the `wasmtime` crate itself to lightbeam compilation support. * The `Config::strategy` method is now fallible. It returns a runtime error if support for the selected strategy wasn't compiled in. * The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely add variants over time to it. This reduces the public crate dependencies of the `wasmtime` crate itself, removing the need to reach into internal crates even more! cc bytecodealliance#708
* Remove usage of `CompilationStrategy` from `Config` This commit removes the public API usage of the internal `CompilationStrategy` enumeration from the `Config` type in the `wasmtime` crate. To do this the `enum` was copied locally into the crate and renamed `Strategy`. The high-level description of this change is: * The `Config::strategy` method now takes a locally-defined `Strategy` enumeration instead of an internal type. * The contents of `Strategy` are always the same, not relying on Cargo features to indicate which variants are present. This avoids unnecessary downstream `#[cfg]`. * A `lightbeam` feature was added to the `wasmtime` crate itself to lightbeam compilation support. * The `Config::strategy` method is now fallible. It returns a runtime error if support for the selected strategy wasn't compiled in. * The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely add variants over time to it. This reduces the public crate dependencies of the `wasmtime` crate itself, removing the need to reach into internal crates even more! cc #708 * Fix fuzz targets * Update nightly used to build releases * Run rustfmt
This commit goes through the public API of the `wasmtime` crate and removes the need for `HostRef<Store>`, as discussed in bytecodealliance#708. This commit is accompanied with a few changes: * The `Store` type now also implements `Default`, creating a new `Engine` with default settings and returning that. * The `Store` type now implements `Clone`, and is documented as being a "cheap clone" aka being reference counted. As before there is no supported way to create a deep clone of a `Store`. * All APIs take/return `&Store` or `Store` instead of `HostRef<Store>`, and `HostRef<T>` is left as purely a detail of the C API. * The `global_exports` function is tagged as `#[doc(hidden)]` for now while we await its removal. * The `Store` type is not yet `Send` nor `Sync` due to the usage of `global_exports`, but it is intended to become so eventually.
This commit goes through the public API of the `wasmtime` crate and removes the need for `HostRef<Store>`, as discussed in bytecodealliance#708. This commit is accompanied with a few changes: * The `Store` type now also implements `Default`, creating a new `Engine` with default settings and returning that. * The `Store` type now implements `Clone`, and is documented as being a "cheap clone" aka being reference counted. As before there is no supported way to create a deep clone of a `Store`. * All APIs take/return `&Store` or `Store` instead of `HostRef<Store>`, and `HostRef<T>` is left as purely a detail of the C API. * The `global_exports` function is tagged as `#[doc(hidden)]` for now while we await its removal. * The `Store` type is not yet `Send` nor `Sync` due to the usage of `global_exports`, but it is intended to become so eventually.
This commit goes through the public API of the `wasmtime` crate and removes the need for `HostRef<Store>`, as discussed in bytecodealliance#708. This commit is accompanied with a few changes: * The `Store` type now also implements `Default`, creating a new `Engine` with default settings and returning that. * The `Store` type now implements `Clone`, and is documented as being a "cheap clone" aka being reference counted. As before there is no supported way to create a deep clone of a `Store`. * All APIs take/return `&Store` or `Store` instead of `HostRef<Store>`, and `HostRef<T>` is left as purely a detail of the C API. * The `global_exports` function is tagged as `#[doc(hidden)]` for now while we await its removal. * The `Store` type is not yet `Send` nor `Sync` due to the usage of `global_exports`, but it is intended to become so eventually.
* Remove the need for `HostRef<Store>` This commit goes through the public API of the `wasmtime` crate and removes the need for `HostRef<Store>`, as discussed in #708. This commit is accompanied with a few changes: * The `Store` type now also implements `Default`, creating a new `Engine` with default settings and returning that. * The `Store` type now implements `Clone`, and is documented as being a "cheap clone" aka being reference counted. As before there is no supported way to create a deep clone of a `Store`. * All APIs take/return `&Store` or `Store` instead of `HostRef<Store>`, and `HostRef<T>` is left as purely a detail of the C API. * The `global_exports` function is tagged as `#[doc(hidden)]` for now while we await its removal. * The `Store` type is not yet `Send` nor `Sync` due to the usage of `global_exports`, but it is intended to become so eventually. * Touch up comments on some examples * Run rustfmt
This commit continues previous work and also bytecodealliance#708 by removing the need to use `HostRef<Module>` in the API of the `wasmtime` crate. The API changes performed here are: * The `Module` type is now itself internally reference counted. * The `Module::store` function now returns the `Store` that was used to create a `Module` * Documentation for `Module` and its methods have been expanded.
This commit removes the `unsafe impl Send` for `Trap` by removing the internal `HostRef` and leaving `HostRef` entirely as an implementation detail of the C API. cc bytecodealliance#708
This commit removes the need to use `HostRef<Engine>` in the Rust API. Usage is retained in the C API in one location, but otherwise `Engine` can always be used directly. This is the first step of progress on bytecodealliance#708 for the `Engine` type. Changes here include: * `Engine` is now `Clone`, and is documented as being cheap. It's not intended that cloning an engine creates a deep copy. * `Engine` is now both `Send` and `Sync`, and asserted to be so. * Usage of `Engine` in APIs no longer requires or uses `HostRef`.
…#764) * Remove usage of `CompilationStrategy` from `Config` This commit removes the public API usage of the internal `CompilationStrategy` enumeration from the `Config` type in the `wasmtime` crate. To do this the `enum` was copied locally into the crate and renamed `Strategy`. The high-level description of this change is: * The `Config::strategy` method now takes a locally-defined `Strategy` enumeration instead of an internal type. * The contents of `Strategy` are always the same, not relying on Cargo features to indicate which variants are present. This avoids unnecessary downstream `#[cfg]`. * A `lightbeam` feature was added to the `wasmtime` crate itself to lightbeam compilation support. * The `Config::strategy` method is now fallible. It returns a runtime error if support for the selected strategy wasn't compiled in. * The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely add variants over time to it. This reduces the public crate dependencies of the `wasmtime` crate itself, removing the need to reach into internal crates even more! cc bytecodealliance#708 * Fix fuzz targets * Update nightly used to build releases * Run rustfmt
* Remove the need for `HostRef<Store>` This commit goes through the public API of the `wasmtime` crate and removes the need for `HostRef<Store>`, as discussed in bytecodealliance#708. This commit is accompanied with a few changes: * The `Store` type now also implements `Default`, creating a new `Engine` with default settings and returning that. * The `Store` type now implements `Clone`, and is documented as being a "cheap clone" aka being reference counted. As before there is no supported way to create a deep clone of a `Store`. * All APIs take/return `&Store` or `Store` instead of `HostRef<Store>`, and `HostRef<T>` is left as purely a detail of the C API. * The `global_exports` function is tagged as `#[doc(hidden)]` for now while we await its removal. * The `Store` type is not yet `Send` nor `Sync` due to the usage of `global_exports`, but it is intended to become so eventually. * Touch up comments on some examples * Run rustfmt
This commit continues previous work and also bytecodealliance#708 by removing the need to use `HostRef<Module>` in the API of the `wasmtime` crate. The API changes performed here are: * The `Module` type is now itself internally reference counted. * The `Module::store` function now returns the `Store` that was used to create a `Module` * Documentation for `Module` and its methods have been expanded.
* Remove the need for `HostRef<Module>` This commit continues previous work and also #708 by removing the need to use `HostRef<Module>` in the API of the `wasmtime` crate. The API changes performed here are: * The `Module` type is now itself internally reference counted. * The `Module::store` function now returns the `Store` that was used to create a `Module` * Documentation for `Module` and its methods have been expanded. * Fix compliation of test programs harness * Fix the python extension * Update `CodeMemory` to be `Send + Sync` This commit updates the `CodeMemory` type in wasmtime to be both `Send` and `Sync` by updating the implementation of `Mmap` to not store raw pointers. This avoids the need for an `unsafe impl` and leaves the unsafety as it is currently. * Fix a typo
This commit continues previous work and also #708 by removing the need to use `HostRef<Module>` in the API of the `wasmtime` crate. The API changes performed here are: * The `Module` type is now itself internally reference counted. * The `Module::store` function now returns the `Store` that was used to create a `Module` * Documentation for `Module` and its methods have been expanded.
This commit removes all remaining usages of `HostRef` in the public API
of the `wasmtime` crate. This involved a number of API decisions such
as:
* None of `Func`, `Global`, `Table`, or `Memory` are wrapped in `HostRef`
* All of `Func`, `Global`, `Table`, and `Memory` implement `Clone` now.
* Methods called `type` are renamed to `ty` to avoid typing `r#type`.
* Methods requiring mutability for external items now no longer require
mutability. The mutable reference here is sort of a lie anyway since
the internals are aliased by the underlying module anyway. This
affects:
* `Table::set`
* `Table::grow`
* `Memory::grow`
* `Instance::set_signal_handler`
* The `Val::FuncRef` type is now no longer automatically coerced to
`AnyRef`. This is technically a breaking change which is pretty bad,
but I'm hoping that we can live with this interim state while we sort
out the `AnyRef` story in general.
* The implementation of the C API was refactored and updated in a few
locations to account for these changes:
* Accessing the exports of an instance are now cached to ensure we
always hand out the same `HostRef` values.
* `wasm_*_t` for external values no longer have internal cache,
instead they all wrap `wasm_external_t` and have an unchecked
accessor for the underlying variant (since the type is proof that
it's there). This makes casting back and forth much more trivial.
This is all related to bytecodealliance#708 and while there's still more work to be done
in terms of documentation, this is the major bulk of the rest of the
implementation work on bytecodealliance#708 I believe.
This commit removes all remaining usages of `HostRef` in the public API
of the `wasmtime` crate. This involved a number of API decisions such
as:
* None of `Func`, `Global`, `Table`, or `Memory` are wrapped in `HostRef`
* All of `Func`, `Global`, `Table`, and `Memory` implement `Clone` now.
* Methods called `type` are renamed to `ty` to avoid typing `r#type`.
* Methods requiring mutability for external items now no longer require
mutability. The mutable reference here is sort of a lie anyway since
the internals are aliased by the underlying module anyway. This
affects:
* `Table::set`
* `Table::grow`
* `Memory::grow`
* `Instance::set_signal_handler`
* The `Val::FuncRef` type is now no longer automatically coerced to
`AnyRef`. This is technically a breaking change which is pretty bad,
but I'm hoping that we can live with this interim state while we sort
out the `AnyRef` story in general.
* The implementation of the C API was refactored and updated in a few
locations to account for these changes:
* Accessing the exports of an instance are now cached to ensure we
always hand out the same `HostRef` values.
* `wasm_*_t` for external values no longer have internal cache,
instead they all wrap `wasm_external_t` and have an unchecked
accessor for the underlying variant (since the type is proof that
it's there). This makes casting back and forth much more trivial.
This is all related to bytecodealliance#708 and while there's still more work to be done
in terms of documentation, this is the major bulk of the rest of the
implementation work on bytecodealliance#708 I believe.
* Remove `HostRef` from the `wasmtime` public API
This commit removes all remaining usages of `HostRef` in the public API
of the `wasmtime` crate. This involved a number of API decisions such
as:
* None of `Func`, `Global`, `Table`, or `Memory` are wrapped in `HostRef`
* All of `Func`, `Global`, `Table`, and `Memory` implement `Clone` now.
* Methods called `type` are renamed to `ty` to avoid typing `r#type`.
* Methods requiring mutability for external items now no longer require
mutability. The mutable reference here is sort of a lie anyway since
the internals are aliased by the underlying module anyway. This
affects:
* `Table::set`
* `Table::grow`
* `Memory::grow`
* `Instance::set_signal_handler`
* The `Val::FuncRef` type is now no longer automatically coerced to
`AnyRef`. This is technically a breaking change which is pretty bad,
but I'm hoping that we can live with this interim state while we sort
out the `AnyRef` story in general.
* The implementation of the C API was refactored and updated in a few
locations to account for these changes:
* Accessing the exports of an instance are now cached to ensure we
always hand out the same `HostRef` values.
* `wasm_*_t` for external values no longer have internal cache,
instead they all wrap `wasm_external_t` and have an unchecked
accessor for the underlying variant (since the type is proof that
it's there). This makes casting back and forth much more trivial.
This is all related to #708 and while there's still more work to be done
in terms of documentation, this is the major bulk of the rest of the
implementation work on #708 I believe.
* More API updates
* Run rustfmt
* Fix a doc test
* More test updates
This can be inferred from the `Module` argument. Additionally add a `store` accessor to an `Instance` in case it's needed to instantiate another `Module`. cc bytecodealliance#708
alexcrichton
added
the
wasmtime:api
* Don't require `Store` in `Instance` constructor This can be inferred from the `Module` argument. Additionally add a `store` accessor to an `Instance` in case it's needed to instantiate another `Module`. cc #708 * Update more constructors * Fix a doctest * Don't ignore store in `wasm_instance_new` * Run rustfmt
|
With #814 I believe everything here is either fixed or split out into a separate issue, so I'm going to close! |
I've been reviewing the
wasmtimecrate from a Rust API perspective and ended up realizing that there's actually quite a few changes that I would like to make to the crate. I think that these changes are far too large so simply send in a PR, so I wanted to make sure that we had some discussion of this first!In this issue I hope to lay out a vision for an end-state
wasmtimecrate and what the API might look like. I'm assuming that we can incrementally reach this end-goal over time and the exact route through which we get here isn't too too important. In any case, I'm curious if others have thoughts on all this!Some of these items below may warrant their own separate issue as well, but I wanted to make sure that I had this all written down in one location first
High-level changes
wasmtime-*crates should be private dependenciesStoreshould implementDefaultSendandSyncof all typesHostRef,HostRefis hiddenInstanceis no longer reference counted to ensure that it'sSendmeaning globals/tables/functions are no longer reference counted either.Configwasmtime-*typesSendandSyncEngineDefault(uses defaultConfig)SendandSyncClone)StoreDefault(uses defaultConfigandEngine)Should be(moved to TheSendandSyncwasmtime::Storetype should be thread-safe #777)Clone)wasmtime-*accessorsModuleShould be(covered by TheSendandSyncwasmtime::Storetype should be thread-safe #777)Clone)&StoreaccessorInstanceStore(inferred fromModule)Trap(handled ifTrapis anError)moduleandstoreaccessorSend~~ (moved to Reconcilingwasmtime::InstanceandSend#793)find_export_by_name=>get_exportGlobal,Table,MemoryMemoryshould be fine to leave with an internalArcEverything should be(moved to ReconcilingSend, onlyMemoryshould beSyncwasmtime::InstanceandSend#793)Table::growshouldn't needmutGlobal::setshouldn't needmuttypemethods renamed totyMemory::growshouldn't needmutThese types are only accessed via a reference when fetched through an
Instance.FuncThis type I think needs a lot of improvements. I think it's best to separate out these concerns into a separate issue, however. Some high-level unbaked thoughts are:
usize) and registering it with a type signature. This would skip all trampolines/etc and we'd document the expected ABI. This would be anunsafemethod.MemoryType,TableType,GlobalTypeMutability, otherwise seems good!ImportType,ExportTypeFuncTypeStore, but probably fine for now.TrapErrortraitClone,Send, andSyncValAnyRefwill become some custom form ofRcdefined bywasmtimeitselfBox<u128>for now to avoid blowing up the sizeValan entirely opaque struct, perhaps with adecodemethod which returns a fullenum. We may not want to commit to a public byte-by-byte representation of aValjust yet. This seems like an advanced concern we can punt though.WASI
wasmtimecratefn wasi_unstable(&WasiConfiguration) -> HashMap<String, Extern>Instancesomehow?Name Resolution
Instantiation is a bit wonky where you have to line up imports 1:1 with the expected imports of the module. We should explore ideas where we have a more name resolution based mechanism which leverages the module system. Would perhaps make it much easier to slot in WASI or slot in a module. Pretty tricky API though so we'd have to think elsewhere about this.
The text was updated successfully, but these errors were encountered: