Tracking issue for implementing the component model #4185
Labels
wasm-proposal:component-model
Issues related to the WebAssembly Component Model proposal
wasmtime
Issues about wasmtime that don't fall into another label
Comments
alexcrichton
added
wasmtime
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this issue
Jun 17, 2022
This commit is an implementation of a number of features of the
component model including:
* Defining nested components
* Outer aliases to components and modules
* Instantiating nested components
The implementation here is intended to be a foundational pillar of
Wasmtime's component model support since recursion and nested components
are the bread-and-butter of the component model. At a high level the
intention for the component model implementation in Wasmtime has long
been that the recursive nature of components is "erased" at compile time
to something that's more optimized and efficient to process. This commit
ended up exemplifying this quite well where the vast majority of the
internal changes here are in the "compilation" phase of a component
rather than the runtime instantiation phase. The support in the
`wasmtime` crate, the runtime instantiation support, only had minor
updates here while the internals of translation have seen heavy updates.
The `translate` module was greatly refactored here in this commit.
Previously it would, as a component is parsed, create a final
`Component` to hand off to trampoline compilation and get persisted at
runtime. Instead now it's a thin layer over `wasmparser` which simply
records a list of `LocalInitializer` entries for how to instantiate the
component and its index spaces are built. This internal representation
of the instantiation of a component is pretty close to the binary format
intentionally.
Instead of performing dataflow legwork the `translate` phase of a
component is now responsible for two primary tasks:
1. All components and modules are discovered within a component. They're
assigned `Static{Component,Module}Index` depending on where they're
found and a `{Module,}Translation` is prepared for each one. This
"flattens" the recursive structure of the binary into an indexed list
processable later.
2. The lexical scope of components is managed here to implement outer
module and component aliases. This is a significant design
implementation because when closing over an outer component or module
that item may actually be imported or something like the result of a
previous instantiation. This means that the capture of
modules and components is both a lexical concern as well as a runtime
concern. The handling of the "runtime" bits are handled in the next
phase of compilation.
The next and currently final phase of compilation is a new pass where
much of the historical code in `translate.rs` has been moved to (but
heavily refactored). The goal of compilation is to produce one "flat"
list of initializers for a component (as happens prior to this PR) and
to achieve this an "inliner" phase runs which runs through the
instantiation process at compile time to produce a list of initializers.
This `inline` module is the main addition as part of this PR and is now
the workhorse for dataflow analysis and tracking what's actually
referring to what.
During the `inline` phase the local initializers recorded in the
`translate` phase are processed, in sequence, to instantiate a
component. Definitions of items are tracked to correspond to their root
definition which allows seeing across instantiation argument boundaries
and such. Handling "upvars" for component outer aliases is handled in
the `inline` phase as well by creating state for a component whenever a
component is defined as was recorded during the `translate` phase.
Finally this phase is chiefly responsible for doing all string-based
name resolution at compile time that it can. This means that at runtime
no string maps will need to be consulted for item exports and such.
The final result of inlining is a list of "global initializers" which is
a flat list processed during instantiation time. These are almost
identical to the initializers that were processed prior to this PR.
There are certainly still more gaps of the component model to implement
but this should be a major leg up in terms of functionality that
Wasmtime implements. This commit, however leaves behind a "hole" which
is not intended to be filled in at this time, namely importing and
exporting components at the "root" level from and to the host. This is
tracked and explained in more detail as part of bytecodealliance#4283.
cc bytecodealliance#4185 as this completes a number of items there
alexcrichton
added a commit
that referenced
this issue
Jun 21, 2022
* Add support for nested components
This commit is an implementation of a number of features of the
component model including:
* Defining nested components
* Outer aliases to components and modules
* Instantiating nested components
The implementation here is intended to be a foundational pillar of
Wasmtime's component model support since recursion and nested components
are the bread-and-butter of the component model. At a high level the
intention for the component model implementation in Wasmtime has long
been that the recursive nature of components is "erased" at compile time
to something that's more optimized and efficient to process. This commit
ended up exemplifying this quite well where the vast majority of the
internal changes here are in the "compilation" phase of a component
rather than the runtime instantiation phase. The support in the
`wasmtime` crate, the runtime instantiation support, only had minor
updates here while the internals of translation have seen heavy updates.
The `translate` module was greatly refactored here in this commit.
Previously it would, as a component is parsed, create a final
`Component` to hand off to trampoline compilation and get persisted at
runtime. Instead now it's a thin layer over `wasmparser` which simply
records a list of `LocalInitializer` entries for how to instantiate the
component and its index spaces are built. This internal representation
of the instantiation of a component is pretty close to the binary format
intentionally.
Instead of performing dataflow legwork the `translate` phase of a
component is now responsible for two primary tasks:
1. All components and modules are discovered within a component. They're
assigned `Static{Component,Module}Index` depending on where they're
found and a `{Module,}Translation` is prepared for each one. This
"flattens" the recursive structure of the binary into an indexed list
processable later.
2. The lexical scope of components is managed here to implement outer
module and component aliases. This is a significant design
implementation because when closing over an outer component or module
that item may actually be imported or something like the result of a
previous instantiation. This means that the capture of
modules and components is both a lexical concern as well as a runtime
concern. The handling of the "runtime" bits are handled in the next
phase of compilation.
The next and currently final phase of compilation is a new pass where
much of the historical code in `translate.rs` has been moved to (but
heavily refactored). The goal of compilation is to produce one "flat"
list of initializers for a component (as happens prior to this PR) and
to achieve this an "inliner" phase runs which runs through the
instantiation process at compile time to produce a list of initializers.
This `inline` module is the main addition as part of this PR and is now
the workhorse for dataflow analysis and tracking what's actually
referring to what.
During the `inline` phase the local initializers recorded in the
`translate` phase are processed, in sequence, to instantiate a
component. Definitions of items are tracked to correspond to their root
definition which allows seeing across instantiation argument boundaries
and such. Handling "upvars" for component outer aliases is handled in
the `inline` phase as well by creating state for a component whenever a
component is defined as was recorded during the `translate` phase.
Finally this phase is chiefly responsible for doing all string-based
name resolution at compile time that it can. This means that at runtime
no string maps will need to be consulted for item exports and such.
The final result of inlining is a list of "global initializers" which is
a flat list processed during instantiation time. These are almost
identical to the initializers that were processed prior to this PR.
There are certainly still more gaps of the component model to implement
but this should be a major leg up in terms of functionality that
Wasmtime implements. This commit, however leaves behind a "hole" which
is not intended to be filled in at this time, namely importing and
exporting components at the "root" level from and to the host. This is
tracked and explained in more detail as part of #4283.
cc #4185 as this completes a number of items there
* Tweak code to work on stable without warning
* Review comments
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this issue
Jun 22, 2022
This commit implements the `post-return` feature of the canonical ABI in the component model. This attribute is an optionally-specified function which is to be executed after the return value has been processed by the caller to optionally clean-up the return value. This enables, for example, returning an allocated string and the host then knows how to clean it up to prevent memory leaks in the original module. The API exposed in this PR changes the prior `TypedFunc::call` API in behavior but not in its signature. Previously the `TypedFunc::call` method would set the `may_enter` flag on the way out, but now that operation is deferred until a new `TypedFunc::post_return` method is called. This means that once a method on an instance is invoked then nothing else can be done on the instance until the `post_return` method is called. Note that the method must be called irrespective of whether the `post-return` canonical ABI option was specified or not. Internally wasm will be invoked if necessary. This is a pretty wonky and unergonomic API to work with. For now I couldn't think of a better alternative that improved on the ergonomics. In the theory that the raw Wasmtime bindings for a component may not be used all that heavily (instead `wit-bindgen` would largely be used) I'm hoping that this isn't too much of an issue in the future. cc bytecodealliance#4185
14 tasks
alexcrichton
added a commit
to alexcrichton/wasmtime
that referenced
this issue
Jun 23, 2022
This commit implements the `post-return` feature of the canonical ABI in the component model. This attribute is an optionally-specified function which is to be executed after the return value has been processed by the caller to optionally clean-up the return value. This enables, for example, returning an allocated string and the host then knows how to clean it up to prevent memory leaks in the original module. The API exposed in this PR changes the prior `TypedFunc::call` API in behavior but not in its signature. Previously the `TypedFunc::call` method would set the `may_enter` flag on the way out, but now that operation is deferred until a new `TypedFunc::post_return` method is called. This means that once a method on an instance is invoked then nothing else can be done on the instance until the `post_return` method is called. Note that the method must be called irrespective of whether the `post-return` canonical ABI option was specified or not. Internally wasm will be invoked if necessary. This is a pretty wonky and unergonomic API to work with. For now I couldn't think of a better alternative that improved on the ergonomics. In the theory that the raw Wasmtime bindings for a component may not be used all that heavily (instead `wit-bindgen` would largely be used) I'm hoping that this isn't too much of an issue in the future. cc bytecodealliance#4185
alexcrichton
added a commit
that referenced
this issue
Jun 23, 2022
This commit implements the `post-return` feature of the canonical ABI in the component model. This attribute is an optionally-specified function which is to be executed after the return value has been processed by the caller to optionally clean-up the return value. This enables, for example, returning an allocated string and the host then knows how to clean it up to prevent memory leaks in the original module. The API exposed in this PR changes the prior `TypedFunc::call` API in behavior but not in its signature. Previously the `TypedFunc::call` method would set the `may_enter` flag on the way out, but now that operation is deferred until a new `TypedFunc::post_return` method is called. This means that once a method on an instance is invoked then nothing else can be done on the instance until the `post_return` method is called. Note that the method must be called irrespective of whether the `post-return` canonical ABI option was specified or not. Internally wasm will be invoked if necessary. This is a pretty wonky and unergonomic API to work with. For now I couldn't think of a better alternative that improved on the ergonomics. In the theory that the raw Wasmtime bindings for a component may not be used all that heavily (instead `wit-bindgen` would largely be used) I'm hoping that this isn't too much of an issue in the future. cc #4185
This was referenced Jun 23, 2022
|
Hi, do we have a rough estimate on the component-model release? I am trying to understand even there's some pending items, is it working now? Is there a quickstart guidance on the usage? |
|
There's no esimate at this time. Some parts work but not all, the un-finished issues above should give an idea of how much is left to do. No documentation on usage yet. |
2 tasks
|
Once all the boxes are checked don't forget a 'hello component' minimal example. |
|
Hey @alexcrichton, what's the relationship of this issue with #6370? Are you still actively tracking issues here, or is the state of landing WASI preview 2 support a better proxy for component model progress in wasmtime? |
|
It depends on why you need to know. If you are trying to contribute to the component model or WASI design and implementation, they are separate standards where WASI builds on the component model, but only using the pieces that are complete in this list. For example, now that component model resources are complete here in wasmtime and are nearing completion out in wit-bindgen and other tools, we'll be swapping out our u32 placeholders for proper resources in WASI shortly. If you are just looking to write or embed components, you want both the component model and WASI to be stable: right now their definitions are still in flux, and wasmtime's implementations may not even match the spec definitions. We intend to stabilize both of those, in both spec and implementation, as Preview 2 later this year. The full set of implementation work required to finish Preview 2 isn't captured anywhere right now, but we do have a rough plan on the spec side: WebAssembly/WASI#550 |
|
I'll add to what @pchickey said already and note that this issue has I think outlived its usefulness. At this time all major efforts for implementing the component model have finished except for the never-ending continued polish in addition to the preview2 implementation (which involves shaping and stabilizing preview2 as well). In light of that I'm going to go ahead and close this as the component model implementation in Wasmtime is more-or-less complete (with usefulness coming with other preview2-related things). Any follow-ups and remaining items should be opened as separate issues at this point for separate discussion. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is intended to be a tracking issue for the implementation of the component model proposal of WebAssembly. I'm in the progress of implementing this with work in Wasmtime starting at #4005 and continuing in further PRs. I hope to leave checkbox items here to ensure I don't forget about them and to also track the progress of implementing the component model.
Items to be implemented
wasmparsercomponentsubmodule of thewasmtimecrateComponentValueinto lifting and lowering traits&[u64]views and similarDouble-check behavior of traps with respect to validating in-bounds pointers and intermediate writes of params/results when lifting/loweringgiven the "trap lock" I don't think that this mattersunimplemented!for outer core aliasesunit, etc.wasmtimecrate API)Separate work items
I'm starting to separate off work items from this issue to separate tracking issues with more detail. These will all need completion to finish the component model work but these issues are suitable to be picked up by anyone in theory.
record,variant,enum,union,flagsUnsure how to implement so far
valueranging from imports to exports to thestartfunctionOpen questions
&[u8]to WebAssembly to amemcpy?How to handle validation ofValue<T>coming out of wasm?Not planned at this time
The text was updated successfully, but these errors were encountered: