Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a package resolver #3831

Merged
merged 21 commits into from
May 4, 2023
Merged

Introduce a package resolver #3831

merged 21 commits into from
May 4, 2023

Conversation

Michael-F-Bryan
Copy link
Contributor

@Michael-F-Bryan Michael-F-Bryan commented Apr 28, 2023

Description

This adds a PackageResolver getter to WasiRuntime and wires it up to the ModuleCache.

TL;DR:

  • The WasiRuntime trait now has a package_resolver() method which gives you an Arc<dyn PackageResolver>
  • A PackageResolver will resolve package identifiers to a BinaryPackage (expect the return type to change - eventually I want to return a wasmer_wasix::runtime::resolver::ResolvedPackage)
  • The PluggableRuntime will use the "builtin" resolver from lib/wasi/src/runtime/resolver/builtin.rs by default, this just calls into the existing wasmer_wasix::wapm::fetch_webc() code which uses the WAPM backend and will cache *.webc files to disk
    • Both the cache directory and URL for the WAPM backend are configurable
    • there is a BuiltinResolver::from_env() constructor that will eventually set things up using $WASMER_HOME/wasmer.toml (I've just added a quick'n'dirty implementation for now)
  • The ModuleCache::get_webc() module now just defers to runtime.package_resolver().resolve_package(...) because the PackageResolver implementation should be in charge of caching
  • Now that ModuleCache doesn't keep the in-memory cache of resolved BinaryPackages I've added a with_cache() method that returns InMemoryCache<Self> so you can wrap an existing PackageResolver with an in-memory caching layer
  • Removed the --include-webc flag from wasmer run because it used to explicitly add items to the ModuleCache's WEBC cache and all that code has been deleted (not sure if this should have been a hard delete or if we should warn people when they use it that --include-webc is now a no-op)
  • Removed the engine() getter from the WasiRuntime trait so we can introduce a separate mechanism for module compilation and caching (nothing used this anyway, they just called WasiRuntime::new_store())

CC: #3603.

@Michael-F-Bryan Michael-F-Bryan force-pushed the resolver-abstraction branch from b483615 to f16a6da Compare May 1, 2023 11:14
@Michael-F-Bryan Michael-F-Bryan marked this pull request as ready for review May 1, 2023 11:50
lib/wasi/src/runtime/mod.rs Outdated Show resolved Hide resolved
lib/wasi/src/runtime/resolver/builtin.rs Outdated Show resolved Hide resolved
lib/wasi/src/runtime/resolver/types.rs Show resolved Hide resolved
lib/wasi/src/runtime/resolver/types.rs Outdated Show resolved Hide resolved
lib/wasi/src/runtime/resolver/types.rs Show resolved Hide resolved
lib/wasi/src/runtime/resolver/builtin.rs Outdated Show resolved Hide resolved
lib/wasi/src/runtime/resolver/builtin.rs Outdated Show resolved Hide resolved
@Michael-F-Bryan Michael-F-Bryan force-pushed the resolver-abstraction branch from 76ead4b to fbb72ab Compare May 3, 2023 03:46
@Michael-F-Bryan Michael-F-Bryan enabled auto-merge May 3, 2023 03:48
@Michael-F-Bryan Michael-F-Bryan disabled auto-merge May 3, 2023 03:55
@syrusakbary
Copy link
Member

(let’s not merge until we release Wasmer 3.3)

@Michael-F-Bryan Michael-F-Bryan force-pushed the resolver-abstraction branch from d2862f2 to 2f83b25 Compare May 4, 2023 08:35
@Michael-F-Bryan Michael-F-Bryan merged commit 5cab85f into master May 4, 2023
@Michael-F-Bryan Michael-F-Bryan deleted the resolver-abstraction branch May 4, 2023 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants