Introduce a module cache abstraction #3841

Michael-F-Bryan · 2023-05-03T07:28:21Z

This PR adds an abstraction for caching modules to the WasiRuntime, plus some implementations that can be composed to implement tiered module caching.

TL;DR:

Introduce a CompiledModuleCache trait that has async save() and load() methods
- Both methods accept a &dyn VirtualTaskManager at the moment in case we need to spawn blocking operations, but that'll probably be moved around
Add a and_then() combinator to CompiledModuleCache so you can layer your caching
I've taken the three different layers of caching in ModuleCache and split them into their own types - ThreadLocalCache, SharedCache, and OnDiskCache
There is a wasmer_wasix::runtime::module_cache::default_cache() constructor which chains the thread-local, Arc<RwLock<HashMap>>, and on-disk caches together
The ModuleCache is now just a wrapper around an instance of that default cache

CC: #3603 and #3831.

…odeJS

lib/wasi/src/runtime/module_cache/types.rs

lib/wasi/src/runtime/module_cache/shared.rs

lib/wasi/src/runtime/module_cache/mod.rs

lib/wasi/src/runtime/module_cache/types.rs

…ls into account

lib/wasi/src/runtime/module_cache/disabled.rs

lib/wasi/src/runtime/mod.rs

lib/wasi/src/runtime/module_cache/mod.rs

lib/wasi/src/runtime/module_cache/on_disk.rs

lib/wasi/src/runtime/module_cache/types.rs

lib/wasi/src/bin_factory/exec.rs

lib/wasi/src/runtime/module_cache/types.rs

lib/wasi/src/runtime/mod.rs

lib/wasi/src/runtime/module_cache/filesystem.rs

lib/wasi/src/runtime/module_cache/types.rs

…renamed it appropriately

…n cache by deterministic ID

lib/wasi/src/runtime/module_cache/types.rs

lib/wasi/src/runtime/module_cache/filesystem.rs

lib/wasi/src/runtime/mod.rs

lib/cli/src/commands/run_unstable.rs

Michael-F-Bryan changed the base branch from master to resolver-abstraction May 3, 2023 07:29

Michael-F-Bryan force-pushed the resolver-abstraction branch from fbb72ab to d2862f2 Compare May 3, 2023 14:28

Michael-F-Bryan force-pushed the module-cache branch from fae66f0 to d7a7080 Compare May 3, 2023 14:29

Michael-F-Bryan force-pushed the resolver-abstraction branch from d2862f2 to 2f83b25 Compare May 4, 2023 08:35

Base automatically changed from resolver-abstraction to master May 4, 2023 10:36

Michael-F-Bryan added 4 commits May 4, 2023 19:05

Created a set of tiered module caches

2062125

Switched ModuleCache over to the new CompiledModuleCache infrastructure

f4beab1

Made ModuleCache sync again

a8864aa

Make spawn_exec() asynchronous

3ed1192

Michael-F-Bryan force-pushed the module-cache branch from d7a7080 to 3ed1192 Compare May 4, 2023 12:10

Removed the task manager from CompiledModuleCache so tests build on N…

09d5493

…odeJS

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Outdated Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/shared.rs Outdated Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/shared.rs Outdated Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/shared.rs Outdated Show resolved Hide resolved

Switch everything from ModuleCache to the CompiledModuleCache trait

86ee27c

Michael-F-Bryan marked this pull request as ready for review May 5, 2023 04:03

theduke reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/mod.rs Outdated Show resolved Hide resolved

lib/wasi/src/runtime/module_cache/types.rs Outdated Show resolved Hide resolved

Michael-F-Bryan added 3 commits May 5, 2023 12:36

Switch SharedCache over to DashMap

fb059e0

Rename default_cache() to in_memory() and take browser-specific detai…

0667415

…ls into account

Switch cache keys from a String to a bespoke Key type

5af1b6a

Michael-F-Bryan enabled auto-merge May 5, 2023 05:57

Fixed a bunch of broken doc links

f25b3f1

theduke requested changes May 5, 2023

View reviewed changes

Michael-F-Bryan commented May 5, 2023

View reviewed changes

lib/wasi/src/runtime/mod.rs Outdated Show resolved Hide resolved

Rewrote the FileSystem cache to take @theduke's comments into account

87f684a

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/filesystem.rs Outdated Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Show resolved Hide resolved

Deleted the Disabled cache and made a ModuleCache compulsory

e633eb3

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Outdated Show resolved Hide resolved

syrusakbary reviewed May 5, 2023

View reviewed changes

lib/wasi/src/runtime/module_cache/types.rs Outdated Show resolved Hide resolved

Michael-F-Bryan mentioned this pull request May 8, 2023

WASIX's file-based module cache will block the runtime thread #3851

Closed

Michael-F-Bryan added 2 commits May 8, 2023 14:16

Changed the cache Key's semantics so it is *just* a module hash, and …

8828b04

…renamed it appropriately

Add an &Engine parameter to ModuleCache::save() so implementations ca…

e4796bd

…n cache by deterministic ID

Michael-F-Bryan disabled auto-merge May 8, 2023 07:33

theduke requested changes May 8, 2023

View reviewed changes

Michael-F-Bryan added 4 commits May 8, 2023 16:15

Adding docs and more tests

658d2e3

Use the tempfile crate for the filesystem cache's temporary file

efc811f

Removed the debug_assert!() from ModuleHash::sha256()

0a7a909

Make sure "wasmer run" uses the on-disk cache

5bc4237

Michael-F-Bryan enabled auto-merge May 8, 2023 08:55

theduke approved these changes May 8, 2023

View reviewed changes

Michael-F-Bryan merged commit b4f431f into master May 8, 2023

Michael-F-Bryan deleted the module-cache branch May 8, 2023 09:54

Michael-F-Bryan mentioned this pull request May 9, 2023

Re-work package resolution and make runners use it #3852

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce a module cache abstraction #3841

Introduce a module cache abstraction #3841

Michael-F-Bryan commented May 3, 2023 •

edited

Loading

Introduce a module cache abstraction #3841

Introduce a module cache abstraction #3841

Conversation

Michael-F-Bryan commented May 3, 2023 • edited Loading

Michael-F-Bryan commented May 3, 2023 •

edited

Loading