Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Base support for WASI in wasmer-js #2491

Merged
merged 90 commits into from
Aug 31, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
4200f2e
Base support for WASI in wasmer-js
syrusakbary Jul 24, 2021
65dacb5
Removed unnecessary comments
syrusakbary Jul 24, 2021
2e9cc52
Fix format
syrusakbary Jul 24, 2021
58bd122
Commented unused code
syrusakbary Jul 24, 2021
576312e
Get serializer back into working
syrusakbary Jul 24, 2021
fb616cf
Improved default fs backing
syrusakbary Jul 24, 2021
e79b8a0
Make WASI serialization optional
syrusakbary Jul 24, 2021
b2b576f
Removed all warnings
syrusakbary Jul 24, 2021
f15f9de
Fixed most TODOs
syrusakbary Jul 24, 2021
c810940
Remove WASI rename_file default implementation
syrusakbary Jul 24, 2021
1855700
Fix compilation
syrusakbary Jul 24, 2021
876eefe
Improved logging for tests
syrusakbary Jul 27, 2021
75340e1
Moved rename_file to the filesystem. Fix fs readdir
syrusakbary Jul 27, 2021
616d199
Fix compilation/linting issues
syrusakbary Jul 27, 2021
cc6013c
Ignore some tests for now
syrusakbary Jul 27, 2021
55ff142
Fix compilation in windows
syrusakbary Jul 27, 2021
d371fd2
Fixed tests
syrusakbary Jul 27, 2021
5120b71
Disabled mem_fs by default
syrusakbary Jul 27, 2021
fe49a91
Fixed wasmer-js
syrusakbary Jul 27, 2021
03e3af4
Fixed build wasmer in debug mode
syrusakbary Jul 27, 2021
fa1c634
Merge branch 'master' into js-api-wasi-base
syrusakbary Jul 30, 2021
926df5c
Added into js::Object for the ImportObject
syrusakbary Aug 10, 2021
67e24c3
Update lib/vfs/src/host_fs.rs
syrusakbary Aug 13, 2021
ea26266
fix(vfs) Remove the commented `vfs` virtual FS.
Hywan Aug 16, 2021
f639dd9
fix(vfs) Make `host_fs` the default file system.
Hywan Aug 16, 2021
52be548
feat(vfs) Make `host_fs` and `mem_fs` mutually exclusives.
Hywan Aug 16, 2021
5b53c54
fix(c-api,cli) Use the default features from `wasmer-wasi`.
Hywan Aug 16, 2021
6ff50be
feat: Replace `fs_metadata_to_metadata` by a `TryInto` implementation.
Hywan Aug 16, 2021
fde80a8
chore(vfs) Use `fs` rather than `std::fs`.
Hywan Aug 16, 2021
56b007a
Merge pull request #2526 from Hywan/chore-vfs-use-std-fs-alias
syrusakbary Aug 16, 2021
29af515
Merge pull request #2525 from Hywan/feat-vfs-host-tryfrom-metadata
syrusakbary Aug 16, 2021
9827358
Merge pull request #2522 from Hywan/fix-vfs-remove-vfs
syrusakbary Aug 16, 2021
b4a7c35
Update lib/wasi/Cargo.toml
syrusakbary Aug 16, 2021
0a1ad75
Merge branch 'js-api-wasi' into fix-vfs-make-features-exclusive
Hywan Aug 17, 2021
0172463
feat(vfs) `host_fs` and `mem_fs` are not mutually exclusive.
Hywan Aug 17, 2021
2240fa9
Merge pull request #2523 from Hywan/fix-vfs-make-features-exclusive
Hywan Aug 17, 2021
f211130
feat(vfs) Unified API.
Hywan Aug 16, 2021
5583631
Merge branch 'js-api-wasi' into feat-vfs-unified-fd
Hywan Aug 17, 2021
ab83efe
Fix merge conflict.
Hywan Aug 17, 2021
9e6c07d
Fix merge conflict.
Hywan Aug 17, 2021
ff9c7f1
fix(vfs) Change `FileDescriptor` to be unit.
Hywan Aug 17, 2021
cbc20c1
Merge pull request #2528 from Hywan/feat-vfs-unified-fd
Hywan Aug 17, 2021
d0fe468
fix(wasi) Re-organize the Cargo features.
Hywan Aug 17, 2021
2d8996a
fix(wasi) Update according to the changes made earlier.
Hywan Aug 17, 2021
dd80b39
Merge pull request #2529 from Hywan/fix-vfs-wasi-default-features
Hywan Aug 17, 2021
20d20ed
fix(vfs) Do not unwrap in `create_dir` when the path has a parent not…
Hywan Aug 17, 2021
3017a37
test(vfs) Add tests for `FileSystem::craete_dir`.
Hywan Aug 17, 2021
56d2cf0
fix(vfs) Do not unwrap in `remove_dir`.
Hywan Aug 17, 2021
743e302
test(vfs) Add test cases for `mem_fs::FileSystem::remove_dir`.
Hywan Aug 19, 2021
e617937
feat(vfs) Rewrite the in-memory filesystem.
Hywan Aug 19, 2021
e011c3b
Merge pull request #3 from Hywan/feat-vfs-mem-reboot
Hywan Aug 24, 2021
9e6a427
feat(vfs) Continue the rewrite of the in-memory filesystem.
Hywan Aug 26, 2021
b66e591
Merge branch 'test-vfs-mem' of github.com:Hywan/wasmer into test-vfs-mem
Hywan Aug 26, 2021
047256f
feat(vfs) Continue the rewrite of the in-memory filesystem.
Hywan Aug 26, 2021
3d69ec7
chore(carg) Updating `crossbeam-deque.
Hywan Aug 27, 2021
45e0340
chore(cargo) Solve RUSTSEC-2021-0080 by updating `tar`.
Hywan Aug 27, 2021
31f5c1a
fix(vfs) Fix a type error on Windows.
Hywan Aug 27, 2021
bea626b
fix(vfs) Add the `no-time` feature if `std::time` is not available.
Hywan Aug 27, 2021
2df47f6
fix(vfs) Correct support of Windows for `FileDescriptor`.
Hywan Aug 27, 2021
d8c4cc4
feat(wasi) Clarify an error message.
Hywan Aug 27, 2021
bfbdf01
!debug
Hywan Aug 27, 2021
0613f87
fix(vfs) Metadata returns 0 for accessed & co times on error.
Hywan Aug 30, 2021
9741c28
test: Update the Github Actions workflow files.
Hywan Aug 30, 2021
7cbff0f
chore(makefile) Print `rustc` and `node` versions.
Hywan Aug 30, 2021
2a246cf
!undebug
Hywan Aug 30, 2021
ac95425
test(vfs) Turn on `host-fs` _and_ `mem-fs` on by default.
Hywan Aug 30, 2021
fbe6a04
Merge pull request #2530 from Hywan/test-vfs-mem
Hywan Aug 30, 2021
71e0da5
Update Makefile
Hywan Aug 30, 2021
19f5846
fix(cli) Only enable `host-fs`, exclude `mem-fs`.
Hywan Aug 30, 2021
167d926
Merge branch 'master' into js-api-wasi
Hywan Aug 30, 2021
4f59083
fix(vfs) Remove a Clippy error.
Hywan Aug 30, 2021
9d712fd
Fixed Github workflow names
syrusakbary Aug 30, 2021
2cd2b9d
Merge branch 'js-api-wasi' of github.com:wasmerio/wasmer into js-api-…
syrusakbary Aug 30, 2021
dac47a8
fix(vfs) Fix `inode_of` on Windows.
Hywan Aug 30, 2021
e0a18a3
fix(vfs) Better handle path canonicalization on Windows.
Hywan Aug 30, 2021
fd4d341
test(wasi) Install Node v16 for running the tests.
Hywan Aug 30, 2021
18a642d
chore(github) Format workflow.
Hywan Aug 30, 2021
d0a95ac
fix(vfs) Fix 4f5908392.
Hywan Aug 30, 2021
bb47a47
chore(vfs) Remove Clippy warnings.
Hywan Aug 30, 2021
347f4c6
fix(api) Remove the temporary `Memory::uint8` method.
Hywan Aug 30, 2021
5ae6b98
fix(cli) Simplify a pattern matching.
Hywan Aug 30, 2021
eb19c98
Revert "fix(api) Remove the temporary `Memory::uint8` method."
Hywan Aug 30, 2021
2ff9838
doc(api) Document that `Memory::uint8view` is going to be refactored.
Hywan Aug 30, 2021
a4525e0
chore(types) Remove commented code.
Hywan Aug 30, 2021
768eb5c
feat(wasi) Update `WasiStateCreationgError::FsError` to `…::FileSyste…
Hywan Aug 30, 2021
650352d
feat(wasi) Clean commented code.
Hywan Aug 30, 2021
87558a9
feat(vfs) `mem-fs` does not support `enable-serde`.
Hywan Aug 30, 2021
4eaa864
doc(wasi) Document that `WasiStateBuilder::build` is changing interna…
Hywan Aug 31, 2021
a12ec28
feat(wasi) Provide `WasiState::(un)freeze` if `enable-serde` is enabled.
Hywan Aug 31, 2021
c28499e
chore(cargo) Update `Cargo.lock`.
Hywan Aug 31, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/documentation.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
name: Documentation

on:
push:
branches:
- 'master'

name: Documentation

jobs:
documentation:
name: Documentation
Expand Down
26 changes: 18 additions & 8 deletions .github/workflows/js.yaml → .github/workflows/test-js.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
name: Runtime tests - JS

env:
RUST_BACKTRACE: 1

on:
push:
branches:
Expand All @@ -8,24 +13,29 @@ on:
# this is _not_ a regex, see: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
- '[0-9]+.[0-9]+.[0-9]+*'

name: wasmer-js tests

env:
RUST_BACKTRACE: 1

jobs:
wasmer-js:
name: wasmer-js
test:
name: Test on NodeJS

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Install Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
override: true

- name: Install NodeJS
uses: actions/setup-node@v2
with:
node-version: 16

- name: Install wasm-pack
run: |
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
- name: Test wasmer-js

- name: Compile Wasmer to WebAssembly and test with a JavaScript host
run: make test-js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: core tests
name: Runtime tests

env:
RUST_BACKTRACE: 1
Expand Down
101 changes: 98 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 7 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ members = [
"lib/engine-dylib",
"lib/engine-staticlib",
"lib/object",
"lib/vfs",
"lib/vm",
"lib/wasi",
"lib/wasi-types",
Expand All @@ -55,6 +56,7 @@ members = [
"tests/integration/ios",
"fuzz",
]
resolver = "2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


[build-dependencies]
test-generator = { path = "tests/lib/test-generator" }
Expand All @@ -72,6 +74,10 @@ wasmer-engine-dummy = { path = "tests/lib/engine-dummy" }
compiler-test-derive = { path = "tests/lib/compiler-test-derive" }
tempfile = "3.1"
loupe = "0.1"
# For logging tests using the `RUST_LOG=debug` when testing
test-env-log = { version = "0.2", default-features = false, features = ["trace"] }
tracing = { version = "0.1", default-features = false, features = ["log"] }
tracing-subscriber = { version = "0.2", default-features = false, features = ["env-filter", "fmt"] }

[features]
# Don't add the compiler features in default, please add them on the Makefile
Expand All @@ -80,7 +86,6 @@ default = [
"wat",
"wast",
"universal",
"dylib",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing dylib here (and keeping staticlib for example)?

"staticlib",
"cache",
"wasi",
Expand Down Expand Up @@ -285,4 +290,4 @@ required-features = ["cranelift"]
[[example]]
name = "features"
path = "examples/features.rs"
required-features = ["cranelift"]
required-features = ["cranelift"]
16 changes: 11 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -318,10 +318,12 @@ ifneq (, $(LIBC))
endif
$(info Enabled Compilers: $(bold)$(green)$(subst $(space),$(reset)$(comma)$(space)$(bold)$(green),$(compilers))$(reset).)
$(info Testing the following compilers & engines:)
$(info * API: $(bold)$(green)${compilers_engines}$(reset))
$(info * C-API: $(bold)$(green)${capi_compilers_engines}$(reset))
$(info   * API: $(bold)$(green)${compilers_engines}$(reset),)
$(info   * C-API: $(bold)$(green)${capi_compilers_engines}$(reset).)
$(info Cargo features:)
$(info * Compilers: `$(bold)$(green)${compiler_features}$(reset)`.)
$(info   * Compilers: `$(bold)$(green)${compiler_features}$(reset)`.)
$(info Rust version: $(bold)$(green)$(shell rustc --version)$(reset).)
$(info NodeJS version: $(bold)$(green)$(shell node --version)$(reset).)
$(info )
$(info )
$(info --------------)
Expand Down Expand Up @@ -358,7 +360,7 @@ build-wasmer:
cargo build --release --manifest-path lib/cli/Cargo.toml $(compiler_features) --bin wasmer

build-wasmer-debug:
cargo build --manifest-path lib/cli/Cargo.toml $(compiler_features) --bin wasmer
cargo build --manifest-path lib/cli/Cargo.toml $(compiler_features) --features "debug" --bin wasmer

bench:
cargo bench $(compiler_features)
Expand Down Expand Up @@ -505,9 +507,13 @@ test-packages:
cargo test --manifest-path lib/compiler-singlepass/Cargo.toml --release --no-default-features --features=std
cargo test --manifest-path lib/cli/Cargo.toml $(compiler_features) --release

test-js:
test-js: test-js-api test-js-wasi

test-js-api:
cd lib/api && wasm-pack test --node -- --no-default-features --features js-default,wat

test-js-wasi:
cd lib/wasi && wasm-pack test --node -- --no-default-features --features test-js

#####
#
Expand Down
5 changes: 4 additions & 1 deletion lib/api/src/js/externals/memory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,10 @@ impl Memory {
unimplemented!("The view function is not yet implemented in Wasmer Javascript");
}

/// example view
/// A theoretical alais to `Self::view::<u8>` but it returns a `js::Uint8Array` in this case.
///
/// This code is going to be refactored. Use it as your own risks.
#[doc(hidden)]
pub fn uint8view(&self) -> js_sys::Uint8Array {
js_sys::Uint8Array::new(&self.vm_memory.memory.buffer())
}
Expand Down
17 changes: 17 additions & 0 deletions lib/api/src/js/import_object.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,23 @@ impl ImportObject {
}
}

impl Into<js_sys::Object> for ImportObject {
fn into(self) -> js_sys::Object {
let guard = self.map.lock().unwrap();
let map = guard.borrow();

let imports = js_sys::Object::new();
for (module, ns) in map.iter() {
let import_namespace = js_sys::Object::new();
for (name, exp) in ns.get_namespace_exports() {
js_sys::Reflect::set(&import_namespace, &name.into(), exp.as_jsvalue()).unwrap();
}
js_sys::Reflect::set(&imports, &module.into(), &import_namespace.into()).unwrap();
}
imports
}
}

impl NamedResolver for ImportObject {
fn resolve_by_name(&self, module: &str, name: &str) -> Option<Export> {
self.get_export(module, name)
Expand Down
Loading