Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove backend::Backend from runtime-core #1099

Merged
merged 31 commits into from
Jan 13, 2020
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
303d44c
Remove backend code
syrusakbary Dec 21, 2019
d4e9645
Move requires pre validation into the ModuleCodeGenerator
syrusakbary Dec 21, 2019
d7154fe
Remove Backend dependency
syrusakbary Dec 21, 2019
18421e3
Make all tests pass
syrusakbary Dec 21, 2019
f3b9ecb
Remove backend specific features from root Cargo
syrusakbary Dec 21, 2019
71be5be
Improved syntax
syrusakbary Dec 21, 2019
0cb3df2
Removed unnecessary checks
syrusakbary Dec 21, 2019
5d2ea93
Fixed changelog link
syrusakbary Dec 21, 2019
ac0c5c9
Fixed lint
syrusakbary Dec 21, 2019
e57677b
Make cranelift optional for middleware
syrusakbary Dec 21, 2019
d36d883
Fix lint
syrusakbary Dec 21, 2019
8cff1ad
Fix wasmer binary
syrusakbary Dec 21, 2019
720d4ec
Fix checks on binary
syrusakbary Dec 21, 2019
f967770
Fixed linting
syrusakbary Dec 21, 2019
63f1d6b
Fixed path variable name
syrusakbary Jan 7, 2020
3b9cdc3
Recovered deleted tests
syrusakbary Jan 7, 2020
59945a8
Use detault
syrusakbary Jan 7, 2020
ee6949b
Add checks back into makefile
syrusakbary Jan 7, 2020
d544f5a
Run formatting
syrusakbary Jan 7, 2020
f353ac0
Remove unused backend flags
syrusakbary Jan 10, 2020
9ca9770
Merge branch 'master' into backend-refactor
syrusakbary Jan 10, 2020
cc28804
Update the name of experimental IO devices in fs
Jan 11, 2020
5b52589
Update the experimental io device path to include /dev/
Jan 11, 2020
8fd1433
Bump getrandom from 0.1.13 to 0.1.14
dependabot-preview[bot] Jan 13, 2020
00618e8
Bump parking_lot from 0.9.0 to 0.10.0
dependabot-preview[bot] Jan 13, 2020
f18a428
Bump regex from 1.3.1 to 1.3.3
dependabot-preview[bot] Jan 13, 2020
22e35fc
Bump cc from 1.0.48 to 1.0.50
dependabot-preview[bot] Jan 13, 2020
aa6189a
Make all tests pass
syrusakbary Dec 21, 2019
4faa78b
Fixed changelog link
syrusakbary Dec 21, 2019
846c632
Fixed runtime check
syrusakbary Jan 10, 2020
3892ea8
Fixed lint
syrusakbary Jan 13, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## **[Unreleased]**

- [#1099](https://github.com/wasmerio/wasmer/pull/1099) Remove `backend::Backend` from `wasmer_runtime_core`
- [#1097](https://github.com/wasmerio/wasmer/pull/1097) Move inline breakpoint outside of runtime backend
- [#1095](https://github.com/wasmerio/wasmer/pull/1095) Update to cranelift 0.52.
- [#1092](https://github.com/wasmerio/wasmer/pull/1092) Add `get_utf8_string_with_nul` to `WasmPtr` to read nul-terminated strings from memory.
Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 0 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -86,19 +86,16 @@ extra-debug = ["wasmer-clif-backend/debug", "wasmer-runtime-core/debug"]
fast-tests = []
backend-cranelift = [
"wasmer-clif-backend",
"wasmer-runtime-core/backend-cranelift",
"wasmer-runtime/cranelift",
"wasmer-middleware-common-tests/clif",
]
backend-llvm = [
"wasmer-llvm-backend",
"wasmer-runtime-core/backend-llvm",
"wasmer-runtime/llvm",
"wasmer-middleware-common-tests/llvm",
]
backend-singlepass = [
"wasmer-singlepass-backend",
"wasmer-runtime-core/backend-singlepass",
"wasmer-runtime/singlepass",
"wasmer-middleware-common-tests/singlepass",
]
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ check: check-bench
--features=llvm,default-backend-llvm,debug
$(RUNTIME_CHECK) --release \
--features=llvm,default-backend-llvm
--features=default-backend-singlepass,singlepass,cranelift,llvm,cache,debug,deterministic-execution

# Release
release:
Expand Down
6 changes: 3 additions & 3 deletions lib/clif-backend/src/code.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use std::mem;
use std::sync::{Arc, RwLock};
use wasmer_runtime_core::error::CompileError;
use wasmer_runtime_core::{
backend::{Backend, CacheGen, Token},
backend::{CacheGen, Token},
cache::{Artifact, Error as CacheError},
codegen::*,
memory::MemoryType,
Expand Down Expand Up @@ -58,8 +58,8 @@ impl ModuleCodeGenerator<CraneliftFunctionCodeGenerator, Caller, CodegenError>
unimplemented!("cross compilation is not available for clif backend")
}

fn backend_id() -> Backend {
Backend::Cranelift
fn backend_id() -> String {
"cranelift".to_string()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to have typed backend ids. An approach similar to InternalField for middlewares might work:

pub struct InternalField {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do this with a wrapper struct and (static) strings too.

I think we'll probably want to have a way to print out the name of the backend in either case, so we'll either need to have the string be part of the id or have a mapping from id to string.

}

fn check_precondition(&mut self, _module_info: &ModuleInfo) -> Result<(), CodegenError> {
Expand Down
6 changes: 3 additions & 3 deletions lib/llvm-backend/src/code.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ use std::{
};

use wasmer_runtime_core::{
backend::{Backend, CacheGen, CompilerConfig, Token},
backend::{CacheGen, CompilerConfig, Token},
cache::{Artifact, Error as CacheError},
codegen::*,
memory::MemoryType,
Expand Down Expand Up @@ -8721,8 +8721,8 @@ impl<'ctx> ModuleCodeGenerator<LLVMFunctionCodeGenerator<'ctx>, LLVMBackend, Cod
}
}

fn backend_id() -> Backend {
Backend::LLVM
fn backend_id() -> String {
"llvm".to_string()
}

fn check_precondition(&mut self, _module_info: &ModuleInfo) -> Result<(), CodegenError> {
Expand Down
4 changes: 2 additions & 2 deletions lib/middleware-common-tests/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ publish = false
[dependencies]
wasmer-runtime-core = { path = "../runtime-core", version = "0.12.0" }
wasmer-middleware-common = { path = "../middleware-common", version = "0.12.0" }
wasmer-clif-backend = { path = "../clif-backend", version = "0.12.0" }
wasmer-clif-backend = { path = "../clif-backend", version = "0.12.0", optional = true }
wasmer-llvm-backend = { path = "../llvm-backend", version = "0.12.0", features = ["test"], optional = true }
wasmer-singlepass-backend = { path = "../singlepass-backend", version = "0.12.0", optional = true }

[features]
clif = []
clif = ["wasmer-clif-backend"]
llvm = ["wasmer-llvm-backend"]
singlepass = ["wasmer-singlepass-backend"]

Expand Down
51 changes: 21 additions & 30 deletions lib/middleware-common-tests/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,47 +3,33 @@ mod tests {
use wabt::wat2wasm;

use wasmer_middleware_common::metering::*;
use wasmer_runtime_core::backend::RunnableModule;
use wasmer_runtime_core::codegen::ModuleCodeGenerator;
use wasmer_runtime_core::codegen::{MiddlewareChain, StreamingCompiler};
use wasmer_runtime_core::fault::{pop_code_version, push_code_version};
use wasmer_runtime_core::state::CodeVersion;
use wasmer_runtime_core::{
backend::{Backend, Compiler},
compile_with, imports, Func,
};
use wasmer_runtime_core::{backend::Compiler, compile_with, imports, Func};

#[cfg(feature = "llvm")]
fn get_compiler(limit: u64) -> (impl Compiler, Backend) {
use wasmer_llvm_backend::ModuleCodeGenerator as LLVMMCG;
let c: StreamingCompiler<LLVMMCG, _, _, _, _> = StreamingCompiler::new(move || {
let mut chain = MiddlewareChain::new();
chain.push(Metering::new(limit));
chain
});
(c, Backend::LLVM)
}
use wasmer_llvm_backend::ModuleCodeGenerator as MCG;

#[cfg(feature = "singlepass")]
fn get_compiler(limit: u64) -> (impl Compiler, Backend) {
use wasmer_singlepass_backend::ModuleCodeGenerator as SinglePassMCG;
let c: StreamingCompiler<SinglePassMCG, _, _, _, _> = StreamingCompiler::new(move || {
use wasmer_singlepass_backend::ModuleCodeGenerator as MCG;

#[cfg(feature = "clif")]
compile_error!("cranelift does not implement metering yet");

fn get_compiler(limit: u64) -> impl Compiler {
let c: StreamingCompiler<MCG, _, _, _, _> = StreamingCompiler::new(move || {
let mut chain = MiddlewareChain::new();
chain.push(Metering::new(limit));
chain
});
(c, Backend::Singlepass)
c
}

#[cfg(not(any(feature = "llvm", feature = "clif", feature = "singlepass")))]
compile_error!("compiler not specified, activate a compiler via features");

#[cfg(feature = "clif")]
fn get_compiler(_limit: u64) -> (impl Compiler, Backend) {
compile_error!("cranelift does not implement metering");
use wasmer_clif_backend::CraneliftCompiler;
(CraneliftCompiler::new(), Backend::Cranelift)
}

// Assemblyscript
// export function add_to(x: i32, y: i32): i32 {
// for(var i = 0; i < x; i++){
Expand Down Expand Up @@ -109,7 +95,7 @@ mod tests {

let limit = 100u64;

let (compiler, backend_id) = get_compiler(limit);
let compiler = get_compiler(limit);
let module = compile_with(&wasm_binary, &compiler).unwrap();

let import_object = imports! {};
Expand All @@ -124,8 +110,8 @@ mod tests {
baseline: true,
msm: msm,
base: instance.module.runnable_module.get_code().unwrap().as_ptr() as usize,
backend: MCG::backend_id(),
runnable_module: instance.module.runnable_module.clone(),
backend: backend_id,
});
true
} else {
Expand All @@ -151,7 +137,7 @@ mod tests {

let limit = 100u64;

let (compiler, backend_id) = get_compiler(limit);
let compiler = get_compiler(limit);
let module = compile_with(&wasm_binary, &compiler).unwrap();

let import_object = imports! {};
Expand All @@ -161,12 +147,17 @@ mod tests {

let add_to: Func<(i32, i32), i32> = instance.func("add_to").unwrap();

let cv_pushed = if let Some(msm) = instance.module.runnable_module.get_module_state_map() {
let cv_pushed = if let Some(msm) = instance
.module
.runnable_module
.borrow()
.get_module_state_map()
{
push_code_version(CodeVersion {
baseline: true,
msm: msm,
base: instance.module.runnable_module.get_code().unwrap().as_ptr() as usize,
backend: backend_id,
backend: MCG::backend_id(),
runnable_module: instance.module.runnable_module.clone(),
});
true
Expand Down
2 changes: 1 addition & 1 deletion lib/runtime-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ cc = "1.0"
[features]
debug = []
trace = ["debug"]
# backend flags used in conditional compilation of Backend::variants
# backend flags no longer used
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer just deleting them if we're not using them!

"backend-cranelift" = []
"backend-singlepass" = []
"backend-llvm" = []
Expand Down
70 changes: 0 additions & 70 deletions lib/runtime-core/src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,60 +22,6 @@ pub mod sys {
}
pub use crate::sig_registry::SigRegistry;

/// Enum used to select which compiler should be used to generate code.
#[derive(Serialize, Deserialize, Debug, Copy, Clone, PartialEq, Eq)]
pub enum Backend {
Cranelift,
Singlepass,
LLVM,
Auto,
}

impl Backend {
/// Get a list of the currently enabled (via feature flag) backends.
pub fn variants() -> &'static [&'static str] {
&[
#[cfg(feature = "backend-cranelift")]
"cranelift",
#[cfg(feature = "backend-singlepass")]
"singlepass",
#[cfg(feature = "backend-llvm")]
"llvm",
"auto",
]
}

/// Stable string representation of the backend.
/// It can be used as part of a cache key, for example.
pub fn to_string(&self) -> &'static str {
match self {
Backend::Cranelift => "cranelift",
Backend::Singlepass => "singlepass",
Backend::LLVM => "llvm",
Backend::Auto => "auto",
}
}
}

impl Default for Backend {
fn default() -> Self {
Backend::Cranelift
}
}

impl std::str::FromStr for Backend {
type Err = String;
fn from_str(s: &str) -> Result<Backend, String> {
match s.to_lowercase().as_str() {
"singlepass" => Ok(Backend::Singlepass),
"cranelift" => Ok(Backend::Cranelift),
"llvm" => Ok(Backend::LLVM),
"auto" => Ok(Backend::Auto),
_ => Err(format!("The backend {} doesn't exist", s)),
}
}
}

/// The target architecture for code generation.
#[derive(Copy, Clone, Debug)]
pub enum Architecture {
Expand Down Expand Up @@ -104,22 +50,6 @@ pub struct InlineBreakpoint {
pub ty: InlineBreakpointType,
}

#[cfg(test)]
mod backend_test {
use super::*;
use std::str::FromStr;

#[test]
fn str_repr_matches() {
// if this test breaks, think hard about why it's breaking
// can we avoid having these be different?

for &backend in &[Backend::Cranelift, Backend::LLVM, Backend::Singlepass] {
assert_eq!(backend, Backend::from_str(backend.to_string()).unwrap());
}
}
}

syrusakbary marked this conversation as resolved.
Show resolved Hide resolved
/// This type cannot be constructed from
/// outside the runtime crate.
pub struct Token {
Expand Down
28 changes: 3 additions & 25 deletions lib/runtime-core/src/cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,9 @@
//! serializing compiled wasm code to a binary format. The binary format can be persisted,
//! and loaded to allow skipping compilation and fast startup.

use crate::{
backend::Backend,
module::{Module, ModuleInfo},
sys::Memory,
};
use crate::{module::ModuleInfo, sys::Memory};
use blake2b_simd::blake2bp;
use std::{fmt, io, mem, slice};
use std::{io, mem, slice};

/// Indicates the invalid type of invalid cache file
#[derive(Debug)]
Expand All @@ -35,7 +31,7 @@ pub enum Error {
/// The cached binary has been invalidated.
InvalidatedCache,
/// The current backend does not support caching.
UnsupportedBackend(Backend),
UnsupportedBackend(String),
}

impl From<io::Error> for Error {
Expand Down Expand Up @@ -246,24 +242,6 @@ impl Artifact {
}
}

/// A generic cache for storing and loading compiled wasm modules.
///
/// The `wasmer-runtime` supplies a naive `FileSystemCache` api.
pub trait Cache {
/// Error type to return when load error occurs
type LoadError: fmt::Debug;
/// Error type to return when store error occurs
type StoreError: fmt::Debug;

/// loads a module using the default `Backend`
fn load(&self, key: WasmHash) -> Result<Module, Self::LoadError>;
/// loads a cached module using a specific `Backend`
fn load_with_backend(&self, key: WasmHash, backend: Backend)
-> Result<Module, Self::LoadError>;
/// Store a module into the cache with the given key
fn store(&mut self, key: WasmHash, module: Module) -> Result<(), Self::StoreError>;
}

/// A unique ID generated from the version of Wasmer for use with cache versioning
pub const WASMER_VERSION_HASH: &'static str =
include_str!(concat!(env!("OUT_DIR"), "/wasmer_version_hash.txt"));
Loading