Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add builder API for WasiState #850

Merged
merged 6 commits into from
Oct 2, 2019
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Blocks of changes will separated by version increments.

## **[Unreleased]**

- [#850](https://github.com/wasmerio/wasmer/pull/850) New `WasiStateBuilder` API. small, add misc. breaking changes to existing API (for example, changing the preopen dirs arg on `wasi::generate_import_object` from `Vec<String>` to `Vec<Pathbuf>`)
- [#852](https://github.com/wasmerio/wasmer/pull/852) Make minor grammar/capitalization fixes to README.md
- [#841](https://github.com/wasmerio/wasmer/pull/841) Slightly improve rustdoc documentation and small updates to outdated info in readme files
- [#835](https://github.com/wasmerio/wasmer/pull/836) Update Cranelift fork version to `0.44.0`
Expand Down
5 changes: 3 additions & 2 deletions lib/wasi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@ pub struct ExitCode {
pub code: syscalls::types::__wasi_exitcode_t,
}

/// Creates a WasiImport object with `WasiState`.
/// Creates a Wasi [`ImportObject`] with [`WasiState`].
pub fn generate_import_object(
args: Vec<Vec<u8>>,
envs: Vec<Vec<u8>>,
preopened_files: Vec<String>,
preopened_files: Vec<PathBuf>,
mapped_dirs: Vec<(String, PathBuf)>,
) -> ImportObject {
let state_gen = move || {
Expand All @@ -63,6 +63,7 @@ pub fn generate_import_object(
}
let preopened_files = preopened_files.clone();
let mapped_dirs = mapped_dirs.clone();
//let wasi_builder = create_wasi_instance();

let state = Box::new(WasiState {
fs: WasiFs::new(&preopened_files, &mapped_dirs).expect("Could not create WASI FS"),
Expand Down
293 changes: 293 additions & 0 deletions lib/wasi/src/state/builder.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,293 @@
//! Builder code for [`WasiState`]

use crate::state::{WasiFs, WasiState};
use std::path::{Path, PathBuf};

/// Creates an empty [`WasiStateBuilder`].
pub fn create_wasi_state() -> WasiStateBuilder {
WasiStateBuilder::default()
}

/// Type for building an instance of [`WasiState`]
///
/// Usage:
///
/// ```
/// # use wasmer_wasi::state::create_wasi_state;
/// create_wasi_state()
/// .env(b"HOME", "/home/home".to_string())
/// .arg("--help")
/// .envs({ let mut hm = std::collections::HashMap::new();
/// hm.insert("COLOR_OUTPUT", "TRUE");
/// hm.insert("PATH", "/usr/bin");
/// hm
/// })
/// .args(&["--verbose", "list"])
/// .preopen_dir("src")
/// .map_dir("dot", ".")
/// .build()
/// .unwrap();
/// ```
Copy link
Member

@syrusakbary syrusakbary Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that perhaps this API might be simpler? (removing crate_wasi_state and renaming WasiStateBuilder to WASICommand )

use wasmer_wasi::WASICommand;

WASICommand::new("")
   .env(b"HOME", "/home/home".to_string())
   .arg("--help")
   .envs({ let mut hm = std::collections::HashMap::new();
           hm.insert("COLOR_OUTPUT", "TRUE");
           hm.insert("PATH", "/usr/bin");
           hm
         })
   .args(&["--verbose", "list"])
   .preopen_dir("src")
   .map_dir("dot", ".")
   .build()
   .unwrap();

#[derive(Debug, Default, Clone, PartialEq)]
pub struct WasiStateBuilder {
args: Vec<Vec<u8>>,
envs: Vec<Vec<u8>>,
preopened_files: Vec<PathBuf>,
mapped_dirs: Vec<(String, PathBuf)>,
}

/// Error type returned when bad data is given to [`WasiStateBuilder`].
#[derive(Debug, PartialEq, Eq)]
pub enum WasiStateCreationError {
EnvironmentVariableFormatError(String),
ArgumentContainsNulByte(String),
PreopenedDirectoryNotFound(PathBuf),
MappedDirAliasFormattingError(String),
WasiFsCreationError(String),
}

fn validate_mapped_dir_alias(alias: &str) -> Result<(), WasiStateCreationError> {
for byte in alias.bytes() {
match byte {
b'/' => {
return Err(WasiStateCreationError::MappedDirAliasFormattingError(
format!("Alias \"{}\" contains the character '/'", alias),
));
}
b'\0' => {
return Err(WasiStateCreationError::MappedDirAliasFormattingError(
format!("Alias \"{}\" contains a nul byte", alias),
));
}
_ => (),
}
}

Ok(())
}

impl WasiStateBuilder {
/// Add an environment variable pair.
/// Environment variable keys and values must not contain the byte `=` (0x3d)
/// or nul (0x0).
pub fn env<Key, Value>(&mut self, key: Key, value: Value) -> &mut Self
where
Key: AsRef<[u8]>,
Value: AsRef<[u8]>,
{
let key_b = key.as_ref();
let val_b = value.as_ref();

let length = key_b.len() + val_b.len() + 1;
let mut byte_vec = Vec::with_capacity(length);

byte_vec.extend_from_slice(&key_b);
byte_vec.push(b'=');
byte_vec.extend_from_slice(&val_b);

self.envs.push(byte_vec);

self
}

/// Add an argument.
/// Arguments must not contain the nul (0x0) byte
pub fn arg<Arg>(&mut self, arg: Arg) -> &mut Self
where
Arg: AsRef<[u8]>,
{
let arg_b = arg.as_ref();
let mut byte_vec = Vec::with_capacity(arg_b.len());
byte_vec.extend_from_slice(&arg_b);
self.args.push(byte_vec);

self
}

/// Add multiple environment variable pairs.
/// Keys and values must not contain the `=` (0x3d) or nul (0x0) byte.
pub fn envs<I, Key, Value>(&mut self, env_pairs: I) -> &mut Self
where
I: IntoIterator<Item = (Key, Value)>,
Key: AsRef<[u8]>,
Value: AsRef<[u8]>,
{
for (key, value) in env_pairs {
let key_b = key.as_ref();
let val_b = value.as_ref();

let length = key_b.len() + val_b.len() + 1;
let mut byte_vec = Vec::with_capacity(length);

byte_vec.extend_from_slice(&key_b);
byte_vec.push(b'=');
byte_vec.extend_from_slice(&val_b);

self.envs.push(byte_vec);
}

self
}

/// Add multiple arguments.
/// Arguments must not contain the nul (0x0) byte
pub fn args<I, Arg>(&mut self, args: I) -> &mut Self
where
I: IntoIterator<Item = Arg>,
Arg: AsRef<[u8]>,
{
for arg in args {
let arg_b = arg.as_ref();
let mut byte_vec = Vec::with_capacity(arg_b.len());
byte_vec.extend_from_slice(&arg_b);
self.args.push(byte_vec);
}

self
}

/// Preopen a directory
/// This opens the given directory at the virtual root, `/`, and allows
/// the WASI module to read and write to the given directory.
// TODO: design a simple API for passing in permissions here (i.e. read-only)
pub fn preopen_dir<FilePath>(&mut self, po_dir: FilePath) -> &mut Self
where
FilePath: AsRef<Path>,
{
let path = po_dir.as_ref();
self.preopened_files.push(path.to_path_buf());

self
}

/// Preopen a directory
/// This opens the given directory at the virtual root, `/`, and allows
/// the WASI module to read and write to the given directory.
pub fn preopen_dirs<I, FilePath>(&mut self, po_dirs: I) -> &mut Self
where
I: IntoIterator<Item = FilePath>,
FilePath: AsRef<Path>,
{
for po_dir in po_dirs {
let path = po_dir.as_ref();
self.preopened_files.push(path.to_path_buf());
}

self
}

/// Preopen a directory with a different name exposed to the WASI.
pub fn map_dir<FilePath>(&mut self, alias: &str, po_dir: FilePath) -> &mut Self
where
FilePath: AsRef<Path>,
{
let path = po_dir.as_ref();
self.mapped_dirs
.push((alias.to_string(), path.to_path_buf()));

self
}

/// Consumes the [`WasiStateBuilder`] and produces a [`WasiState`]
///
/// Returns the error from `WasiFs::new` if there's an error
pub fn build(&mut self) -> Result<WasiState, WasiStateCreationError> {
for arg in self.args.iter() {
for b in arg.iter() {
if *b == 0 {
return Err(WasiStateCreationError::ArgumentContainsNulByte(
std::str::from_utf8(arg)
.unwrap_or("Inner error: arg is invalid_utf8!")
.to_string(),
));
}
}
}
for env in self.envs.iter() {
let mut eq_seen = false;
for b in env.iter() {
match *b {
b'=' => {
if eq_seen {
return Err(WasiStateCreationError::EnvironmentVariableFormatError(
format!(
"found '=' in env var string \"{}\" (key=value)",
std::str::from_utf8(env)
.unwrap_or("Inner error: env var is invalid_utf8!")
),
));
}
eq_seen = true;
}
0 => {
return Err(WasiStateCreationError::EnvironmentVariableFormatError(
format!(
"found nul byte in env var string \"{}\" (key=value)",
std::str::from_utf8(env)
.unwrap_or("Inner error: env var is invalid_utf8!")
),
));
}
_ => (),
}
}
}

for po_f in self.preopened_files.iter() {
if !po_f.exists() {
return Err(WasiStateCreationError::PreopenedDirectoryNotFound(
po_f.clone(),
));
}
}

for (alias, po_f) in self.mapped_dirs.iter() {
if !po_f.exists() {
return Err(WasiStateCreationError::PreopenedDirectoryNotFound(
po_f.clone(),
));
}
validate_mapped_dir_alias(&alias)?;
}
Ok(WasiState {
fs: WasiFs::new(&self.preopened_files, &self.mapped_dirs)
.map_err(WasiStateCreationError::WasiFsCreationError)?,
args: self.args.clone(),
envs: self.envs.clone(),
})
}
}

#[cfg(test)]
mod test {
use super::*;

#[test]
fn env_var_errors() {
let output = create_wasi_state().env("HOM=E", "/home/home").build();
match output {
Err(WasiStateCreationError::EnvironmentVariableFormatError(_)) => assert!(true),
_ => assert!(false),
}

let output = create_wasi_state().env("HOME\0", "/home/home").build();
match output {
Err(WasiStateCreationError::EnvironmentVariableFormatError(_)) => assert!(true),
_ => assert!(false),
}
}

#[test]
fn nul_character_in_args() {
let output = create_wasi_state().arg("--h\0elp").build();
match output {
Err(WasiStateCreationError::ArgumentContainsNulByte(_)) => assert!(true),
_ => assert!(false),
}
let output = create_wasi_state().args(&["--help", "--wat\0"]).build();
match output {
Err(WasiStateCreationError::ArgumentContainsNulByte(_)) => assert!(true),
_ => assert!(false),
}
}
}
17 changes: 10 additions & 7 deletions lib/wasi/src/state/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@
//! You can implement `WasiFile` for your own types to get custom behavior and extend WASI, see the
//! [WASI plugin example](https://github.com/wasmerio/wasmer/blob/master/examples/plugin.rs).

mod builder;
mod types;

pub use self::builder::*;
pub use self::types::*;
use crate::syscalls::types::*;
use generational_arena::Arena;
Expand Down Expand Up @@ -140,7 +142,7 @@ pub struct WasiFs {

impl WasiFs {
pub fn new(
preopened_dirs: &[String],
preopened_dirs: &[PathBuf],
mapped_dirs: &[(String, PathBuf)],
) -> Result<Self, String> {
debug!("wasi::fs::inodes");
Expand Down Expand Up @@ -191,8 +193,7 @@ impl WasiFs {
debug!("Attempting to preopen {}", &dir);
// TODO: think about this
let default_rights = 0x1FFFFFFF; // all rights
let cur_dir = PathBuf::from(dir);
let cur_dir_metadata = cur_dir.metadata().map_err(|e| {
let cur_dir_metadata = dir.metadata().map_err(|e| {
format!(
"Could not get metadata for file {:?}: {}",
dir,
Expand All @@ -202,18 +203,18 @@ impl WasiFs {
let kind = if cur_dir_metadata.is_dir() {
Kind::Dir {
parent: Some(root_inode),
path: cur_dir.clone(),
path: dir.clone(),
entries: Default::default(),
}
} else {
return Err(format!(
"WASI only supports pre-opened directories right now; found \"{}\"",
&dir
&dir.to_string_lossy()
));
};
// TODO: handle nested pats in `file`
let inode = wasi_fs
.create_inode(kind, true, dir.to_string())
.create_inode(kind, true, dir.to_string_lossy().into_owned())
.map_err(|e| {
format!(
"Failed to create inode for preopened dir: WASI error code: {}",
Expand All @@ -231,7 +232,9 @@ impl WasiFs {
.map_err(|e| format!("Could not open fd for file {:?}: {}", dir, e))?;
if let Kind::Root { entries } = &mut wasi_fs.inodes[root_inode].kind {
// todo handle collisions
assert!(entries.insert(dir.to_string(), inode).is_none())
assert!(entries
.insert(dir.to_string_lossy().into_owned(), inode)
.is_none())
}
wasi_fs.preopen_fds.push(fd);
}
Expand Down
2 changes: 1 addition & 1 deletion src/bin/wasmer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ struct Run {

/// WASI pre-opened directory
#[structopt(long = "dir", multiple = true, group = "wasi")]
pre_opened_directories: Vec<String>,
pre_opened_directories: Vec<PathBuf>,

/// Map a host directory to a different location for the wasm module
#[structopt(long = "mapdir", multiple = true)]
Expand Down