Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update wasmer --version logic, integrate wapm-cli #3215

Merged
merged 94 commits into from
Oct 25, 2022
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
6cd80f4
Update clap, rewrite main CLI logic
fschutt Oct 4, 2022
771e1e3
Added `RunWithoutFile` to prepare downloading + GraphQL schema
fschutt Oct 4, 2022
d74419d
Don't panic when trying to run wasmer run [package]
fschutt Oct 4, 2022
ba8b13c
Add spinner when installing package
fschutt Oct 4, 2022
3b15e75
Add GraphQL querying
fschutt Oct 4, 2022
83c50d3
Add code for querying GraphQL
fschutt Oct 4, 2022
af01356
Implement wasmer -vV
fschutt Oct 4, 2022
6c57e1e
Finished adding wasmer -vV and --help messages
fschutt Oct 4, 2022
4bf7c43
Query packages correctly from wapm registry and debug error msg
fschutt Oct 5, 2022
2c1cf2e
Implemented "did you mean" functionality
fschutt Oct 5, 2022
374494e
Install packages correctly
fschutt Oct 6, 2022
efdb1f4
cargo fmt
fschutt Oct 6, 2022
8178673
Fail on ambigouus package name
fschutt Oct 6, 2022
0068d79
Do not re-download already-downloaded package
fschutt Oct 6, 2022
2414d44
Merge branch 'master' into wasmer-vv
fschutt Oct 6, 2022
2640a6d
Pass filesystem mappings from wapm.toml to --mapdir
fschutt Oct 6, 2022
5238c8e
Fix errors in debug mode, implement getenv / setenv on emscripten
fschutt Oct 6, 2022
3be8640
Added --env flags for emscripten target, debugged PYTHONHOME
fschutt Oct 6, 2022
da1e295
Fix error in listing local packages
fschutt Oct 7, 2022
50733a5
Try adding support for pkg_fs
fschutt Oct 7, 2022
a6d9e3e
Map every single directory on emscripten
fschutt Oct 7, 2022
7edfd68
cargo fmt
fschutt Oct 7, 2022
e696d78
cargo update
fschutt Oct 7, 2022
7147694
Fix "make lint"
fschutt Oct 7, 2022
0194f8e
cargo fmt
fschutt Oct 7, 2022
862b155
Always set WASMER_BUILD_GIT_HASH
fschutt Oct 7, 2022
b1c3f33
cargo fmt
fschutt Oct 7, 2022
5bc1914
Try fixing errors when parsing CLI args: todo: undo clap migration
fschutt Oct 7, 2022
5fe8167
Downgrade to clap v3
fschutt Oct 9, 2022
5161078
cargo fmt
fschutt Oct 9, 2022
d0372d5
Fix last changes from upgrading to clap v4
fschutt Oct 9, 2022
5b16e05
Fix issue in chrono dependency
fschutt Oct 9, 2022
0ed3be1
Fix clippy issues
fschutt Oct 9, 2022
df047b0
Impl Debug for Binfmt
fschutt Oct 9, 2022
6eade9b
Fix issue in Binfmt::debug
fschutt Oct 9, 2022
25c4fd4
Fix issue with CLI run command not finding PathBuf to wasm file
fschutt Oct 10, 2022
6f15d1a
Fix build issue on linux-musl
fschutt Oct 10, 2022
03fe253
Fix CI
fschutt Oct 10, 2022
5b72f97
Add option to auto-install packages by command name ("wasmer ls")
fschutt Oct 10, 2022
8f29862
Refactor creating SpinnerHandle
fschutt Oct 11, 2022
5a2d45a
Merge remote-tracking branch 'origin/master' into wasmer-vv
fschutt Oct 11, 2022
8d0fbbb
Update Cargo.lock
fschutt Oct 11, 2022
f763887
Merge branch 'master' into wasmer-vv
fschutt Oct 11, 2022
471fd36
Rework main CLI command flow into smaller functions
fschutt Oct 13, 2022
9e278c3
Fix make lint
fschutt Oct 13, 2022
0da844d
CLI: use `get_subcommands()` instead of manual definition
fschutt Oct 13, 2022
600dc2d
Replace package parsing with regex
fschutt Oct 13, 2022
a1548ec
Use split_once instead of split().nth(1)
fschutt Oct 13, 2022
7419be1
Merge RunWithoutFile and Run
fschutt Oct 13, 2022
04b5c43
Lazy-return string on Error
fschutt Oct 13, 2022
b96f354
Drop unnecessary loop labels
fschutt Oct 13, 2022
b4e84c0
Use `unwrap_or(name)`
fschutt Oct 13, 2022
9455b6b
Nit: update version == Some(&v.version)
fschutt Oct 13, 2022
be8689a
Merge branch 'master' into wasmer-vv
fschutt Oct 13, 2022
5f831fc
Fix make lint
fschutt Oct 13, 2022
2c3f164
Use anyhow::ensure
fschutt Oct 13, 2022
f5eb1cb
Added timeout of 5 minutes for re-querying packages from registry
fschutt Oct 14, 2022
f8f33a8
Add --force install method (download packages without cooldown)
fschutt Oct 14, 2022
f8151e1
Merge branch 'master' into wasmer-vv
fschutt Oct 14, 2022
d518173
Fix CI on Linux
fschutt Oct 14, 2022
ef1bc7b
Fix "make lint"
fschutt Oct 14, 2022
b5bf287
Fix CI on Linux again
fschutt Oct 14, 2022
68a55c2
Merge branch 'master' into wasmer-vv
fschutt Oct 17, 2022
a960498
Add unit test and fix problems when installing package
fschutt Oct 17, 2022
db16c20
Rework "wasmer list"
fschutt Oct 17, 2022
7c05da0
Fix issue when running wasmer/wit-pack and wabt
fschutt Oct 18, 2022
0f59d59
Fix "make lint"
fschutt Oct 18, 2022
1a398c4
Fix more wasmer run UX bugs
fschutt Oct 18, 2022
fd5fa25
Do not auto-download packages if the execution failed
fschutt Oct 18, 2022
0c8b40e
Merge branch 'master' into wasmer-vv
fschutt Oct 18, 2022
d6719b7
Add (failing) integration test for wasmer run
fschutt Oct 18, 2022
3dcd2a6
Add support for specifying registry in CLI arg
fschutt Oct 18, 2022
5b165b6
Fix execution of python/python with wrong directory mapping
fschutt Oct 18, 2022
f761ddc
Make python/python run
fschutt Oct 18, 2022
1f59a5c
Run packages via specifying directory instead of package name
fschutt Oct 19, 2022
7487249
Merge branch 'master' into wasmer-vv
fschutt Oct 19, 2022
3de6286
Fix "make lint"
fschutt Oct 19, 2022
b887a83
Fix integration test, add option to run directory with wapm.toml
fschutt Oct 19, 2022
8eec46a
Fix "make lint"
fschutt Oct 19, 2022
2f61896
Add integration test for running directory
fschutt Oct 19, 2022
d89db1f
Fix bug when running file with directory
fschutt Oct 19, 2022
69cd380
Fix issue with wasmer run not working correctly
fschutt Oct 19, 2022
71e7ee1
Fix CI
fschutt Oct 19, 2022
ab4b001
Fix integration test
fschutt Oct 20, 2022
fcc7962
Debug failing test on linux-musl
fschutt Oct 20, 2022
e57b7bb
Ran rustfmt
Oct 21, 2022
b0ba0f6
Merge branch 'master' into wasmer-vv
fschutt Oct 24, 2022
81350bc
Adress review comments
fschutt Oct 24, 2022
acb529a
Disable segfaulting unit test on linux-musl
fschutt Oct 24, 2022
9b0ac57
Fix failing unit test (refactor)
fschutt Oct 24, 2022
90ab3e6
Properly disable unit test on -musl
fschutt Oct 24, 2022
9e4b445
Disable test_wasmer_run_works on musl due to failing network access
fschutt Oct 24, 2022
9a54036
Merge branch 'master' into wasmer-vv
Oct 25, 2022
e268ad1
Merge branch 'master' into wasmer-vv
fschutt Oct 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
970 changes: 870 additions & 100 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ members = [
"lib/c-api/tests/wasmer-c-api-test-runner",
"lib/c-api/examples/wasmer-capi-examples-runner",
"lib/types",
"lib/registry",
"tests/wasi-wast",
"tests/lib/wast",
"tests/lib/compiler-test-derive",
Expand Down
10 changes: 9 additions & 1 deletion lib/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,14 @@ wasmer-wasi-experimental-io-devices = { version = "=3.0.0-beta.2", path = "../wa
wasmer-wast = { version = "=3.0.0-beta.2", path = "../../tests/lib/wast", optional = true }
wasmer-cache = { version = "=3.0.0-beta.2", path = "../cache", optional = true }
wasmer-types = { version = "=3.0.0-beta.2", path = "../types" }
wasmer-registry = { version = "=3.0.0-beta.2", path = "../registry" }
wasmer-object = { version = "=3.0.0-beta.2", path = "../object", optional = true }
wasmer-vfs = { version = "=3.0.0-beta.2", path = "../vfs", default-features = false, features = ["host-fs"] }
atty = "0.2"
colored = "2.0"
anyhow = "1.0"
clap = { version = "3.1", features = ["derive"] }
spinner = "0.5.0"
clap = { version = "3.2.22", features = ["derive"] }
# For the function names autosuggestion
distance = "0.4"
# For the inspect subcommand
Expand All @@ -56,6 +58,12 @@ http_req = { version="^0.8", default-features = false, features = ["rust-tls"],
dirs = { version = "4.0", optional = true }
serde_json = { version = "1.0", optional = true }
target-lexicon = { version = "0.12", features = ["std"] }
prettytable-rs = "0.9.0"
wapm-toml = "0.1.2"
walkdir = "2.3.2"

[build-dependencies]
chrono = { version = "^0.4", default-features = false, features = [ "std", "clock" ] }

[target.'cfg(target_os = "linux")'.dependencies]
unix_mode = "0.1.3"
Expand Down
26 changes: 24 additions & 2 deletions lib/cli/build.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,26 @@
use chrono::prelude::*;
use std::process::Command;

pub fn main() {
println!("cargo:rerun-if-changed=build.rs");
println!("cargo:rerun-if-env-changed=WASMER_INSTALL_PREFIX");
fschutt marked this conversation as resolved.
Show resolved Hide resolved
// Set WASMER_GIT_HASH
let git_hash = Command::new("git")
.args(&["rev-parse", "HEAD"])
.output()
.ok()
.and_then(|output| String::from_utf8(output.stdout).ok())
.unwrap_or_default();
println!("cargo:rustc-env=WASMER_BUILD_GIT_HASH={}", git_hash);

if git_hash.len() > 5 {
println!(
"cargo:rustc-env=WASMER_BUILD_GIT_HASH_SHORT={}",
&git_hash[..5]
);
} else {
println!("cargo:rustc-env=WASMER_BUILD_GIT_HASH_SHORT=??????");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only thing I'm unsure about - for some reason the git hash is not available on -musl, not sure if it's 100% necessary, this is more of a quick hack

}

let utc: DateTime<Utc> = Utc::now();
let date = utc.format("%Y-%m-%d").to_string();
println!("cargo:rustc-env=WASMER_BUILD_DATE={}", date);
}
280 changes: 274 additions & 6 deletions lib/cli/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,12 @@ use crate::commands::CreateExe;
use crate::commands::CreateObj;
#[cfg(feature = "wast")]
use crate::commands::Wast;
use crate::commands::{Cache, Config, Inspect, Run, SelfUpdate, Validate};
use crate::commands::{Cache, Config, Inspect, Run, RunWithoutFile, SelfUpdate, Validate};
use crate::error::PrettyError;
use anyhow::Result;

use clap::{ErrorKind, Parser};
use wasmer_registry::get_all_local_packages;

#[derive(Parser)]
#[derive(Parser, Debug)]
#[cfg_attr(
not(feature = "headless"),
clap(
Expand Down Expand Up @@ -146,7 +145,7 @@ enum WasmerCLIOptions {
}

impl WasmerCLIOptions {
fn execute(&self) -> Result<()> {
fn execute(&self) -> Result<(), anyhow::Error> {
match self {
Self::Run(options) => options.execute(),
Self::SelfUpdate(options) => options.execute(),
Expand Down Expand Up @@ -174,6 +173,10 @@ pub fn wasmer_main() {
#[cfg(windows)]
colored::control::set_virtual_terminal(true).unwrap();

PrettyError::report(wasmer_main_inner())
}

fn wasmer_main_inner() -> Result<(), anyhow::Error> {
// We try to run wasmer with the normal arguments.
// Eg. `wasmer <SUBCOMMAND>`
// In case that fails, we fallback trying the Run subcommand directly.
Expand All @@ -183,6 +186,32 @@ pub fn wasmer_main() {
// we assume that we're registered via binfmt_misc
let args = std::env::args().collect::<Vec<_>>();
let binpath = args.get(0).map(|s| s.as_ref()).unwrap_or("");

let firstarg = args.get(1).map(|s| s.as_str());
let secondarg = args.get(2).map(|s| s.as_str());

match (firstarg, secondarg) {
(None, _) | (Some("help"), _) | (Some("--help"), _) => {
return print_help(true);
}
(Some("-h"), _) => {
return print_help(false);
}
(Some("-vV"), _)
| (Some("version"), Some("--verbose"))
| (Some("--version"), Some("--verbose")) => {
return print_version(true);
}

(Some("-v"), _) | (Some("-V"), _) | (Some("version"), _) | (Some("--version"), _) => {
return print_version(false);
}
(Some("list"), _) => {
return print_packages();
}
_ => {}
}

let command = args.get(1);
let options = if cfg!(target_os = "linux") && binpath.ends_with("wasmer-binfmt-interpreter") {
WasmerCLIOptions::Run(Run::from_binfmt_args())
Expand All @@ -204,5 +233,244 @@ pub fn wasmer_main() {
}
};

PrettyError::report(options.execute());
// Check if the file is a package name
if let WasmerCLIOptions::Run(r) = &options {
if !r.path.exists() {
let package = format!("{}", r.path.display());
if let Ok(mut sv) = split_version(&package) {
let mut package_download_info = None;
if !sv.package.contains('/') {
let sp = spinner::SpinnerBuilder::new(format!(
"Looking up command {} ...",
sv.package
))
.spinner(vec![
"⣾", "⣽", "⣻", "⢿", "⡿", "⣟", "⣯", "⣷", " ", "⠁", "⠂", "⠄", "⡀", "⢀", "⠠",
"⠐", "⠈",
fschutt marked this conversation as resolved.
Show resolved Hide resolved
])
.start();

for registry in
wasmer_registry::get_all_available_registries().unwrap_or_default()
{
let result =
wasmer_registry::query_command_from_registry(&registry, &sv.package);
print!("\r\n");
let command = sv.package.clone();
if let Ok(o) = result {
package_download_info = Some(o.clone());
sp.close();
sv.package = o.package;
sv.version = Some(o.version);
sv.command = Some(command);
break;
}
}
}

if let Some(package) =
wasmer_registry::get_local_package(&sv.package, sv.version.as_deref())
{
let local_package_wasm_path = wasmer_registry::get_package_local_wasm_file(
&package.registry,
&package.name,
&package.version,
)
.map_err(|e| anyhow!("{e}"))?;

// Try finding the local package
let mut args_without_package = args.clone();
args_without_package.remove(1);

let mut run_args = RunWithoutFile::try_parse_from(args_without_package.iter())?;
run_args.command_name = sv.command.clone();
return run_args
.into_run_args(local_package_wasm_path, Some(package.manifest))
.execute();
}

// else: local package not found
let sp =
spinner::SpinnerBuilder::new(format!("Installing package {} ...", sv.package))
.spinner(vec![
"⣾", "⣽", "⣻", "⢿", "⡿", "⣟", "⣯", "⣷", " ", "⠁", "⠂", "⠄", "⡀", "⢀",
"⠠", "⠐", "⠈",
fschutt marked this conversation as resolved.
Show resolved Hide resolved
])
.start();

let v = sv.version.as_deref();
let result =
wasmer_registry::install_package(&sv.package, v, package_download_info);
sp.close();
print!("\r\n");
match result {
Ok((package, buf)) => {
// Try auto-installing the remote package
let mut args_without_package = args.clone();
args_without_package.remove(1);

let mut run_args =
RunWithoutFile::try_parse_from(args_without_package.iter())?;
run_args.command_name = sv.command.clone();

return run_args
.into_run_args(buf, Some(package.manifest))
.execute();
}
Err(e) => {
println!("{e}");
return Ok(());
}
}
}
}
fschutt marked this conversation as resolved.
Show resolved Hide resolved
}

options.execute()
}

#[derive(Debug, Clone, PartialEq, Default)]
struct SplitVersion {
package: String,
version: Option<String>,
command: Option<String>,
}

fn split_version(s: &str) -> Result<SplitVersion, anyhow::Error> {
let prohibited_package_names = [
"run",
"cache",
"validate",
"compile",
"create-exe",
"create-obj",
"config",
"inspect",
"wast",
"help",
];
fschutt marked this conversation as resolved.
Show resolved Hide resolved

let package_version = s.split('@').collect::<Vec<_>>();
let (mut package, mut version) = match *package_version.as_slice() {
[p, v] => (p.trim().to_string(), Some(v.trim().to_string())),
[p] => (p.trim().to_string(), None),
_ => {
return Err(anyhow!("Invalid package / version: {s:?}"));
}
};

let version_clone = version.clone().unwrap_or_default();
let command = if package.contains(':') {
let package_command = version_clone.split('@').collect::<Vec<_>>();
let (p, c) = match package_command.as_slice() {
[p, v] => (p.trim().to_string(), Some(v.trim().to_string())),
[p] => (p.trim().to_string(), None),
_ => {
return Err(anyhow!("Invalid package / command: {s:?}"));
}
};
package = p;
c
} else if version_clone.contains(':') {
let version_command = version_clone.split('@').collect::<Vec<_>>();
let (v, command) = match version_command.as_slice() {
[p, v] => (p.trim().to_string(), Some(v.trim().to_string())),
[p] => (p.trim().to_string(), None),
_ => {
return Err(anyhow!("Invalid version / command: {s:?}"));
}
};
version = Some(v);
command
} else {
None
};
fschutt marked this conversation as resolved.
Show resolved Hide resolved

if prohibited_package_names.contains(&package.trim()) {
return Err(anyhow::anyhow!("Invalid package name {package:?}"));
}
fschutt marked this conversation as resolved.
Show resolved Hide resolved

Ok(SplitVersion {
package,
version,
command,
})
}

fn print_packages() -> Result<(), anyhow::Error> {
fschutt marked this conversation as resolved.
Show resolved Hide resolved
use prettytable::{format, row, Table};

let rows = get_all_local_packages()
.into_iter()
.map(|pkg| {
let commands = pkg
.manifest
.command
.unwrap_or_default()
.iter()
.map(|c| c.get_name())
.collect::<Vec<_>>()
.join(" \r\n");

row![pkg.registry, pkg.name, pkg.version, commands]
})
.collect::<Vec<_>>();

let empty_table = rows.is_empty();
if empty_table {
println!("--------------------------------------");
println!("Registry Package Version Commands ");
println!("======================================");
println!();
} else {
let mut table = Table::init(rows);
table.set_titles(row!["Registry", "Package", "Version", "Commands"]);
table.set_format(*format::consts::FORMAT_NO_LINESEP_WITH_TITLE);
table.set_format(*format::consts::FORMAT_NO_COLSEP);
let _ = table.printstd();
}
fschutt marked this conversation as resolved.
Show resolved Hide resolved

Ok(())
}

fn print_help(verbose: bool) -> Result<(), anyhow::Error> {
use clap::CommandFactory;
let mut cmd = WasmerCLIOptions::command();
if verbose {
let _ = cmd.print_long_help();
} else {
let _ = cmd.print_help();
}
Ok(())
}

#[allow(unused_mut, clippy::vec_init_then_push)]
fn print_version(verbose: bool) -> Result<(), anyhow::Error> {
if !verbose {
println!("wasmer {}", env!("CARGO_PKG_VERSION"));
} else {
println!(
"wasmer {} ({} {})",
env!("CARGO_PKG_VERSION"),
env!("WASMER_BUILD_GIT_HASH_SHORT"),
env!("WASMER_BUILD_DATE")
);
println!("binary: {}", env!("CARGO_PKG_NAME"));
println!("commit-hash: {}", env!("WASMER_BUILD_GIT_HASH"));
println!("commit-date: {}", env!("WASMER_BUILD_DATE"));
println!("host: {}", target_lexicon::HOST);
println!("compiler: {}", {
let mut s = Vec::<&'static str>::new();

#[cfg(feature = "singlepass")]
s.push("singlepass");
#[cfg(feature = "cranelift")]
s.push("cranelift");
#[cfg(feature = "llvm")]
s.push("llvm");

s.join(",")
});
}
Ok(())
}
4 changes: 2 additions & 2 deletions lib/cli/src/commands/binfmt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use std::os::unix::fs::MetadataExt;
use std::path::{Path, PathBuf};
use Action::*;

#[derive(Parser, Clone, Copy)]
#[derive(Debug, Parser, Clone, Copy)]
enum Action {
/// Register wasmer as binfmt interpreter
Register,
Expand All @@ -22,7 +22,7 @@ enum Action {
///
/// Check the wasmer repository for a systemd service definition example
/// to automate the process at start-up.
#[derive(Parser)]
#[derive(Debug, Parser)]
pub struct Binfmt {
// Might be better to traverse the mount list
/// Mount point of binfmt_misc fs
Expand Down
Loading