Redesign Wasmtime's CLI

[alexcrichton]

This commit follows through on discussion from #6741 to redesign the flags that the wasmtime binary accepts on the CLI. Almost all flags have been renamed/moved and will require callers to update. The main motivation here is to cut down on the forest of options in wasmtime -h which are difficult to mentally group together and understand.

The main change implemented here is to move options behind "option groups" which are intended to be abbreviated with a single letter:

• -O foo - an optimization or performance-tuning related option
• -C foo - a codegen option affecting the compilation process.
• -D foo - a debug-related option
• -W foo - a wasm-related option, for example changing wasm semantics
• -S foo - a WASI-related option, configuring various proposals for example

Each option group can be explored by passing help, for example -O help. This will print all options within the group along with their help message. Additionally -O help-long can be passed to print the full comment for each option if desired.

Option groups can be specified multiple times on the command line, for example -Wrelaxed-simd -Wthreads. They can also be combined together with commas as -Wrelaxed-simd,threads. Configuration works as a "last option wins" so -Ccache,cache=n would end up with a compilation cache disabled.

Boolean options can be specified as -C foo to enable foo, or they can be specified with -Cfoo=$val with any of y, n, yes, no, true, or false. All other options require a =foo value to be passed and the parsing depends on the type.

This commit additionally applies a few small refactorings to the CLI as well. For example the help text no longer prints information about wasm features after printing the option help. This is still available via -Whelp as all wasm features have moved from --wasm-features to -W. Additionally flags are no longer conditionally compiled in, but instead all flags are always supported. A runtime error is returned if support for a flag is not compiled in. Additionally the "experimental" name of WASI proposals has been dropped in favor of just the name of the proposal, for example --wasi nn instead of --wasi-modules experimental-wasi-nn. This is intended to mirror how wasm proposals don't have "experimental" in the name and an opt-in is required regardless.

Full listing of flags and how they have changed

old cli flag	new cli flag
-O, --optimize	removed
--opt-level <LEVEL>	-O opt-level=N
--dynamic-memory-guard-size <SIZE>	-O dynamic-memory-guard-size=...
--static-memory-forced	-O static-memory-forced
--static-memory-guard-size <SIZE>	-O static-memory-guard-size=N
--static-memory-maximum-size <MAXIMUM>	-O static-memory-maximum-size=N
--dynamic-memory-reserved-for-growth <SIZE>	-O dynamic-memory-reserved-for-growth=...
--pooling-allocator	-O pooling-allocator
--disable-memory-init-cow	-O memory-init-cow=no
--compiler <COMPILER>	-C compiler=..
--enable-cranelift-debug-verifier	-C cranelift-debug-verifier
--cranelift-enable <SETTING>	-C cranelift-NAME
--cranelift-set <NAME=VALUE>	-C cranelift-NAME=VALUE
--config <CONFIG_PATH>	-C cache-config=..
--disable-cache	-C cache=no
--disable-parallel-compilation	-C parallel-compilation=no
-g	-D debug-info
--disable-address-map	-D address-map=no
--disable-logging	-D logging=no
--log-to-files	-D log-to-files
--coredump-on-trap <PATH>	-D coredump=..
--wasm-features all	-W all-proposals
--wasm-features -all	-W all-proposals=n
--wasm-features bulk-memory	-W bulk-memory
--wasm-features multi-memory	-W multi-memory
--wasm-features multi-value	-W multi-value
--wasm-features reference-types	-W reference-types
--wasm-features simd	-W simd
--wasm-features tail-call	-W tail-call
--wasm-features threads	-W threads
--wasm-features memory64	-W memory64
--wasm-features copmonent-model	-W component-model
--wasm-features function-references	-W function-references
--relaxed-simd-deterministic	-W relaxed-simd-deterministic
--enable-cranelift-nan-canonicalization	-W nan-canonicalization
--fuel <N>	-W fuel=N
--epoch-interruption	-W epoch-interruption
--allow-unknown-exports	-W unknown-exports-allow
--trap-unknown-imports	-W unknown-imports-trap
--default-values-unknown-imports	-W unknown-imports-default
--max-instances <MAX_INSTANCES>	-W max-instances=N
--max-memories <MAX_MEMORIES>	-W max-memories=N
--max-memory-size <BYTES>	-W max-memory-size=N
--max-table-elements <MAX_TABLE_ELEMENTS>	-W max-table-elements=N
--max-tables <MAX_TABLES>	-W max-tables=N
--max-wasm-stack <MAX_WASM_STACK>	-W max-wasm-stack=N
--trap-on-grow-failure	-W trap-on-grow-failure
--wasm-timeout <TIME>	-W timeout=N
--wmemcheck	-W wmemcheck
--wasi-modules default	removed
--wasi-modules -default	removed
--wasi-modules wasi-common	-S common
--wasi-modules -wasi-common	-S common=n
--wasi-modules experimental-wasi-nn	-S nn
--wasi-modules experimental-wasi-threads	-S threads
--wasi-modules experimental-wasi-http	-S http
--listenfd	-S listenfd
--tcplisten <SOCKET ADDRESS>	-S tcplisten=...
--wasi-nn-graph <FORMAT::HOST>	-S nn-graph=FORMAT::HOST
--preview2	-S preview2
--dir <DIRECTORY>	--dir ...
--mapdir <GUEST_DIR::HOST_DIR>	--dir a::b

[alexcrichton]

I'll note that this is a breaking change and we're in the period of "no breaking changes before WasmCon", so this shouldn't land for a few weeks. Additionally some more background for this is available at #6741 too.

Finally here's some snippets:

./target/debug/wasmtime -h

Wasmtime WebAssembly Runtime

Usage: wasmtime [OPTIONS] <WASM>...
       wasmtime <COMMAND>

Commands:
  config    Controls Wasmtime configuration settings
  compile   Compiles a WebAssembly module
  explore   Explore the compilation of a WebAssembly module to native code
  run       Runs a WebAssembly module
  settings  Displays available Cranelift settings for a target
  wast      Runs a WebAssembly test script file
  help      Print this message or the help of the given subcommand(s)

Arguments:
  <WASM>...  The WebAssembly module to run and arguments to pass to it

Options:
  -O, --optimize <KEY[=VAL[,..]]>    Optimization and tuning related options for wasm performance, `-O help` to see all
  -C, --codegen <KEY[=VAL[,..]]>     Codegen-related configuration options, `-C help` to see all
  -D, --debug <KEY[=VAL[,..]]>       Debug-related configuration options, `-D help` to see all
  -W, --wasm <KEY[=VAL[,..]]>        Options for configuring semantic execution of WebAssembly, `-W help` to see all
  -S, --wasi <KEY[=VAL[,..]]>        Options for configuring WASI and its proposals, `-S help` to see all
      --allow-precompiled            Allow executing precompiled WebAssembly modules as `*.cwasm` files
      --dir <GUEST_DIR[::HOST_DIR]>  Grant access of a host directory to a guest
      --env <NAME[=VAL]>             Pass an environment variable to the program
      --invoke <FUNCTION>            The name of the function to run
      --preload <NAME=MODULE_PATH>   Load the given WebAssembly module before the main module
      --profile <STRATEGY>           Profiling strategy (valid options are: perfmap, jitdump, vtune, guest)
  -h, --help                         Print help (see more with '--help')
  -V, --version                      Print version

If a subcommand is not provided, the `run` subcommand will be used.

Usage examples:

Running a WebAssembly module with a start function:

  wasmtime example.wasm

Passing command line arguments to a WebAssembly module:

  wasmtime example.wasm arg1 arg2 arg3

Invoking a specific function (e.g. `add`) in a WebAssembly module:

  wasmtime example.wasm --invoke add 1 2

./target/debug/wasmtime run -h

Runs a WebAssembly module

Usage: wasmtime run [OPTIONS] <WASM>...

Arguments:
  <WASM>...  The WebAssembly module to run and arguments to pass to it

Options:
  -O, --optimize <KEY[=VAL[,..]]>    Optimization and tuning related options for wasm performance, `-O help` to see all
  -C, --codegen <KEY[=VAL[,..]]>     Codegen-related configuration options, `-C help` to see all
  -D, --debug <KEY[=VAL[,..]]>       Debug-related configuration options, `-D help` to see all
  -W, --wasm <KEY[=VAL[,..]]>        Options for configuring semantic execution of WebAssembly, `-W help` to see all
  -S, --wasi <KEY[=VAL[,..]]>        Options for configuring WASI and its proposals, `-S help` to see all
      --allow-precompiled            Allow executing precompiled WebAssembly modules as `*.cwasm` files
      --dir <GUEST_DIR[::HOST_DIR]>  Grant access of a host directory to a guest
      --env <NAME[=VAL]>             Pass an environment variable to the program
      --invoke <FUNCTION>            The name of the function to run
      --preload <NAME=MODULE_PATH>   Load the given WebAssembly module before the main module
      --profile <STRATEGY>           Profiling strategy (valid options are: perfmap, jitdump, vtune, guest)
  -h, --help                         Print help (see more with '--help')

./target/debug/wasmtime compile -h

Compiles a WebAssembly module

Usage: wasmtime compile [OPTIONS] <MODULE>

Arguments:
  <MODULE>  The path of the WebAssembly to compile

Options:
  -O, --optimize <KEY[=VAL[,..]]>  Optimization and tuning related options for wasm performance, `-O help` to see all
  -C, --codegen <KEY[=VAL[,..]]>   Codegen-related configuration options, `-C help` to see all
  -D, --debug <KEY[=VAL[,..]]>     Debug-related configuration options, `-D help` to see all
  -W, --wasm <KEY[=VAL[,..]]>      Options for configuring semantic execution of WebAssembly, `-W help` to see all
  -S, --wasi <KEY[=VAL[,..]]>      Options for configuring WASI and its proposals, `-S help` to see all
      --target <TARGET>            The target triple; default is the host triple
  -o, --output <OUTPUT>            The path of the output compiled module; defaults to <MODULE>.cwasm
      --emit-clif <PATH>           The directory path to write clif files into, one clif file per wasm function
  -h, --help                       Print help
  -V, --version                    Print version

By default, no CPU features or presets will be enabled for the compilation.

Usage examples:

Compiling a WebAssembly module for the current platform:

  wasmtime compile example.wasm

Specifying the output file:

  wasmtime compile -o output.cwasm input.wasm

Compiling for a specific platform (Linux) and CPU preset (Skylake):

  wasmtime compile --target x86_64-unknown-linux -Ccranelift-skylake foo.wasm

./target/debug/wasmtime -O help

Available optimize options:

  -O                    opt-level=0|1|2|s -- Optimization level of generated code (0-2, s; default: 0)
  -O          dynamic-memory-guard-size=N -- Byte size of the guard region after dynamic memories are allocated
  -O           static-memory-forced[=y|n] -- Force using a "static" style for all wasm memories
  -O         static-memory-maximum-size=N -- Maximum size in bytes of wasm memory before it becomes dynamically relocatable instead of up-front-reserved.
  -O           static-memory-guard-size=N -- Byte size of the guard region after static memories are allocated
  -O dynamic-memory-reserved-for-growth=N -- Bytes to reserve at the end of linear memory for growth for dynamic memories.
  -O              pooling-allocator[=y|n] -- Enable the pooling allocator, in place of the on-demand allocator.
  -O                memory-init-cow[=y|n] -- Configure attempting to initialize linear memory via a copy-on-write mapping (default: yes)

pass `-O help-long` to see longer-form explanations

./target/debug/wasmtime -C help

Available codegen options:

  -C       compiler=winch|cranelift -- Either `cranelift` or `winch`.
  -C cranelift-debug-verifier[=y|n] -- Enable Cranelift's internal debug verifier (expensive)
  -C                    cache[=y|n] -- Whether or not to enable caching of compiled modules.
  -C               cache-config=val -- Configuration for compiled module caching.
  -C     parallel-compilation[=y|n] -- Whether or not to enable parallel compilation of modules.
  -C            cranelift-<KEY>=val -- Set a cranelift-specific option. Use `wasmtime settings` to see all.

pass `-C help-long` to see longer-form explanations

./target/debug/wasmtime -W help

Available wasm options:

  -W       nan-canonicalization[=y|n] -- Enable canonicalization of all NaN values.
  -W                           fuel=N -- Enable execution fuel with N units fuel, trapping after running out of fuel.
  -W         epoch-interruption[=y|n] -- Yield when a global epoch counter changes, allowing for async operation without blocking the executor.
  -W                 max-wasm-stack=N -- Maximum stack size, in bytes, that wasm is allowed to consume before a stack overflow is reported.
  -W      unknown-exports-allow[=y|n] -- Allow unknown exports when running commands.
  -W       unknown-imports-trap[=y|n] -- Allow the main module to import unknown functions, using an implementation that immediately traps, when running commands.
  -W    unknown-imports-default[=y|n] -- Allow the main module to import unknown functions, using an implementation that returns default values, when running commands.
  -W                  wmemcheck[=y|n] -- Enables memory error checking. (see wmemcheck.md for more info)
  -W                max-memory-size=N -- Maximum size, in bytes, that a linear memory is allowed to reach.
  -W             max-table-elements=N -- Maximum size, in table elements, that a table is allowed to reach.
  -W                  max-instances=N -- Maximum number of WebAssembly instances allowed to be created.
  -W                     max-tables=N -- Maximum number of WebAssembly tables allowed to be created.
  -W                   max-memories=N -- Maximum number of WebAssembly linear memories allowed to be created.
  -W       trap-on-grow-failure[=y|n] -- Force a trap to be raised on `memory.grow` and `table.grow` failure instead of returning -1 from these instructions.
  -W              timeout=N|Ns|Nms|.. -- Maximum execution time of wasm code before timing out (1, 2s, 100ms, etc)
  -W              all-proposals[=y|n] -- Configures support for all WebAssembly proposals implemented.
  -W                bulk-memory[=y|n] -- Configure support for the bulk memory proposal.
  -W               multi-memory[=y|n] -- Configure support for the multi-memory proposal.
  -W                multi-value[=y|n] -- Configure support for the multi-value proposal.
  -W            reference-types[=y|n] -- Configure support for the reference-types proposal.
  -W                       simd[=y|n] -- Configure support for the simd proposal.
  -W               relaxed-simd[=y|n] -- Configure support for the relaxed-simd proposal.
  -W relaxed-simd-deterministic[=y|n] -- Configure forcing deterministic and host-independent behavior of the relaxed-simd instructions.
  -W                  tail-call[=y|n] -- Configure support for the tail-call proposal.
  -W                    threads[=y|n] -- Configure support for the threads proposal.
  -W                   memory64[=y|n] -- Configure support for the memory64 proposal.
  -W            component-model[=y|n] -- Configure support for the component-model proposal.
  -W        function-references[=y|n] -- Configure support for the function-references proposal.

pass `-W help-long` to see longer-form explanations

./target/debug/wasmtime -S help

Available wasi options:

  -S             common[=y|n] -- Enable support for WASI common APIs
  -S                 nn[=y|n] -- Enable suport for WASI neural network API (experimental)
  -S            threads[=y|n] -- Enable suport for WASI threading API (experimental)
  -S               http[=y|n] -- Enable suport for WASI HTTP API (experimental)
  -S           listenfd[=y|n] -- Inherit environment variables and file descriptors following the systemd listen fd specification (UNIX only)
  -S            tcplisten=val -- Grant access to the given TCP listen socket
  -S           preview2[=y|n] -- Implement WASI with preview2 primitives (experimental).
  -S nn-graph=<format>::<dir> -- Pre-load machine learning graphs (i.e., models) for use by wasi-nn.

pass `-S help-long` to see longer-form explanations

EDIT: updated with Andrew's suggestion below

[alexcrichton]

Also, as a final note, there's a shed with bikes here and we gotta figure out the color. I'm happy to hand out paint brushes to anyone willing! AKA now's the time for bikeshedding this where we only get to do this once really IMO

[abrown]

bikeshed: in the printed --help output, can we get all the val (I'm only looking under -S but this may apply elsewhere) to be printed as y|n or <encoding>::<dir> or whatever that flag expects? Essentially communicate more clearly what kinds of things are valid after =?

[alexcrichton]

Good point! Updated to that now

[github-actions]

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasmtime:api", "wasmtime:config", "wasmtime:docs"

Thus the following users have been cc'd because of the following labels:

• peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[github-actions]

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

• If you added a new Config method, you wrote extensive documentation for
  it.

  Our documentation should be of the following form:

  Short, simple summary sentence.
  
  More details. These details can be multiple paragraphs. There should be
  information about not just the method, but its parameters and results as
  well.
  
  Is this method fallible? If so, when can it return an error?
  
  Can this method panic? If so, when does it panic?
  
  # Example
  
  Optional example here.
  

• If you added a new Config method, or modified an existing one, you
  ensured that this configuration is exercised by the fuzz targets.

  For example, if you expose a new strategy for allocating the next instance
  slot inside the pooling allocator, you should ensure that at least one of our
  fuzz targets exercises that new strategy.

  Often, all that is required of you is to ensure that there is a knob for this
  configuration option in wasmtime_fuzzing::Config (or one
  of its nested structs).

  Rarely, this may require authoring a new fuzz target to specifically test this
  configuration. See our docs on fuzzing for more details.

• If you are enabling a configuration option by default, make sure that it
  has been fuzzed for at least two weeks before turning it on by default.

---

To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

[abrown]

Makes sense to me! Others may have opinions but I like the idea of making the flags easier to use.

[alexcrichton]

From today's Wasmtime meeting discussion the conclusion is that this PR will land in Wasmtime 14.0.0 and will be coupled with #6737. Despite #6737 being a landed PR on main it was reverted for 12.0.0 and is being reverted for the upcoming 13.0.0 branch. That'll re-land via a separate future PR for the 14.0.0 release once the 13.0.0 branch is created.

[alexcrichton]

I've converted this PR to a draft while we wait for Wasmtime 13.0.0 to get branched. In the meantime I've created a stream on Zulip to discuss these changes and there's a sibling PR for how Wasmtime's options are required to be first now.

[alexcrichton]

Ok now that we're post-WasmCon and given @abrown's previous approval I'm going to go ahead and land this.

[ydnar]

This broke TinyGo (and also mainline Go’s wasip1 runner): tinygo-org/tinygo#3970

Would it be possible to introduce a minor change for backwards compatibility? I think these would be sufficient to fix at least TinyGo and Go:

1. Support --mapdir=GUEST::HOST as an alias for --dir=HOST::GUEST
2. If the command args includes --, then interpret everything before the -- as arguments to wasmtime, and everything after -- as arguments to the guest program

Edit: the above changes would fix TinyGo, but not mainline Go, which uses --max-wasm-stack N (now changed to -W max-wasm-stack=N

[alexcrichton]

Thanks for the report! I suspect that #7336 is a good place to continue discussion perhaps? (looks like y'all raced in reports!)

While it would be possible to implement something like what you mention, is it possible to require Wasmtime 14 and update scripts for Wasmtime 14?

[ydnar]

I made a PR for TinyGo (tinygo-org/tinygo#3972), but landing a change to mainline Go will take time.

We got bit by this on Friday when CI started breaking. While it’s our fault for not pinning the version of wasmtime, I’m somewhat surprised y’all didn’t release at least one major version of wasmtime with a compatibility shim and deprecation warnings, so dependents would have time to migrate.

Edit: for example:

--max-wasm-stack: deprecated in wasmtime v14. Did you mean -W max-wasm-stack=N?

[alexcrichton]

Yes we decided in the development of this that smoothing the path with warnings and such was unfortunately not tenable in this case. For example many previous invocations are reinterpreted entirely differently and we didn't want one release of some breakage and another release of even more breakage. If it works in your CI I'd definitely recommend pinning Wasmtime versions with explicit updates.

FWIW if you're curious we attempted to discuss this broadly with lots of lead time about this. If you've got suggestions for how to improve the communication process and ensure this doesn't slip through the cracks again for y'all, please let us know!

• original issue
• Zulip discussion
• Discussed in a bi-weekly meeting