tests: add WASI integration test support via wasmtime by DePasqualeOrg · Pull Request #11717 · uutils/coreutils

DePasqualeOrg · 2026-04-08T19:10:42Z

This PR adds WASI integration test support, enabling ~1,250 existing integration tests across 26 tools to run against the WASI binary via wasmtime.

The changes have gone through multiple rounds of review and refinement with Claude Code and Codex. I have tested them on macOS and Linux.

The test framework now supports a UUTESTS_WASM_RUNNER environment variable. When set (e.g., to wasmtime), the test harness wraps each command invocation through the runner instead of executing the binary directly. The host-compiled integration tests exercise the WASI binary end to end without needing to compile the test harness itself for WASI.

Key changes:

tests/tests.rs: Allow UUTESTS_BINARY_PATH to be overridden by the environment, so tests can target a WASI binary instead of the host binary
tests/uutests/src/lib/util.rs: When UUTESTS_WASM_RUNNER is set, wrap command invocation through the runner with --dir, --argv0, and --env flags to map the working directory, set the binary name, and forward environment variables to the WASI guest
tests/uutests/src/lib/util.rs: Adjust usage_error assertions to use the filename (from --argv0) instead of the full path when a WASM runner is active
.github/workflows/wasi.yml: Add an integration test step that builds the WASI binary and runs tests through wasmtime
tests/by-util/test_basenc.rs, test_comm.rs, test_od.rs, test_tr.rs, test_wc.rs: Add conditional expectations for error messages, formatting, and collation behavior that differ between host libc and wasi-libc

Tools tested (26)

base32, base64, basenc, basename, comm, cut, dirname, echo, expand, factor, false, fold, head, link, nl, numfmt, od, paste, printf, shuf, sum, tee, tr, true, truncate, unexpand, unlink, wc

Adding more tools

Tools not yet included in the integration test run are tracked in a TODO comment in .github/workflows/wasi.yml. As WASI support is extended for these tools, they can be moved from the TODO list to the test list, with any necessary skips added for WASI-specific limitations.

Skipped tests by category

Tests are skipped for documented WASI platform limitations:

Host paths not visible in guest (20): Tests using /proc, /sys, or /dev paths that are not available inside the WASI sandbox (the guest root is mapped to the test's working directory)
WASI spec: argv/filenames must be valid UTF-8 (19): wasmtime enforces valid UTF-8 for command-line arguments and filenames per the WASI spec
No FIFO/mkfifo support (3): WASI preview1 does not support FIFOs
No pipe/signal support (2): WASI preview1 does not support pipe signals
No subprocess spawning (3): Tests that shell out or invoke another binary
stdin file position (3): stdin file position not preserved through wasmtime

Add UUTESTS_WASM_RUNNER support to the test framework, enabling host-compiled integration tests to exercise a WASI binary through wasmtime. This runs ~1,230 existing tests across 26 tools without needing to compile the test harness for WASI.

oech3 · 2026-04-09T00:12:16Z

Why do not just use #[cfg] ~~#[ignore]~~ for the target at test code?

DePasqualeOrg · 2026-04-09T08:30:03Z

#[cfg] conditionally compiles based on what platform the test binary is targeting. But here the test harness runs natively on the host; only the coreutils binary under test is compiled for WASI. So #[cfg(target_os = "wasi")] would never fire in the test code.

Compiling the test harness itself for WASI isn't practical, since it depends on Unix-specific crates like nix and rlimit, and uucore features like utmpx that have no WASI equivalent. The runtime env var check is how the host-side test code knows it's exercising a WASI binary through wasmtime rather than a native one.

oech3 · 2026-04-09T08:32:10Z

I meant excluding unsupported tests by cfg instead of maintaining cargo's args at *.yml.

DePasqualeOrg · 2026-04-09T08:46:58Z

The alternative would be annotating dozens of tests across many files, and the skip reasons (no FIFO, no pipes, no UTF-8 filenames, etc.) are easier to understand when kept in one place. Also, as WASI support expands (for example, with the other two PRs that I submitted), tests will move off the skip lists, and having them in one place makes that easier to track.

github-actions · 2026-04-09T08:53:24Z

GNU testsuite comparison:

Note: The gnu test tests/csplit/csplit-heap is now being skipped but was previously passing.
Congrats! The gnu test tests/cut/cut-huge-range is now passing!

github-actions · 2026-04-09T09:00:37Z

GNU testsuite comparison:

Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/cut/cut-huge-range is now passing!

sylvestre · 2026-04-09T09:22:08Z

.github/workflows/wasi.yml

+        #   vdir yes
+        UUTESTS_BINARY_PATH="$(pwd)/target/wasm32-wasip1/debug/coreutils.wasm" \
+        UUTESTS_WASM_RUNNER=wasmtime \
+        cargo test --test tests -- \


this long list isn't ideal
we should #ignore them for wasi in the .rs files

I can change this if you feel strongly about it, but then we lose the centralized tracking of what's not yet supported, but soon could be.

Or, if a separate centralized list is maintained somewhere, it would need to be kept up to date with the #ignore annotations.

GitHub has search feature.

you can "grep" it easily in the code

The tradeoff is that there would not be one place where the coverage is understandable at a glance, and that information would need to be aggregated from many files.

I'm happy to make whatever changes you require, but just wanted to make the tradeoff clear. Having a centralized account of gaps in support would be helpful for me, since I hope to expand WASI support for these tools over time.

I switched to test annotations in the latest commit. Does this look good to you now?

github-actions · 2026-04-09T12:34:52Z

GNU testsuite comparison:

Skip an intermittent issue tests/date/resolution (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/cp/link-heap is now passing!
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/seq/seq-epipe is now passing!

sylvestre · 2026-04-09T20:02:12Z

please fix:
Error: ERROR: cspell: Unknown word 'seekable' (file:'docs/src/wasi-test-gaps.md', line:33)

This was referenced Apr 8, 2026

ln: add WASI support via symlink_path #11713

Open

wasi: atomics-aware threading and synchronous sort pipeline #11712

Open

Fix spelling and formatting

1396871

Fix linter errors

45048e8

sylvestre reviewed Apr 9, 2026

View reviewed changes

move WASI skip list from workflow into test annotations

6e530d6

Update jargon.wordlist.txt

7d666b9

Uh oh!

Conversation

DePasqualeOrg commented Apr 8, 2026

Tools tested (26)

Adding more tools

Skipped tests by category

Uh oh!

oech3 commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DePasqualeOrg commented Apr 9, 2026

Uh oh!

oech3 commented Apr 9, 2026

Uh oh!

DePasqualeOrg commented Apr 9, 2026

Uh oh!

github-actions bot commented Apr 9, 2026

Uh oh!

github-actions bot commented Apr 9, 2026

Uh oh!

sylvestre Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

DePasqualeOrg Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

oech3 Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

sylvestre Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

DePasqualeOrg Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

DePasqualeOrg Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 9, 2026

Uh oh!

sylvestre commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oech3 commented Apr 9, 2026 •

edited

Loading