[Brainstorming] High-level API based on wasi-common?

[sunfishcode]

Would it make sense to create a high-level std::fs-like API, but capability-based, on top of wasi-common? I'm not aware of a present need for such a thing, so this is just brainstorming, but I'm interested if others have thoughts on this.

It is interesting that Rust’s from_raw_fd is unsafe. It turns out that this is an arbitrary choice. Rust could have decided that any code can take any integer and do any I/O operation on it (on unix-style platforms), and it wouldn’t lead to any memory unsafety or any undefined behavior. But instead, Rust chose for safe to mean more than just a lack of undefined behavior; it also applies to I/O.

This means that once you have an open I/O stream, the only way to access that stream (without unsafe) is through the Rust object representing it. This has close parallels to a capability-based security model. However, Rust still lets you open files by absolute path.

So what if we made a high-level API that was similar to std::fs, but capability-based, on top of wasi-common? Instead of File::open and read_dir being standalone functions (ambient authority) that take Paths, there'd be a Dir struct representing a directory handle, with methods like Dir::open_file and Dir::open_dir that only open files and directories relative to and within that directory, and a Dir::read which would return a ReadDir for iterating over its contents.

[jedisct1]

That would be very useful and make common operations easier.

But this is something that, eventually, will have to be written for every single programming language.

We had to do something similar for AssemblyScript already. Because exposing only file descriptors is neither idiomatic nor convenient.

The problem with everybody reimplementing high-level wrappers is that they may end up being completely different from one implementation to another.

Having implementations explicitly described as reference implementations may help having something a little more uniform, at least for how functions and structures are named.

As a side note, at least for filesystem operations, I really wish we adopted mechanisms similar to OpenBSD's unveil(2) instead of pre-opened descriptors. If only because application themselves can easily restrict the filesystem's view as needed. For other capabilities, an equivalent to pledge(2) would also be a great step forward. Because once again, applications know when they can drop capabilities, and which ones.

[sunfishcode]

The long-term goal here is not to reimplement it for every programming langauge, but to build libraries that could be reused, and eventually uses OS's that provide support directly, such as via the proposed openat2 for Linux and Capsicum, which will hopefully obviate much of path_get on applicable platforms.

The pre-open equivalent of dropping privleges would be to close the pre-opened descriptors. wasi-common doesn't support that yet, however thinking about it in this context, it seems like it would be useful. I've now filed https://github.com/WebAssembly/WASI/issues/86 to track this.

Concerning unveil and pledge; they're process-scoped. And unveil is path-based. They're improvements over traditional Unix, and relatively easy to fit into existing Unix applications. But the object-capability model (and to a lesser degree, its integer file-descriptor-based approximation) can describe capabilities with finer precision and make it easier to eg. apply PoLA to libraries without the complexity and overhead of creating new processes and talking to them through IPC.

[sunfishcode]

Closing this issue; the fs module mock-up is now in the tree and we can build from there.