aws-config depends on tokio+mio - breaks on WASM

[simbleau]

Describe the bug

mio is not wasm-compatible, which is a dep of tokio.

In the latest version of aws-config (0.56) I believe there was a change, specifically around tls, which started to bring this in.

There are the features in aws-config:

[features]
allow-compilation = []
client-hyper = ["aws-smithy-client/client-hyper"]
credentials-sso = ["dep:aws-sdk-sso", "dep:ring", "dep:hex", "dep:zeroize"]
default = ["client-hyper", "rustls", "rt-tokio", "credentials-sso"]
native-tls = []
rt-tokio = ["aws-smithy-async/rt-tokio", "tokio/rt"]
rustls = ["aws-smithy-client/rustls", "client-hyper"]

So I think the idea is we are all migrating from native-tls to rustls?

Here's whats in my toml:

aws-config = { version = "0.56", default-features = false, features = [
  "rustls",
] }

And wallah - Here's my cargo tree:

├── aws-config v0.56.0
│   ├── aws-credential-types v0.56.0
│   │   ├── aws-smithy-async v0.56.0
│   │   │   ├── futures-util v0.3.28 (*)
│   │   │   ├── pin-project-lite v0.2.12
│   │   │   ├── tokio v1.31.0
│   │   │   │   ├── bytes v1.4.0 (*)
│   │   │   │   ├── libc v0.2.147
│   │   │   │   ├── mio v0.8.8            <---- NOT GOOD!
│   │   │   │   │   └── libc v0.2.147
│   │   │   │   ├── num_cpus v1.16.0
│   │   │   │   │   └── libc v0.2.147

Expected Behavior

safe compile with wasm-pack

Current Behavior

You get this heap of errors related to mio:
seanmonstar/reqwest#1300

Reproduction Steps

wasm-pack test --node

Possible Solution

No response

Additional Information/Context

No response

Version

aws-config 0.56

Environment details (OS name and version, etc.)

MacOS Ventura

Logs

No response

[Velfi]

Hey @simbleau, thanks for submitting this issue. We'll add it to our backlog.

I'll do a quick check today to see if this is an easy fix.

[Velfi]

Ongoing investigative PR: smithy-lang/smithy-rs#2919

[rcoh]

Why are you adding the rustls feature? Does that have a use in WASM? If you're trying to use rustls on a connector you probably want to use it directly—our rustls implementation works with Hyper (which you probably don't want!)

[simbleau]

@rcoh - I was using the native-tls feature before, which didn't break on WASM. Since native-tls was removed, the only option was rustls.

Without the rustls option, I received a panic on the connector. I'm not sure what you mean by "use it directly" - you mean without the feature? It doesn't work due to connector issues (I'm unaware of how this works).

[jdisanti]

It looks like you're targeting Node from the wasmpack command you gave. Wouldn't you need the WASM code to call into Javascript to use its HTTP client, or is networking exposed in some other way to the WASM runtime in this case?

[simbleau]

Would node make a difference here vs wasm-pack test --chrome/--safari/--firefox?

[jdisanti]

The target matters quite a bit right now since the SDK doesn't have a default HTTP connector that will work for WASM targets. The HTTP connector needs to be implemented and configured. This is illustrated in the WebAssembly example, which wires up a connector for use in a web browser:
https://github.com/awslabs/aws-sdk-rust/blob/main/examples/webassembly/src/lib.rs

[jdisanti]

There have been a number of PRs to add better WASM support, so I suspect this has been resolved. We have a WASM integration that that is using aws-config with default-features = false and features = ["rt-tokio", "behavior-version-latest"].

It looks like mio is no longer pulled into aws-config in that configuration:

$ cargo tree --no-default-features --features rt-tokio,behavior-version-latest --edges no-build,no-dev | grep mio

Is this still an issue for you with the latest SDK?

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

[simbleau]

Hey I'll look into this soon. Don't close it :)

[ArniDagur]

This is still an issue, even with the suggested configuration of default-features = false, and features set to ["rt-tokio","behavior-version-latest"]. I want to use AWS DynamoDB with Cloudflare Workers, but it is difficult with this bug in place.

My cargo.toml is:

[package]
name = "worker-rust"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
worker = { version="0.0.24", features=["http"] }
aws-sdk-dynamodb = { version = "1.0.0", default-features=false, features=["rt-tokio","behavior-version-latest"]}

Compiling gives the following error:

$ cargo build --target wasm32-unknown-unknown
   Compiling quote v1.0.35
   Compiling wasm-bindgen-shared v0.2.92
   Compiling tokio v1.37.0
   Compiling slab v0.4.9
   Compiling num-traits v0.2.18
   Compiling once_cell v1.19.0
   Compiling percent-encoding v2.3.1
   Compiling generic-array v0.14.7
   Compiling typenum v1.17.0
   Compiling rustversion v1.0.14
error: Only features sync,macros,io-util,rt,time are supported on wasm.
   --> /Users/arni/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/lib.rs:470:1
    |
470 | compile_error!("Only features sync,macros,io-util,rt,time are supported on wasm.");
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

   Compiling serde v1.0.197
   Compiling syn v2.0.57
   Compiling tracing-core v0.1.32
   Compiling phf_shared v0.11.2
   Compiling rand_core v0.6.4
   Compiling regex-syntax v0.8.3
   Compiling semver v1.0.22
   Compiling zeroize v1.7.0
   Compiling num-integer v0.1.46
   Compiling rand v0.8.5
   Compiling wasm-bindgen v0.2.92
   Compiling form_urlencoded v1.2.1
   Compiling async-trait v0.1.79
   Compiling phf_generator v0.11.2
   Compiling subtle v2.5.0
   Compiling phf_codegen v0.11.2
   Compiling phf v0.11.2
error: could not compile `tokio` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

[simbleau]

I also still get mio.

[omarabid]

I was able to get it to work on Cloudflare workers but you have to disable retries, timeouts and stalled_stream_protection. Essentially anything that has to do with time as Cloudflare workers return a fixed time for the whole duration of the execution.

[landonxjames]

Disclaimer: I have not used Cloudflare workers (and if they really return a fixed time for an entire execution that is definitely going to break things)

I have an example of using the SDK in a WASM environment here: https://github.com/landonxjames/wasmtest (although it is based on WASM components so applicability to your use case might depend on if your WASM runtime supports components)

Since it looks like Cloudflare workers support WASI that would probably be an easier path and we have an example of using the SDK in a WASI environment here: https://github.com/smithy-lang/smithy-rs/tree/main/tools/ci-cdk/canary-wasm and it makes use of the WasiHttpConnector from https://docs.rs/aws-smithy-wasm/0.1.3/aws_smithy_wasm/

WASM support is one of our most upvoted issues (#59) and everything does work, but it is not well documented or particularly ergonomic, so I am hoping to work on improving that this year.

[omarabid]

@landonxjames Cloudflare workers doesn't really support Wasi in a usable matter.

and if they really return a fixed time for an entire execution that is definitely going to break things

Do you have an idea what kind of "things" will break? I could forgo retries and stuff like that but just wondering if there are libraries/features that won't work because of it.

[landonxjames]

Do you have an idea what kind of "things" will break? I could forgo retries and stuff like that but just wondering if there are libraries/features that won't work because of it.

I read into this a bit more and the Cloudflare worker documentation on timers makes it a bit more nuanced. Seemingly the time in the environment only advances when IO has been performed. For our cases that could potentially allow some things to work, but without dedicated testing it is very hard to say.

We didn't factor environments that provide IO based time stepping into our designs for the SDK. Off the top of my head this could break retries, credential refreshing, and stalled stream protection.