dynamic_modules: scaffolds Rust SDK #35914
Conversation
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
|
|
||
| Currently, the ABI binding ([src/abi.rs](./src/abi.rs)) is manually generated by [`bindgen`](https://github.com/rust-lang/rust-bindgen) for the ABI header. Ideally, we should be able to do the bindgen in the build.rs file. | ||
|
|
||
| TODO(@mathetake): figure out how to properly setup the bindgen in build.rs with rules_rust. The most recommended way is to use [crate_universe](https://bazelbuild.github.io/rules_rust/crate_universe.html#setup) and use bindgen as a dev-dependency. However, it seems that crate_universe tries to use the underlying gcc system linker which we cannot assume always available. |
There was a problem hiding this comment.
note: i tried so hard (like spent the entire two days) to get the crate_universe working in our envoy build settings but no luck. basically it is officially recommended way to deal with cargo dependencies but it seems not that compatible with our settings. So I had to choose to do the manual bindgen for now - which of course not ideal so I left this TODO.
There was a problem hiding this comment.
We use crate universe here with bazel: https://github.com/bitdriftlabs/capture-sdk. Not sure if there is anything to be learned from that.
There was a problem hiding this comment.
I identified the root cause: rules_rust repo declared in Envoy is a bit old and patched version (used by the comptime option Wasmtime dependency and tests for Wasm extensions. They will never be able to use the vanilla rules_rust), hence crate_universe_dependencies requires bootstrap = True argument due to the reason descrbied in the crate_universe doc:
Note that if the current version of rules_rust is not a release artifact, you may need to set additional flags such as bootstrap = True on the crate_universe_dependencies call above or crates_repository::generator_urls in uses of crates_repository.
This results in rebuilding cargo-bazel which in turns tries to use the system gcc linker. Since we cannot assume (at least for the current build container and CI settings) gcc exists in the system and set in the PATH, that results in the build failure.
I think we have a few options here:
- Nukes Wasmtime and Proxy-Wasm Rust SDK dependency in Envoy Wasm sources/tests. This will allows us to use the latest vanilla rules_rust.
- For Wasmtime, I believe nobody uses it because in anyway it's build time option. If ever there's a user, they can build Envoy by themselves. And for Proxy-Wasm Rust SDK, it's only used in tests, so easy to justify.
- Even simply updating rules_rust is not impossible due to the usage of old APIs by Proxy-Wasm C++ host.
- Declare the another rules_rust with the non-patched latest version as
rules_rust_2or whatever different repo name. - Install gcc in the build containers and CI env - but this will break the existing dev environment - we need to inform everyone to install gcc which is obviously not good. Also, this will break for Windows for sure.
what do you think? I think 2. is the easiest one but to be honest, that could be the beginning of a mess. If it's ok, I would go for 1., but not sure either.
There was a problem hiding this comment.
see #35323 for discussion of the deps clusterfu
i think the solution is a fix in rules_rust (bazelbuild/rules_rust#2665)
cc @martijneken
There was a problem hiding this comment.
ok cool, more complex than I expected - I think for now we should go with as-is while waiting for the @phlax's PR got merged. I will migrate here to use crate_universe once it's done
There was a problem hiding this comment.
ok no luck - i will revert the commits trying to use crate_universe
There was a problem hiding this comment.
heads up - rules_rust is about to be updated
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
|
cc @marc-barry |
|
|
||
| Currently, the ABI binding ([src/abi.rs](./src/abi.rs)) is manually generated by [`bindgen`](https://github.com/rust-lang/rust-bindgen) for the ABI header. Ideally, we should be able to do the bindgen in the build.rs file. | ||
|
|
||
| TODO(@mathetake): figure out how to properly setup the bindgen in build.rs with rules_rust. The most recommended way is to use [crate_universe](https://bazelbuild.github.io/rules_rust/crate_universe.html#setup) and use bindgen as a dev-dependency. However, it seems that crate_universe tries to use the underlying gcc system linker which we cannot assume always available. |
There was a problem hiding this comment.
We use crate universe here with bazel: https://github.com/bitdriftlabs/capture-sdk. Not sure if there is anything to be learned from that.
Signed-off-by: Takeshi Yoneda <[email protected]>
repokitteh-read-only
bot
added
deps
|
CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to |
|
/retest |
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
LazyLock is stabilized in 1.80 (LazyOnce is in 1.79), but somehoe x64 CI fails with rust version upgrade Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
Signed-off-by: Takeshi Yoneda <[email protected]>
This reverts commit 6923b75. Signed-off-by: Takeshi Yoneda <[email protected]>
This reverts commit fff449c. Signed-off-by: Takeshi Yoneda <[email protected]>
|
added a bunch of commits, but at the end of the day, resulted in only a few lines of change to use atomic: 21053c5...ef12250 |
Commit Message: dynamic_modules: scaffolds Rust SDK
Additional Description:
This scaffolds the Rust SDK of dynamic modules. The same set of
unit tests is passing for Rust SDK like raw C test programs.
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]