[BUG] 0.46.0 Selects wrong Tokio version

[marvin-hansen]

This is something I've encounter while converting a WORKSPACE config to MODULE.

When you run the target with Cargo, it builds and runs:

cargo run --bin server

When you run the same target with Bazel, if fails with an error:

bazel run //crates/grpc_server:server

Error:

Running command line: target-bzl/bin/crates/grpc_server/server GreeterServer listening on [::1]:50051 thread 'main' panicked at external/rules_rust~~i~rules_rust_prost__hyper-0.14.26/src/server/tcp.rs:130:24: there is no reactor running, must be called from the context of a Tokio 1.x runtime note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

The error just says that the Tokio version mismatches for some dependencies, meaning some dependencies
expecting a version 1 whereas something else is there. The error is well explained on SO. However, I've declared version 1.38 as a dependency.

Versions and features of all deps are identical in Cargo and in Bazel.

Also, strangely enough, the same error occurs regardless of using a WORKSPACE or MODULE config.

Repo:
https://github.com/marvin-hansen/bazel_rust_example

[illicitonion]

Thanks for the super detailed repro!

I suspect the problem here is that the prost rules themselves have a dependency on tokio, and the version we give by default is probably different from the one you're using. You're pulling this in using the register_toolchains("@rules_rust//proto/prost:default_prost_toolchain") line in your MODULE.bazel.

Take a look at https://bazelbuild.github.io/rules_rust/rust_proto.html#customizing-prost-and-tonic-dependencies which specifies how you can supply your own prost toolchain. Basically, you need to supply each of those crates from your own crate_universe so that the versions get resolved together.

I haven't managed to get this successfully working yet, but I suspect if you can get into a place where your MODULE.bazel pulls in protoc-gen-prost, prost-types, protoc-gen-tonic, prost, and tonic, and can point at your version of them rather than rules_rust's, things will start working.

[marvin-hansen]

Alright,

thank you for the detailed insight and link to the setup.

As discussed on Slack, I've added a first draft of a custom toolchain Prost config in a separate branch.

https://github.com/marvin-hansen/bazel_rust_example/tree/rust-proto

Toolchain
MODULE

Still working on the proto compiler workaround; I add a comment later once that's solved.

Thank you for all your help and taking a look at the underlying issue.

[marvin-hansen]

Solved with a correct custom toolchain thanks to @illicitonion

https://github.com/marvin-hansen/bazel_rust_example/blob/main/build/prost_toolchain/BUILD.bazel

Updating the Bazelmod docs.

[zachcoleman]

Is it possible to use this toolchain with crate universe?

crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
crate.from_cargo(
    ...

When I do this, I get:

 no such target '@@rules_rust~~crate~cloud_crates//:protoc-gen-prost__protoc-gen-prost'

I don't know how to resolve using both a set of crates derived from Cargo files and having the necessary set files to run the custom toolchain that is normally build with crate.from_spec calls.

Note: I have verified they are in Cargo.lock, but I think this has to do w/ the annotation of generating binaries.

[marvin-hansen]

That's a known bug. You have to add the version number to proto gen prost and run build again. See this example. https://github.com/bazelbuild/examples/blob/main/rust-examples/08-grpc-client-server/build/prost_toolchain/BUILD.bazel Also, I recommend vendoring all deps as shown in the example linked above.

…

On Fri, Sep 13, 2024 at 08:11 Zachary Coleman ***@***.***> wrote: Is it possible to use this toolchain with crate universe? crate = ***@***.***_rust//crate_universe:extension.bzl", "crate") crate.from_cargo( ... When I do this, I get: no such target '@@rules_rust~~crate~cloud_crates//:protoc-gen-prost__protoc-gen-prost' I don't know how to resolve using both a set of crates derived from Cargo files and having the necessary set files to run the custom toolchain that is normally build with crate.from_spec calls. — Reply to this email directly, view it on GitHub <#2689 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFYR7XC7VSOZS7SDNCADVGLZWIUUFAVCNFSM6AAAAABJGDNXK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBXGQ2DANJSGA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[zachcoleman]

Unfortunately that is not resolving it for me:

Though I see them in the external section.

Is this a problem because they're not defined in Cargo.toml w/ the annotation to "gen_binaries" like other crate_universe options vs the from_cargo used here. I would like to consider vendoring in, but do wonder how locking works. Are you able to vendor in and get a locked set from a tool like Carog? In your example, the versions just seem to be picked.

[marvin-hansen]

Please share a reproduction of the issue as GitHub repo and I take a look over the weekend. I saw your slack message and explained a bit of background but for this issue I need a repo to see what can be done.

Thank you.

[zachcoleman]

Please share a reproduction of the issue as GitHub repo and I take a look over the weekend. I saw your slack message and explained a bit of background but for this issue I need a repo to see what can be done.

Thank you.

Here's a reproducible example. I have tried to keep some of the folder structure of my work stuff https://github.com/zachcoleman/bazel-rust-reproducible-prost
running:

bazel run //services/hello_world_grpc_rust/server

will reproduce the error where it can't find the protoc stuff for the prost toolchain
if you comment out the prost toolchain registration in the MODULE file:

# register_toolchains("@//build_tools/prost_toolchain")

You can see the compilation, but failure to bazel run in a tokio runtime