Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rust_tonic_compile rule #391

Merged
merged 1 commit into from
Nov 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@ workspace(name = "vaticle_dependencies")
# Load @vaticle_dependencies #
################################

# Load //build/rust
# Load //builder/rust
load("//builder/rust:deps.bzl", rust_deps = "deps")
rust_deps()
load("@rules_rust//rust:repositories.bzl", "rust_repositories")
rust_repositories(version = "nightly", iso_date = "2021-07-01", edition="2018")

load("@rules_rust//rust:repositories.bzl", "rules_rust_dependencies", "rust_register_toolchains")
rules_rust_dependencies()
rust_register_toolchains(edition = "2021", include_rustc_srcs = True)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to the latest Rust (fixes build issue in ortools, and also is just generally good practice)


# Load //builder/python
load("//builder/python:deps.bzl", python_deps = "deps")
Expand Down
33 changes: 33 additions & 0 deletions builder/grpc/rust/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package contains the rust_tonic_compile rule and the rust_binary that powers it.

Ideally, we'd get this rule from an external repo such as rules_rust or rules_proto_grpc, but they don't have tonic implementations yet.

We may want to relocate this rule to another repo so it can be used by other Rust proto repos. We may also want to ask the rules_proto_grpc maintainers if they'd accept it as a PR. They do have an existing open PR that adds rust_tonic_compile (and more), so we'd start by commenting there.

# Copyright (C) 2022 Vaticle
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

load("@rules_rust//rust:defs.bzl", "rust_binary")
load("@vaticle_dependencies//tool/checkstyle:rules.bzl", "checkstyle_test")

rust_binary(
name = "compile",
srcs = ["compile.rs"],
deps = ["@vaticle_dependencies//library/crates:tonic_build"],
visibility = ["//visibility:public"]
)

checkstyle_test(
name = "checkstyle",
include = glob(["*"]),
license_type = "agpl-header",
size = "small",
)
62 changes: 62 additions & 0 deletions builder/grpc/rust/compile.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rule implementation is quite straightforward:

  1. Define the inputs for the compile script, namely all .proto sources and the protoc executable.
  2. Define the outputs of the compile script. One output is generated for each unique package defined in our .proto files. Bazel can't (easily!) get these package names, so the user is required to supply them as a parameter.
  3. Run the compile script and expose the outputs as source files that any rust_library can depend on.

(The rust_library is then expected to depend on the tonic and prost crates, otherwise it will fail to compile. This is in line with Java, where we expect our java_library to depend on the correct gRPC + Protobuf JARs.)

# Copyright (C) 2022 Vaticle
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

def _rust_tonic_compile_impl(ctx):
protos = [src[ProtoInfo].direct_sources[0] for src in ctx.attr.srcs]

inputs = ctx.attr.protoc.files.to_list() + protos
outputs = [ctx.actions.declare_file("{}.rs".format(package)) for package in ctx.attr.packages]

ctx.actions.run(
inputs = inputs,
outputs = outputs,
executable = ctx.executable._compile_script,
env = {
"OUT_DIR": outputs[0].dirname,
"PROTOC": ctx.attr.protoc.files.to_list()[0].path,
"PROTOS": ";".join([src.path for src in protos]),
"PROTOS_ROOT": ctx.attr.srcs[0][ProtoInfo].proto_source_root,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OUT_DIR and PROTOC are both used internally by the tonic_build crate. So, for convenience, we use environment variables to represent the other necessary input arguments as well.

},
mnemonic = "RustTonicCompileAction"
)

return [DefaultInfo(files = depset(outputs))]

rust_tonic_compile = rule(
implementation = _rust_tonic_compile_impl,
attrs = {
"srcs": attr.label_list(
allow_files = True,
mandatory = True,
doc = "The .proto source files."
),
"packages": attr.string_list(
mandatory = True,
allow_empty = False,
doc = "The Protobuf package names. Each package name corresponds to a single output file."
),
"protoc": attr.label(
default = "@com_google_protobuf//:protoc",
doc = "The protoc executable."
),
"_compile_script": attr.label(
executable = True,
cfg = "host",
default = "@vaticle_dependencies//builder/grpc/rust:compile",
),
}
)
28 changes: 28 additions & 0 deletions builder/grpc/rust/compile.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
//
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We pass in all the tonic_build configuration via environment variables, which are set by the rust_tonic_compile rule.

// Copyright (C) 2022 Vaticle
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as
// published by the Free Software Foundation, either version 3 of the
// License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//

use std::env;

fn main() -> std::io::Result<()> {
let protos_raw = env::var("PROTOS").expect("PROTOS environment variable is not set");
let protos: Vec<&str> = protos_raw.split(";").filter(|&str| !str.is_empty()).collect();

tonic_build::configure()
.compile(&protos, &[
env::var("PROTOS_ROOT").expect("PROTOS_ROOT environment variable is not set")
])
}