-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migration of Automatic Exec Groups [Exec platform per toolchain] in Bazel downstream #19981
Comments
I've read the above, and the referenced doc, but the answer didn't jump out at me. Apologies if I misunderstood the proposal. Question: What should we do for actions that need more than one toolchain? Iiuc, currently we would list both toolchains on the rule(toolchains) attribute, toolchain resolution would take them all into account when choosing an exec platform, and then (aiui) the same exec platform is used for all actions in that rule. |
Duh, found the answer right after posting 😮💨. The new bazel docs explain the answer clearly:
@kotlaja: Iiuc, I think your rules_go PR above does not follow this advice - and should. |
@kotlaja @comius I tried to enable The root cause is that https://github.com/protocolbuffers/protobuf/blob/6690ab42d855ea19d9a24cd99b0375910ea772ca/bazel/private/toolchain_helpers.bzl#L48 and https://github.com/bazelbuild/rules_proto/blob/db09d58959ed951db34c2e056cef084954e55640/proto/private/rules/proto_toolchain_rule.bzl#L23 (depending on where the proto rules are consumed from) are strings, not
We would need to get new releases of |
Was going to update rules_dotnet to support |
FYI,
Let us know when they are ready |
Thanks folks for raising these issues. Let's then wait for |
FYI @kotlaja I ran a compatibility test with the top 100 BCR modules for https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/45 |
@meteorcloudy thanks for gathering that information! I talked to @fmeum and it seems that protocolbuffers/protobuf@e358a40 fixed the issue Fabian noted in this thread. Still, multiple projects are blocked by rules_python, and rules_python is blocked on protobuf. Hence, I opened an issue for protobuf and tried migrating on my own (protocolbuffers/protobuf#19262) in order to unblock other projects. I fixed a few actions and then I came to an obstacle with rules_kotlin, so I'm waiting for protobuf team to let me know their thoughts (if they'd like to try upgrading rules_kotlin since some troubling files changed in the meantime) |
Goals of this document
Goal of this migration
Each toolchain type should have its own execution platform. Before this change, the execution platform was selected on a rule level, not on a toolchain type level.
Each action which uses a tool or executable coming from a toolchain should specify the toolchain type inside ctx.action.{run, run_shell}
toolchain
attribute.What is needed for this migration
In order to use Automatic Exec Groups, two things are required:
Each
ctx.actions.{run, run_shell}
which uses a toolchain as a tool/executable needs to add its toolchain type inside the toolchain parameter. In this way, actions are sent to the right platform.Moreover, rule which creates that action needs to define that toolchain_type inside its own rule definition. In other words, the toolchain must be registered on the rule.
For example:
Why does a rule creator need to take care of a toolchain type?
If an action’s executable is not coming from a toolchain,
toolchain
parameter doesn’t need to be set or it could be set to None.If an action’s executable is coming from a toolchain,
toolchain
parameter should be set. The reason is simple: A correct execution platform will be selected for the action since it will be mapped with this toolchain type.Why must toolchain_type be added manually and not reused from a tool coming from the toolchain?
The tool is a file and we don’t know which toolchain_type it refers to.
Or, the tool is a provider, inside which we could add a toolchain_type but that would increase overall memory.
Which Bazel versions support this migration?
AEGs are fully supported from Bazel 7.
toolchain
parameter is supported from Bazel 6.If your project supports lower Bazel versions, you need to disable AEGs.
How to disable AEGs in case of an error?
Set the incompatible_auto_exec_groups flag to false or disable a particular rule with the
_use_auto_exec_groups
attribute.
Potential errors you could step on while migrating to AEGs
Couldn't identify if tools are from implicit dependencies or a toolchain. Please set the toolchain parameter. If you're not using a toolchain, set it to 'None'.
None
.Action declared for non-existent toolchain '//rule:toolchain_type_1'.
None
inside the action.The text was updated successfully, but these errors were encountered: