Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#17545 might have broken bazel_tools in downstream pipelines #17874

Closed
fweikert opened this issue Mar 23, 2023 · 10 comments
Closed

#17545 might have broken bazel_tools in downstream pipelines #17874

fweikert opened this issue Mar 23, 2023 · 10 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Rules-Python Native rules for Python type: bug

Comments

@fweikert
Copy link
Member

Description of the bug:

The downstream pipeline (https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2921) has several failures due to files in @bazel_tools failing to load @rules_python//python:proto.bzl:

The problems started in #2895, which coincides with #17545

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@ShreeM01 ShreeM01 added the team-Rules-Python Native rules for Python label Mar 23, 2023
@comius comius added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Mar 27, 2023
@comius
Copy link
Contributor

comius commented Mar 27, 2023

The issue looks related to non-bzlmod usage and a requirement to load rules_python in some cases. Possible fix is to update WORKSPACE suffixes to load rules_python.

comius added a commit to comius/rules_kotlin that referenced this issue Apr 17, 2023
This is needed to correctly load py_proto_library from @bazel tools.

Addresses: bazelbuild/bazel#17874
comius added a commit to comius/rules_python that referenced this issue Apr 17, 2023
comius added a commit to comius/rules_oss_audit that referenced this issue Apr 17, 2023
This fixes a failure in downstream, that needs a newer version of rules_python, to support py_proto_library.

Addresses: bazelbuild/bazel#17874
@comius
Copy link
Contributor

comius commented Apr 17, 2023

The problem is created because of an older rules_python imported through WORKSPACE in various project. I produced PRs to upgrade the version.

restingbull pushed a commit to bazelbuild/rules_kotlin that referenced this issue Apr 17, 2023
This is needed to correctly load py_proto_library from @bazel tools.

Addresses: bazelbuild/bazel#17874
@meteorcloudy
Copy link
Member

@comius
Copy link
Contributor

comius commented Apr 21, 2023

maybe more fixes are needed?

rules_kotlin are still failing because they get an old version of rules_python from rules_proto... it's no-bzlmod dependency hell. Any advice how to fix this, without adding more mess into the system?

@meteorcloudy
Copy link
Member

Oh, do you mean the upgrade in https://github.com/bazelbuild/rules_kotlin/pull/958/files#diff-def3077745bf807244580cf012e9d27574b9f337415a6734a186572b424c3badR103 didn't actually take affect because rules_proto introduces an older python version in WORKSPACE first? If so, rules_kotlin needs to figure out a way to prefer the dependencies they declared.

@werkt
Copy link
Contributor

werkt commented Apr 22, 2023

I'm late to this, but for buildfarm's front, it looks like we're not able to depend upon bazel_tools as a result of this change without drastic changes that incorporate bzlmod (which I'm just learning about) or specifying both rules_python and rules_proto, both of which we did without before this.

Am I missing something? Is there supposed to be an easier migration for us not using an experimental system?

@werkt
Copy link
Contributor

werkt commented Apr 22, 2023

Some followup on this. I got local builds to work by importing your indicated rules_python versions, but those versions break python3.6 compatibility indicated bazelbuild/rules_python#818, and this is the python version available on the support-indicated ubuntu1804 version matching bazel CI validation. Attempting to use a supported version with the hermetic toolchain does not resolve the problem, and fails on our integration tests, which run as root under CI containers.

This is a pretty big headache for the invalidation of a previously working toolset under bazel_tools, which only indicates that it needs rules_python 0.4.0 under its MODULE, in disagreement with the requirement for @rules_python//python:proto.bzl, nonexistent at 0.4.0.

werkt added a commit to werkt/bazel-buildfarm that referenced this issue Apr 22, 2023
@bazel_tools//src/main/protocol: has become untraversable without a
compatible rules_python for targets that we do not use. Maintain
compatibility with bazel HEAD CI by removing dependencies on the
package.

References bazelbuild/bazel#17874
werkt added a commit to werkt/bazel-buildfarm that referenced this issue Apr 22, 2023
@bazel_tools//src/main/protocol: has become untraversable without a
compatible rules_python for targets that we do not use. Maintain
compatibility with bazel HEAD CI by removing dependencies on the
package.

References bazelbuild/bazel#17874
werkt added a commit to buildfarm/buildfarm that referenced this issue Apr 22, 2023
@bazel_tools//src/main/protocol: has become untraversable without a
compatible rules_python for targets that we do not use. Maintain
compatibility with bazel HEAD CI by removing dependencies on the
package.

References bazelbuild/bazel#17874
@werkt
Copy link
Contributor

werkt commented Apr 22, 2023

I've removed the reference to @bazel_tools//src/main/protobuf. We will not be using it going forward due to this break and the forced obsolescence/explicit rules_python dependency for a target+language we are not using. Buildfarm should be passing in the disabled stream.

@meteorcloudy
Copy link
Member

meteorcloudy commented Apr 24, 2023

@werkt Thanks for fixing it! In general, I think it's a good thing to not depend on protobuf stuff through @bazel_tools.

I can confirm Buildfarm is now green with Bazel@HEAD, see https://buildkite.com/bazel/bazel-at-head-plus-disabled/builds/1646, re-enabling it now.

meteorcloudy added a commit to bazelbuild/continuous-integration that referenced this issue Apr 24, 2023
meteorcloudy added a commit to bazelbuild/continuous-integration that referenced this issue Apr 24, 2023
rickeylev pushed a commit to bazelbuild/rules_python that referenced this issue Apr 24, 2023
…1173)

The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.

Addresses: bazelbuild/bazel#17874
danielmachlab pushed a commit to vmware-archive/rules_oss_audit that referenced this issue Apr 24, 2023
This fixes a failure in downstream, that needs a newer version of rules_python, to support py_proto_library.

Addresses: bazelbuild/bazel#17874
@meteorcloudy
Copy link
Member

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2995
I can confirm this issue has been fixed in rules_kotlin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Rules-Python Native rules for Python type: bug
Projects
None yet
Development

No branches or pull requests

6 participants