Genrules in 6.0.0-pre.20220112.2 occasionally try to run shared libraries built by go_binary #14626
Comments
aiuto
added
team-Local-Exec
|
Is this only true for Go, or does it happen for other langauges? |
|
Using I'm pretty sure that what you are seeing is essentially a (much more concerning) variant of bazelbuild/rules_go#2999: The If I set @phst Could you open an issue in rules_go? I started to work on resolving these kinds of issues in bazelbuild/rules_go#3005, but at that point didn't notice that it could also have implications for correctness and not just performance. |
By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go.
|
@phst I just finished the work on bazelbuild/rules_go#3108, which should address this issue on a fundamental level. Could you take a look and verify that it fixes this issue (I did, but better to be sure)? |
By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go.
|
Maybe I'm doing something wrong, but with that PR I always get the error message
See e.g. https://github.com/phst/emacs/runs/6029654171 |
|
I've now filed bazelbuild/rules_go#3120 so that this is properly tracked in rules_go. |
I think that the build is failing because you are using the first (bazelbuild/rules_go@9fc6d45) and not the last (bazelbuild/rules_go@efd118d) commit of bazelbuild/rules_go#3108. Could you give it another try with the last commit? |
|
Ah, good catch, I messed up the order of the commits. |
|
@phst https://github.com/phst/emacs/runs/6030260629?check_suite_focus=true is an interesting failure: It looks like my PR ends up reenabling nogo for a Go tool dependency, which is certainly not intended. I will look into this. |
|
This might be another instance of bazelbuild/rules_go#2513, though I'm wondering why Bazel 5 isn't affected. |
|
The dependency chain from The last config does not have the All in all, I think that this is a separate issue and not caused (but also not prevented) by my changes. It would be very helpful if build settings had a @gregestren Have there been any considerations to allow Starlark build settings to provide defaults for the exec configuration? In Java, fragments have |
|
Yes, but that's in pre-design: we know we'd like a better APi than hard-coded |
By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go.
By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go.
By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go.
* Trim transitioned Go settings on non-Go dependencies By resetting Go-specific settings changed by go_transition to their previous values when crossing a non-deps dependency edge, e.g. srcs or data, dependencies through those edges are not affected by the change in Go settings. This has two advantages: * Correctness: Previously, if a go_binary with linkmode = "c-archive" used another go_binary to generate srcs, that go_binary would also be built as a c-archive and thus fail to run during the build. This was observed in bazelbuild/bazel#14626. * Performance: With the new Bazel flag --experimental_output_directory_naming_scheme=diff_against_baseline, transitions can return to the top-level configuration since the Starlark settings that were affected by a transition at some point during the build are no longer tracked explicitly in a setting, but computed as a configuration diff. Resetting the configuration for non- deps dependencies thus prevents redundant rebuilds of non-Go deps caused by changes in Go settings, achieving a version of "configuration trimming" for Go. * Verify transition hermeticity for go_{binary,library,test} The test verifies that Go settings such as gotags and linkmode do not affect dependencies through attributes other than deps.
|
bazelbuild/rules_go#3120 has been fixed, so this issue can be closed. |
Description of the problem / feature request:
It looks like starting with the newest rolling release (6.0.0-pre.20220112.2) genrules occasionally try to execute shared libraries instead of binaries (which obviously fails).
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
Apologies for not providing a more minimal example, but it seems that this issue is a bit subtle. It should be reproducible like this:
USE_BAZEL_VERSION=6.0.0-pre.20220112.2 bazelisk build -j 1 -s //...Note how Bazel for the second action attempts to execute a shared library (libgenheader.so), which segfaults.
This doesn't happen with the previous rolling release (try
USE_BAZEL_VERSION=6.0.0-pre.20220105.5 bazelisk build -j 1 -s //...). My guess is that it's related to "genrule switched to use exec transition instead of host" in https://github.com/bazelbuild/bazel/blob/master/CHANGELOG.md#release-600-pre202201122-2022-01-19. Changing$(location)to$(execpath)in the genrule command doesn't fix the issue.Maybe this is also an issue with rules_go, but I didn't find anything obviously wrong when checking the implementation of go_binary.
What operating system are you running Bazel on?
GNU/Linux, recent Debian testing
What's the output of
bazel info release?What's the output of
git remote get-url origin ; git rev-parse master ; git rev-parse HEAD?Have you found anything relevant by searching the web?
No
The text was updated successfully, but these errors were encountered: