-
-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Configure remote builds to use EngFlow #11887
Conversation
This allows us to use EngFlow's remote build grid to execute our builds. Once we hook this up into our CI actions, this will lead to significantly faster CI runs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see there is a whole new common/remote-build
directory with many autogenerated files. Is there any way to shrink those? For example, I assumed we don't have to manually configure C++ toolchains because I thought it's built-in to Bazel.
The files are auto-generated by (effectively, if not in reality) running bazel on the remote machine and then copying the generated files into the repo. It's not a great idea to modify them. We don't have a hermetic C++ toolchain, so we still need to manage that ourselves. Our remote build images have a C++ toolchain installed, so we need to tell Bazel about it. Because the build is remote, we can't figure this stuff out at run time (since we need to know everything before sending the actions to the remote machine), and this is the way to configure it. |
Got it. I think as long as it's documented how to re-generate these files should we need - we are fine.
Ok, so Bazel doesn't ship with a hermetic C++ toolchain. Would using something like https://github.com/uber/hermetic_cc_toolchain help us? Probably not now, but eventually. |
We just ran https://github.com/bazelbuild/bazel-toolchains
I spent an ungodly amount of time attempting to get the zig-cc toolchain working and gave up. If all we wanted to do was support Linux, it'd be easier, but we want to support the matrix of compiling for and on Windows, Mac OS, and Linux. For that we need various sysroots, and the legality of distributing these is unknown. That makes it a really difficult problem to solve. It's also not something that we can solve in this PR :) |
Codecov ReportPatch coverage:
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more Additional details and impacted files@@ Coverage Diff @@
## trunk #11887 +/- ##
==========================================
+ Coverage 54.87% 54.89% +0.01%
==========================================
Files 85 86 +1
Lines 5698 5733 +35
Branches 230 233 +3
==========================================
+ Hits 3127 3147 +20
- Misses 2341 2353 +12
- Partials 230 233 +3
☔ View full report in Codecov by Sentry. |
This allows us to use EngFlow's remote build grid to execute our builds. Once we hook this up into our CI actions, this will lead to significantly faster CI runs.