-
Notifications
You must be signed in to change notification settings - Fork 438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cache for bazel to prove why bazel rocks #505
Conversation
Please do not press update branch because I want to do that as the second round to prove the benefit :) |
Codecov Report
@@ Coverage Diff @@
## master #505 +/- ##
==========================================
+ Coverage 94.36% 94.37% +0.01%
==========================================
Files 189 189
Lines 8410 8410
==========================================
+ Hits 7936 7937 +1
+ Misses 474 473 -1
|
0814e98
to
5ee3094
Compare
@bogdandrutu - What is the maximum amount of cached artifacts for the cache action - 5 GB? Do you know roughly how much space the Bazel build consumes? vcpkg build with all deps, unfortunately, consumes about 8GB on Windows (exceeds the maximum allowed cache size). |
What I am trying to say is once we start caching for all build loops (not just for Bazel), the cumulative net-benefit of that will be diminished. |
Hey, this gives us more time to run benchmarks! |
@maxgolov right now it is ~1GB per bazel job, but I think with this change we can actually combine multiple jobs and I think the cache will be lower. |
@bogdandrutu - 1GB is very good. I can apply similar logic for CMake + add |
As you mentioned gRPC installation with vcpkg could take much more than 5GB, also vcpkg build debug/release flavors at the same time, perhaps we could separate them into different builds. |
I'm almost thinking now we have to build and cache a docker image nightly? Or similar setup that'd preserve the prebuilt deps in GitHub Action cache. This should include a prebuilt of fresh CMake 3.1x (for older distros), Google Test, Google Framework, and gRPC. We should be able to build the image on Linux, cache the image ( e.g. using this approach https://github.com/marketplace/actions/docker-layer-caching ), then run For Windows / vcpkg: need to see what tools we can fetch, vs. what to cache using GitHub cache action. Huge size of vcpkg package install dir is an issue indeed.. |
@bogdandrutu - Sorry to ask, but would you be able to resolve conflict so that we can push it to master. |
Agree, would be good to test how docker layer caching perform here, and switch to it if we see benefit. @maxgolov has checked-in docker files so these can be reused. |
Signed-off-by: Bogdan Drutu <[email protected]>
Signed-off-by: Bogdan Drutu <[email protected]>
Signed-off-by: Bogdan Drutu <[email protected]>
01a82f4
to
3969d3a
Compare
Some initial numbers.
Before
After:
Signed-off-by: Bogdan Drutu [email protected]