Skip to content

ci: Add azp cache for missing precheck jobs#17155

Merged
lizan merged 1 commit intoenvoyproxy:mainfrom
phlax:ci-precheck-no-cache
Jul 9, 2021
Merged

ci: Add azp cache for missing precheck jobs#17155
lizan merged 1 commit intoenvoyproxy:mainfrom
phlax:ci-precheck-no-cache

Conversation

@phlax
Copy link
Copy Markdown
Member

@phlax phlax commented Jun 25, 2021

Signed-off-by: Ryan Northey ryan@synca.io

Commit Message: ci: Remove azp cache for precheck jobs
Additional Description:

most of the precheck jobs dont use the cache, and im not sure what purpose it serves here

for the most part its quick, if unnecessary, but sometimes there can be contention on the cache and it can take a long time

Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

Copy link
Copy Markdown
Member

@lizan lizan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, docs and format need cache because they build and run protobuf compilers?

@lizan lizan self-assigned this Jun 27, 2021
@phlax
Copy link
Copy Markdown
Member Author

phlax commented Jun 27, 2021

im not sure if i understand what is going on with the cache - afaict it caches the WORKSPACE file and all of the .bzl files

maybe i misunderstand

i would probably remove it elsewhere also if that is the case, but here it just seems to slow down ci

@lizan
Copy link
Copy Markdown
Member

lizan commented Jun 28, 2021

@phlax Its cache key is WORKSPACE and .bzl files, and the cache content is the external repositories artifacts download by Bazel, if we don't do the cache it will download from GitHub every time and causes transient errors and take more time.

@phlax phlax changed the title ci: Remove azp cache for precheck jobs [WIP] ci: Remove azp cache for precheck jobs Jun 28, 2021
@phlax phlax marked this pull request as draft June 28, 2021 17:43
@phlax
Copy link
Copy Markdown
Member Author

phlax commented Jun 28, 2021

@lizan ill investigate this further - afaict it just slows these jobs down - also format_pre builds protos and doesnt have this set atm

@phlax phlax force-pushed the ci-precheck-no-cache branch from 4dc8812 to 1cabe76 Compare June 29, 2021 04:57
@phlax
Copy link
Copy Markdown
Member Author

phlax commented Jun 29, 2021

@lizan ill investigate this further - afaict it just slows these jobs down

on further investigation, i understand now that its caching the bazel repository_cache - ie external downloads - and that its using the hashes of the .bzl files etc as cache key

i think, on balance, it probably speeds up these jobs

not sure why, whether its contention or just service variability, but sometimes the azp cache can go very slow, but for the most part its quick

also format_pre builds protos and doesnt have this set atm

im going to try doing the opposite - adding the cache into the other jobs as they use bazel and, i think, download stuff as part of their builds

@phlax phlax force-pushed the ci-precheck-no-cache branch from d41ab79 to e38c607 Compare June 29, 2021 11:16
Signed-off-by: Ryan Northey <ryan@synca.io>
@phlax phlax force-pushed the ci-precheck-no-cache branch from 1e9ccb8 to 955f969 Compare June 29, 2021 11:35
@phlax phlax changed the title [WIP] ci: Remove azp cache for precheck jobs ci: Add azp cache for missing precheck jobs Jun 29, 2021
@phlax phlax marked this pull request as ready for review June 29, 2021 11:36
@phlax
Copy link
Copy Markdown
Member Author

phlax commented Jun 29, 2021

one thing that occurred to me @lizan is that we dont want these cache jobs to run on postsubmit, i think - as they are only run once

@phlax
Copy link
Copy Markdown
Member Author

phlax commented Jul 7, 2021

@lizan any chance of landing this ?

afaict any argument for keeping the cache on the other precheck jobs holds the same for the ones that do not currently have cache set up - they cache the same data

@lizan lizan merged commit f44ac0c into envoyproxy:main Jul 9, 2021
leyao-daily pushed a commit to leyao-daily/envoy that referenced this pull request Sep 30, 2021
Signed-off-by: Ryan Northey <ryan@synca.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants