Skip to content

ci: s3 cache - #1167

Merged
shumkov merged 82 commits into
v0.25-devfrom
ci/s3_cache
Jul 11, 2023
Merged

ci: s3 cache#1167
shumkov merged 82 commits into
v0.25-devfrom
ci/s3_cache

Conversation

@shumkov

@shumkov shumkov commented Jun 15, 2023

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Self-hosted runners are expensive so we need to utilize cache as much as possible to reduce costs.
This PR is using s3 store as a cache backend for NPM, Cargo, Docker build and Rust build caches in CI

There is some work needed to support ARM builds in a proper way

What was done?

  • Disabled sccache by default
  • Enabled docker mount cache
  • Enabled docker layer cache
  • Use independent envs to setup caching for each service in Dashmate
  • Utilized s3 for the docker layer and mount caches in CI

This PR contains prelumeary work from #1105 and #953

How Has This Been Tested?

Running CI and locally

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@shumkov
shumkov requested a review from antouhou as a code owner June 15, 2023 03:28
@shumkov shumkov changed the title ci: s3 cache [WIP] ci: s3 cache Jun 15, 2023
@shumkov shumkov added this to the v0.24.x milestone Jun 23, 2023
@shumkov
shumkov requested a review from QuantumExplorer as a code owner June 23, 2023 11:13
@shumkov shumkov changed the title [WIP] ci: s3 cache ci: s3 cache Jul 5, 2023

@strophy strophy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good! Let's make a few more changes for readability

Comment thread .github/actions/docker/action.yaml Outdated
Comment thread .github/workflows/all-packages.yml Outdated
Comment thread .github/workflows/all-packages.yml Outdated
Comment thread .github/workflows/js-checks.yml Outdated
Comment thread .github/workflows/rs-drive-abci.yml Outdated

@strophy strophy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@shumkov
shumkov merged commit c435a3e into v0.25-dev Jul 11, 2023
@shumkov
shumkov deleted the ci/s3_cache branch July 11, 2023 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants