Repository navigation
[WIP] ci: testing local/s3 backend for cache #953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 19 commits
Commits
Show all changes
315 commits
Select commit
Hold shift + click to select a range
97a6f36
fix: SyntaxError: Unexpected number
strophy afc4760
ci: add s3 layer cache backend and test cache mount
strophy bef52ba
chore: bump wasm-bindgen-cli version
strophy db69397
chore: debug ci
strophy a243789
build: bulild javascript inside docker container
lklimek f5ced00
build(Docker): remove drive-abci env vars from Dockerfie
lklimek 06fff00
chore(rs-drive-abci): rustfmt
lklimek 53088e4
Merge branch 'feat/ci-cd-caching' into feat/faster-ci
strophy 09d3d35
ci: implement cache mount action
strophy dccd4a0
chore: try to use s5cmd to sync cache mount on s3
strophy 4c0015f
ci: install s5cmd manually
strophy 6123956
ci: install s5cmd as root
strophy a6b26a9
build: unified Dockerfile for all services
lklimek 850bf20
doc: Describe Dockerfile caching
lklimek 563c6d8
build(github): use new unified Dockerifle
lklimek 85615e8
build(dashmate): set SCCACHE build args
lklimek 1436764
build(github): set drive-abci docker timeout to 3h
lklimek 2a80c2b
build: rename CARGO_BUILD_PROFILE `debug` with `dev`
lklimek 8afbb21
build(github): build drive-abci docker on self-hosted runner
lklimek e6af398
build(github): rs-drive-abci uses s3 cache
lklimek 5e58995
build(Dockerfile): run node.js as non-privileged user
lklimek 0d5941d
build(Docker): only delete rust sources starting with rs-
lklimek fd08417
build(dashmate): entrypoint: create user only when run as root
lklimek c30de7e
feat(drive-abci): add drive-abci status command (mock for now)
lklimek 6805a30
chore: minor cleanup of Dockerfile
lklimek a7b7f8b
build: bulild javascript inside docker container
lklimek 89e3330
build(Docker): remove drive-abci env vars from Dockerfie
lklimek 9131e8f
chore(rs-drive-abci): rustfmt
lklimek 081d929
build(github): docker cache mode max
lklimek 353daa7
build(wasm-dpp): use --profile instead of --release
lklimek bbcf8d4
chore(docker): self-review
lklimek 54c7b8c
build(wasm-dpp): autodetect wasm-bindgen-cli version
lklimek de2053f
Merge branch 'feat/ci-cd-caching' into feat/faster-ci
strophy 8e17012
debug: ci after s5cmd
strophy 6f2cb9c
chore: disable gha cache
strophy de2d8b5
chore: bust cache
strophy d7ff5bb
build: unified Dockerfile for all services
lklimek fe6873c
doc: Describe Dockerfile caching
lklimek 24a6ca8
build(github): use new unified Dockerifle
lklimek a0a9cea
build(dashmate): set SCCACHE build args
lklimek 9af4b43
build(github): set drive-abci docker timeout to 3h
lklimek 15ce460
build: rename CARGO_BUILD_PROFILE `debug` with `dev`
lklimek 232297c
build(github): build drive-abci docker on self-hosted runner
lklimek ed7fd70
build(github): rs-drive-abci uses s3 cache
lklimek 4a22fe7
build(Dockerfile): run node.js as non-privileged user
lklimek 7a34aa2
build(Docker): only delete rust sources starting with rs-
lklimek 59f815a
build(dashmate): entrypoint: create user only when run as root
lklimek e77e930
ci: build amd64 only
strophy 29fa1e7
chore: show value of CARGO_HOME
strophy bb477c2
chore: minor cleanup of Dockerfile
lklimek 7dd2f14
build: bulild javascript inside docker container
lklimek 85e83d9
build(Docker): remove drive-abci env vars from Dockerfie
lklimek b6e1e00
build(github): docker cache mode max
lklimek a5e9dd2
build(wasm-dpp): use --profile instead of --release
lklimek ad438cc
chore(docker): self-review
lklimek bc8c283
build(wasm-dpp): autodetect wasm-bindgen-cli version
lklimek 9b6f8e6
build(github): list all targets available in Dockerfile
lklimek 15f17ab
chore: peer review feedback
lklimek 29f0bf1
revert: downgrade bingden to create separate PR
lklimek 19c3368
Revert "build(wasm-dpp): autodetect wasm-bindgen-cli version"
lklimek d3ed462
chore: peer review
lklimek 377b44b
chore: switch to custom action
strophy 98a1253
chore: mount absolute paths
strophy 6eef7e7
fix: typo
strophy 421473a
chore: bust cache
strophy 7c913eb
build(dashmate): optimize list of packages transferred to the final i…
lklimek 293645c
build(dapi): optimize packages transferred to final image
lklimek a01b00f
chore(Dockerfile): remove commented code
lklimek 93e1d33
chore(Dockerfile): remove outdated doc comment
lklimek afa7466
chore: check if enabling rust cache speeds up yarn build
strophy e5c66f2
chore: remove s3 autocache for now
strophy bb6a05f
chore: build deps only
strophy 1dc790c
chore: inspect cache mounts
strophy 1b1c2c0
chore: inspect cache after build
strophy c695835
ci: try to bust cache
strophy 5b17389
ci: re-enable cache
strophy ff12c34
chore: bust layer cache
strophy 7ec3448
chore: enable tmate to inspect build environment
strophy 2f5d64a
chore: debug username
strophy b66b2d4
chore: try to bust cache again
strophy 347d013
chore: bust layer cache
strophy 065fa87
chore: bust layer cache
strophy 0b8db61
doc: add brew install protobuf to dependency docs
lklimek 74c8839
chore(dashmate): remove unused SCCACHE_GHA_ENABLED code
lklimek f762983
build(test-suite): copy only required files
lklimek 09e4898
Merge remote-tracking branch 'origin/v0.25-dev' into feat/ci-cd-caching
lklimek 2f74694
Merge branch 'feat/ci-cd-caching' into feat/faster-ci
strophy 5c8f9fa
chore: manually fix outstanding merge issues
strophy 0b7b666
fix: action sometimes fails with curl exit code 35
strophy 97a6669
fix: build sometimes fails with curl exit code 35
strophy d0cbda2
chore: bust cache
strophy c7222e9
chore: add cache mount ids
strophy 02a3ee0
chore: bust cache
strophy f340d5a
Merge branch 'v0.25-dev' into feat/faster-ci
strophy 06a27f1
chore: bust cache
strophy 3b13e1f
chore: show debug for cargo install
strophy 65462c4
chore: bust cache
strophy 5fda034
chore: bust cache
strophy 3e77787
chore: bust cache
strophy a1e9997
chore: bust cache
strophy 6926094
chore: bust cache
strophy 8c91e58
chore: bust cache
strophy e67ef2b
chore: disable debug output
strophy 2906fbd
chore: bust cache
strophy cb1c929
chore: bust cache
strophy f496f5d
chore: try to build drive
strophy 6894585
Merge branch 'v0.25-dev' into feat/faster-ci
strophy 0f17932
chore: try deps-only build again
strophy bf3755a
chore: try to build two stages without cache mount ids
strophy e6e3f92
chore: bust cache
strophy ee25de3
chore: build drive-abci
strophy b85616a
chore: bust cache
strophy a4419bf
chore: restore mount cache id and bust cache
strophy 2cb99b0
chore: bust cache
strophy e8f6055
ci: disable tmate
strophy af4e1f8
Merge branch 'v0.25-dev' into feat/faster-ci
strophy 26f1fbd
chore: bust cache
strophy 4707a5a
chore: bust cache
strophy 8a0f2b9
chore: try locked cache mounts
strophy 1bf2519
chore: bust second cache only
strophy fdb52ca
build(dapi-grpc): commit generated rust code to avoid ci/cd issues
lklimek a161ac9
chore: single cache mount across stages
strophy 5bf1422
chore: bust cache
strophy 3999832
chore: remove debug output
strophy e3c4e7b
chore: separate namespace for target mounts
strophy 8f01a13
Merge branch 'v0.25-dev' into feat/faster-ci
strophy 509232b
chore: bust cache
strophy 49f20ea
chore: bust drive cache only
strophy 65c4312
chore: show debug
strophy e9a7751
chore: restore shared cache
strophy 6340662
chore: cache in max mode
strophy 547a1ed
chore: enable rust cache fingerprint trace
strophy e130b65
chore: don't show platform contents
strophy 350918d
chore: bust cache
strophy 3482518
chore: disable s5cmd installation on runner
strophy d203d5e
chore: disable s5cmd push
strophy 714503c
chore: bust cache
strophy b4acc2c
chore: bust cache
strophy 900ce09
chore: bust cache
strophy 9069138
chore: access each file before cargo accesses it
strophy 074d375
chore: sleep before build
strophy be87566
chore: try locked cache
strophy 0485cdf
chore: more cache busting and build cleanup
strophy a488d67
chore: enable yarn cache
strophy 7bc2ec0
chore: inspect timestamps before build
strophy e0e6603
chore: bust cache
strophy df12495
chore: separate cache layers for registry
strophy f084ff8
chore: bust cache
strophy d363f80
chore: try again with db in separate dir
strophy 2367b14
chore: try to touch all cache files to make them new
strophy 07c804e
chore: escape command correctly
strophy 34ebc18
fix: find syntax
strophy fbfe93a
chore: make cache older, not newer
strophy 8a8e368
chore: move touch command to action
strophy dde90f9
chore: bust cache
strophy 47cfb55
chore: bust cache
strophy 0bb31c1
chore: remove debug
strophy 738880e
chore: bust cache
strophy d14147e
Revert "chore: remove debug"
strophy effe9a3
chore: bust cache
strophy e85f58e
chore: remove debug
strophy d3c0b42
chore: try shared cache
strophy 9f2696e
chore: try to find drive_target
strophy c75baba
chore: try to debug how docker sees the cache
strophy 27c8eea
chore: install tree
strophy 4d40915
chore: bust cache
strophy 33f09f4
chore: inspect platform dir without cache mount
strophy 31f273f
chore: inspect dir properly
strophy b3b5611
chore: debug target dir again
strophy e78f8fb
chore: configure docker gc
strophy 0498a17
chore: increase cache keep duration to match s3
strophy 523a7e2
chore: bust cache
strophy 88ee38d
chore: remove debug
strophy 79ac634
Merge branch 'v0.25-dev' into feat/faster-ci
strophy 5749e52
chore: single platform dir
strophy 916d7ca
chore: bust cache
strophy 3387c56
chore: bust wasm-bindgen only
strophy 5d1593d
chore: restore fingerprint debug
strophy cc4720d
chore: disable git cli
strophy 095c357
fix: syntax
strophy 07139c8
chore: bust drive cache
strophy fd2be3f
chore: bust deps only
strophy f3023a3
chore: try dynamic linking
strophy f65c732
fix: syntax
strophy 77184cf
fix: bad syntax
strophy ddc78ac
chore: bust wasm
strophy 651ca0d
chore: bust drive
strophy b9cec9c
chore: bust both
strophy 58729a0
chore: bust drive
strophy 7b06d40
chore: bust wasm
strophy 60a5a8e
chore: bust both
strophy 71d4e54
chore: remove debug
strophy 75c18c4
chore: try to cache dapi build and unplugged dir
strophy 60f79fa
chore: enable dapi build
strophy ff95780
fix: tag input missing
strophy 328ac7c
chore: rename unplugged cache mount
strophy e45f879
chore: try to release all docker stages
strophy b966757
chore: disable arm builds
strophy 01ed638
chore: remove cache mount restore from envoy build
strophy ef4dcda
chore: disable s3 cache for envoy
strophy 24649f2
chore: disable envoy build
strophy d20b519
chore: debug dapi build on fail
strophy 332e468
Merge branch 'v0.25-dev' into feat/faster-ci
strophy a15be58
chore: check if updating ssh2 helps
strophy 69d62ff
chore: disable tmate
strophy d23760d
Revert "chore: disable tmate"
strophy 2636551
Revert "chore: check if updating ssh2 helps"
strophy 447ee44
chore: cache .yarn/cache dir
strophy a93dcfc
chore: disable tmate
strophy 8de3dee
Revert "chore: cache .yarn/cache dir"
strophy f412fc5
chore: disable test suite build
strophy 0e6af2b
chore: debug unplugged dir
strophy e8ee4df
chore: try to lock unplugged mount
strophy beaa4cd
chore: try to use yarn 3.6.0
strophy 031489e
Merge branch 'v0.25-dev' into feat/faster-ci
strophy cc667bc
build: provide `node-gyp` deps for ssh2 build
shumkov 62d895d
Merge remote-tracking branch 'origin/feat/faster-ci' into feat/faster-ci
shumkov 001e007
build: fix cache mount for yarn unplugged
shumkov b426c0e
chore: include install-state.gz in cache mount
strophy b3578d3
Merge branch 'v0.25-dev' into feat/faster-ci
strophy fbbcd57
chore: install wasm-bindgen generated artifacts and intermediate file…
strophy 632637d
chore: try to inspect and use $workspace_root
strophy b8f36ed
chore: replace tree with ls command
strophy 6cea887
chore: disable drive build
strophy 880f072
fix: syntax
strophy 2332925
chore: remove broken ls commands
strophy 0d5e72a
chore: bump deps in base image
strophy c18d244
chore: remove debug
strophy 62218e9
chore: remove target-dir
strophy db2cf55
chore: revert to yarn 3.3.0
strophy 8b3a929
chore: revert retry
strophy da006e6
chore: revert npm
strophy 5c6bb3c
fix: unnecessary newline
strophy af5c6f5
chore: remove sccache from release workflows
strophy 4f80c19
chore: remove debug
strophy 0420ed8
chore: enable layer cache for dashmate-helper
strophy a02767b
chore: enable test suite and drive builds
strophy a654b5f
chore: disable non-dapi runs
strophy 2bce90b
chore: try s3 cache backend
strophy 04935fd
fix: A mapping was not expected
strophy fd23027
chore: try manual s3 command
strophy f148fc4
chore: handle install-state.gz
strophy 33b5a3d
chore: remove old stuff
strophy 58b1a6e
chore: cache cargo stuff in non-docker build
strophy d053084
chore: debug
strophy f1cb309
chore: remove debug
strophy aeb8dc8
chore: use absolute dirs because CARGO_HOME is not set
strophy e90c8b4
chore: move cache shuffle after yarn build
strophy 58b7c27
chore: try opendal cache only
strophy 2f42ce1
chore: get credentials from role
strophy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we still need this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed it to try and figure out if it was the cause of step #39 errors