Skip to content

CI: Don't include commit sha in cache key, ensure caches don't stay reserved #104076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

CI: Don't include commit sha in cache key, ensure caches don't stay reserved #104076

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions .github/actions/godot-cache-restore/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,15 @@ runs:
# We try to match an existing cache to restore from it. Each potential key is checked against
# all existing caches as a prefix. E.g. 'linux-template-minimal' would match any cache that
# starts with "linux-template-minimal", such as
# "linux-template-minimal|master|6588a4a29af1621086feac0117d5d4d37af957fd".
# "linux-template-minimal|master".
#
# We check these prefixes in this order:
# 1. An exact match for the base branch, reference name, and SHA hash.
# 2. A partial match for the same cache name and reference name.
# 3. A partial match for the same cache name and default branch name.
# 1. An exact match for the same cache name and reference name.
# 2. A partial match for the same cache name and default branch name.
- name: Restore SCons cache directory
uses: actions/cache/restore@v4
with:
path: ${{ inputs.scons-cache }}
key: ${{ inputs.cache-name }}|${{ github.ref_name }}|${{ github.sha }}
key: ${{ inputs.cache-name }}|${{ github.ref_name }}
restore-keys: |
${{ inputs.cache-name }}|${{ github.ref_name }}
${{ inputs.cache-name }}|${{ github.event.repository.default_branch }}
2 changes: 1 addition & 1 deletion .github/actions/godot-cache-save/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ runs:
uses: actions/cache/save@v4
with:
path: ${{ inputs.scons-cache }}
key: ${{ inputs.cache-name }}|${{ github.ref_name }}|${{ github.sha }}
key: ${{ inputs.cache-name }}|${{ github.ref_name }}
13 changes: 7 additions & 6 deletions .github/workflows/android_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,6 @@ jobs:
distribution: temurin
java-version: 17

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

Expand All @@ -68,6 +62,12 @@ jobs:
- name: Extract pre-built Android Swappy Frame Pacing Library
run: 7za x -y swappy/godot-swappy.7z -o${{github.workspace}}/thirdparty/swappy-frame-pacing

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true
Comment on lines +65 to +69
Copy link
Member Author

@akien-mga akien-mga Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved the cache restore and save as close as possible to the compiling step to minimal the time where the cache is locked.


- name: Compilation
uses: ./.github/actions/godot-build
with:
Expand All @@ -78,6 +78,7 @@ jobs:
scons-cache-limit: ${{ matrix.cache-limit }}

- name: Save Godot build cache
if: always()
Copy link
Member Author

@akien-mga akien-mga Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it run even if previous steps fail, which ensures that if the cache was locked, it gets unlocked.

If the job fails before cache/restore, then this step still runs and passes fine with just warnings:

Warning: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.
Warning: Cache save failed.

Copy link
Member Author

@akien-mga akien-mga Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I haven't tested though is what happens if the job is canceled (e.g. by pushing a new commit to the PR branch). Let's test this in this PR and makes sure it also runs this steps and unlocks the cache key.

uses: ./.github/actions/godot-cache-save
with:
cache-name: ${{ matrix.cache-name }}
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/ios_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ jobs:
with:
submodules: recursive

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
continue-on-error: true

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

- name: Compilation (arm64)
uses: ./.github/actions/godot-build
with:
Expand All @@ -35,6 +35,7 @@ jobs:
scons-cache-limit: 1

- name: Save Godot build cache
if: always()
uses: ./.github/actions/godot-cache-save
continue-on-error: true

Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/linux_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,12 +128,6 @@ jobs:
sudo rm -rf /usr/local/lib/android
echo "Disk usage after:" && df -h

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Setup Python and SCons
if: "!matrix.legacy-scons"
uses: ./.github/actions/godot-deps
Expand All @@ -158,6 +152,12 @@ jobs:
# Targeting the oldest version we want to support to ensure it still builds.
dotnet-version: 8.0.100

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Compilation
uses: ./.github/actions/godot-build
with:
Expand All @@ -176,6 +176,7 @@ jobs:
godot-cpp-branch: ${{ env.GODOT_CPP_BRANCH }}

- name: Save Godot build cache
if: always()
uses: ./.github/actions/godot-cache-save
with:
cache-name: ${{ matrix.cache-name }}
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/macos_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,16 @@ jobs:
with:
submodules: recursive

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

# This also gets cached.
- name: Setup Vulkan SDK
run: |
sh misc/scripts/install_vulkan_sdk_macos.sh
Expand All @@ -68,6 +69,7 @@ jobs:
scons-cache-limit: ${{ matrix.cache-limit }}

- name: Save Godot build cache
if: always()
uses: ./.github/actions/godot-cache-save
with:
cache-name: ${{ matrix.cache-name }}
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/web_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ jobs:
run: |
emcc -v

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

- name: Compilation
uses: ./.github/actions/godot-build
with:
Expand All @@ -65,6 +65,7 @@ jobs:
scons-cache-limit: 0.5

- name: Save Godot build cache
if: always()
uses: ./.github/actions/godot-cache-save
with:
cache-name: ${{ matrix.cache-name }}
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/windows_builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,6 @@ jobs:
with:
submodules: recursive

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Setup Python and SCons
uses: ./.github/actions/godot-deps

Expand All @@ -85,6 +79,12 @@ jobs:
- name: Extract pre-built ANGLE static libraries
run: Expand-Archive -Force angle/angle.zip ${{ github.workspace }}/

- name: Restore Godot build cache
uses: ./.github/actions/godot-cache-restore
with:
cache-name: ${{ matrix.cache-name }}
continue-on-error: true

- name: Compilation
uses: ./.github/actions/godot-build
with:
Expand All @@ -95,6 +95,7 @@ jobs:
scons-cache-limit: ${{ matrix.cache-limit }}

- name: Save Godot build cache
if: always()
uses: ./.github/actions/godot-cache-save
with:
cache-name: ${{ matrix.cache-name }}
Expand Down
Loading