Skip to content

fix(.github/workflows): reuse cached .git repository for branch HEAD commit #3893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 21, 2025
Merged

fix(.github/workflows): reuse cached .git repository for branch HEAD commit #3893

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b742c82
chore(ci): ensure merge queue runs the same CI checks when testing th…
darccio Aug 19, 2025
60729ed
feat(.github/workflows): swap pull-request and main-branch-tests, so …
darccio Aug 19, 2025
d39ddcb
feat(.github/workflows): add pull-request-tests-done job for handling…
darccio Aug 19, 2025
c2e5a78
fix(.github/workflows): reuse cached .git repository for branch HEAD …
darccio Aug 19, 2025
6671f89
fix(.github/workflows): remove left over from a previous life in mult…
darccio Aug 19, 2025
49ba5a8
Merge branch 'dario.castane/ktlo/disable-main-branch-ci' into dario.c…
darccio Aug 21, 2025
ca7461b
feat(.github/workflows): migrate system-tests and replace checkout wi…
darccio Aug 21, 2025
4f673ed
fix(.github/workflows): pull from the PR's HEAD
darccio Aug 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions .github/workflows/main-branch-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
name: Main Branch and Release Tests

on:
workflow_call: # allows to reuse this workflow
inputs:
ref:
description: 'The branch to run the workflow on'
required: true
type: string
Comment on lines -4 to -9
Copy link
Member Author

@darccio darccio Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer needed because this workflow is never used from another one.

push:
branches:
- release-v*
Expand All @@ -31,4 +25,3 @@ jobs:
pull-requests: write
with:
go-version: "1.25" # Should be the highest supported version of Go
ref: ${{ github.sha }}
20 changes: 10 additions & 10 deletions .github/workflows/multios-unit-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,11 @@ on:
workflow_dispatch: # manually
inputs:
go-version:
description: The Go version to use
Copy link
Member Author

@darccio darccio Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added descriptions because a linter was complaining about them.

required: true
type: string
runs-on:
required: true
type: string
ref:
description: 'The branch to run the workflow on'
description: The OS to run the tests on
required: true
type: string
workflow_call:
Expand All @@ -21,10 +19,6 @@ on:
runs-on:
required: true
type: string
ref:
description: 'The branch to run the workflow on'
required: true
type: string

env:
DD_APPSEC_WAF_TIMEOUT: 1m # Increase time WAF time budget to reduce CI flakiness
Expand All @@ -50,10 +44,16 @@ jobs:
shell: pwsh
run: |
"normalized_workspace=${{ github.workspace }}" >> $env:GITHUB_ENV
- name: Restore repo cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-${{ github.repository_id }}-${{ github.sha }}
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v2.7.0
with:
ref: ${{ inputs.ref || github.ref }}
ref: ${{ github.sha }}
clean: false
Comment on lines 52 to +56
Copy link
Member Author

@darccio darccio Aug 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If by any chance the previous step doesn't find the cached repo, checkout will just work as usual by downloading the repository. clean: false is only needed because we'll usually restore from the cached .git directory.

- name: Setup Go and development tools
uses: ./.github/actions/setup-go
with:
Expand All @@ -63,7 +63,7 @@ jobs:
- name: Mac OS Coreutils
if: inputs.runs-on == 'macos-latest'
run: brew install coreutils
- name: "Runner ${{ matrix.runner-index }}: Test Core and Contrib (No Integration Tests)"
- name: "Runner: Test Core and Contrib (No Integration Tests)"
shell: bash
run: |
export PATH="${{ github.workspace }}/bin:${PATH}"
Expand Down
18 changes: 16 additions & 2 deletions .github/workflows/pull-request.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,33 @@ concurrency:
cancel-in-progress: true

jobs:
warm-repo-cache:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-${{ github.repository_id }}-${{ github.sha }}
unit-integration-tests:
name: PR Unit and Integration Tests
needs:
- warm-repo-cache
strategy:
matrix:
go-version: [ "1.24", "1.25" ]
fail-fast: false
uses: ./.github/workflows/unit-integration-tests.yml
with:
go-version: ${{ matrix.go-version }}
ref: ${{ github.sha }}
secrets: inherit
multios-unit-tests:
needs:
- warm-repo-cache
strategy:
matrix:
runs-on: [ macos-latest, windows-latest, ubuntu-latest ]
Expand All @@ -33,7 +48,6 @@ jobs:
with:
go-version: ${{ matrix.go-version }}
runs-on: ${{ matrix.runs-on }}
ref: ${{ github.sha }}
secrets: inherit
# This is a simple join point to make it easy to set up branch protection rules in GitHub.
pull-request-tests-done:
Expand Down
35 changes: 31 additions & 4 deletions .github/workflows/system-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,32 @@ permissions:
packages: write

jobs:
warm-repo-cache:
runs-on: ubuntu-latest
outputs:
sha: ${{ steps.pin.outputs.sha }}
steps:
- name: Checkout system-tests
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: 'DataDog/system-tests'
ref: ${{ inputs.ref }}
- name: Pin exact commit SHA for system-tests
id: pin
run: |
echo "sha=$(git rev-parse FETCH_HEAD)" >> "$GITHUB_OUTPUT"
- name: Cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-system-tests-${{ steps.pin.outputs.sha }}
system-tests:
if: github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'DataDog/dd-trace-go')
# Note: Not using large runners because the jobs spawned by this pipeline
# don't seem to get a noticable speedup from using larger runners.
runs-on: ubuntu-latest
needs:
- warm-repo-cache
strategy:
matrix:
weblog-variant:
Expand Down Expand Up @@ -128,11 +149,17 @@ jobs:
SYSTEM_TESTS_E2E_DD_APP_KEY: ${{ secrets.SYSTEM_TESTS_E2E_DD_APP_KEY }}
name: Test (${{ matrix.weblog-variant }}, ${{ matrix.scenario }})
steps:
- name: Checkout system tests
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Restore repo cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
repository: 'DataDog/system-tests'
ref: ${{ inputs.ref }}
path: .git
key: gitdb-system-tests-${{ needs.warm-repo-cache.outputs.sha }}

- name: Checkout system tests
shell: bash
run: |
git config safe.directory "$GITHUB_WORKSPACE"
git checkout -f ${{ needs.warm-repo-cache.outputs.sha }}

- name: Checkout dd-trace-go
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand Down
30 changes: 21 additions & 9 deletions .github/workflows/unit-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@ on:
go-version:
required: true
type: string
ref:
description: 'The branch to run the workflow on'
required: true
type: string

env:
DD_APPSEC_WAF_TIMEOUT: 1m # Increase time WAF time budget to reduce CI flakiness
Expand All @@ -31,10 +27,16 @@ jobs:
outputs:
matrix: ${{ steps.matrix.outputs.matrix }}
steps:
- name: Restore repo cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-${{ github.repository_id }}-${{ github.sha }}
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.ref || github.ref }}
ref: ${{ github.sha }}
clean: false

- name: Compute Matrix
id: matrix
Expand Down Expand Up @@ -201,11 +203,16 @@ jobs:
ports:
- 4566:4566
steps:
- name: Restore repo cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-${{ github.repository_id }}-${{ github.sha }}
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.ref || github.ref }}
fetch-depth: $(( ${{ github.event_name == 'pull_request' && github.event.pull_request.commits || 0 }} + 1 ))
Copy link
Member Author

@darccio darccio Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default fetch-depth is 1. This expression seems to end with 1 or 2, which I don't think it's required.

ref: ${{ github.sha }}
clean: false

- name: Setup Go and development tools
uses: ./.github/actions/setup-go
Expand Down Expand Up @@ -273,11 +280,16 @@ jobs:
- 8125:8125/udp
- 8126:8126
steps:
- name: Restore repo cache
uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
with:
path: .git
key: gitdb-${{ github.repository_id }}-${{ github.sha }}
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ inputs.ref || github.ref }}
fetch-depth: $(( ${{ github.event_name == 'pull_request' && github.event.pull_request.commits || 0 }} + 1 ))
ref: ${{ github.sha }}
clean: false
- name: Setup Go and development tools
uses: ./.github/actions/setup-go
with:
Expand Down
Loading