-
Notifications
You must be signed in to change notification settings - Fork 9
Split Docker builds: fast PR smoke tests and cron-only publishing #398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
fd2ce60
Split Docker builds: fast PR smoke tests and cron-only publishing
ptr727 1631ca5
Document PR Copilot review process in AGENTS.md
ptr727 13aee88
Address Copilot review: ref fallback, gated login, safe jq, concurrency
ptr727 6c44797
Address Copilot re-review: pin release to main, gate release, status …
ptr727 a3d77b0
Address Copilot re-review: base push/cache key on built ref not trigger
ptr727 71cea50
Document Copilot re-review trigger (requestReviews botIds) in AGENTS.md
ptr727 81b259e
Address Copilot re-review: pin readme to main, always update date badge
ptr727 f947218
Address Copilot re-review: only build base in smoke when base changed
ptr727 6a45f71
Address Copilot re-review: full history for paths-filter changes job
ptr727 eb2729d
Address Copilot re-review: queue publishes instead of cancelling
ptr727 e1d8fd4
Address Copilot re-review: pin GitHub release target to main
ptr727 abec4f0
Address Copilot re-review: smoke covers both branches, sync README ve…
ptr727 671d395
Configure workspace linting for structured files via recommended exte…
ptr727 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,104 +1,168 @@ | ||
| name: Build Docker image task | ||
|
|
||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| push: | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
|
|
||
| jobs: | ||
|
|
||
| get-matrix: | ||
| name: Get matrix job | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| matrix: ${{ steps.getmatrix.outputs.matrix }} | ||
|
|
||
| steps: | ||
|
|
||
| - name: Checkout step | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Load matrix.json step | ||
| id: getmatrix | ||
| run: | | ||
| echo "matrix=$(jq --compact-output '.' ./Make/Matrix.json)" >> $GITHUB_OUTPUT | ||
|
|
||
| get-version: | ||
| name: Get version information job | ||
| uses: ./.github/workflows/get-version-task.yml | ||
| secrets: inherit | ||
|
|
||
| build-base: | ||
| name: Build base image job | ||
| uses: ./.github/workflows/build-base-images-task.yml | ||
| with: | ||
| push: ${{ inputs.push }} | ||
| secrets: inherit | ||
|
|
||
| build-docker: | ||
| name: Build Docker image job | ||
| runs-on: ubuntu-latest | ||
| needs: [get-version, get-matrix, build-base] | ||
|
|
||
| strategy: | ||
| max-parallel: 4 | ||
| matrix: | ||
| images: ${{ fromJson(needs.get-matrix.outputs.matrix).Images }} | ||
|
|
||
| steps: | ||
|
|
||
| - name: Checkout step | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Setup QEMU step | ||
| uses: docker/setup-qemu-action@v4 | ||
| with: | ||
| platforms: linux/amd64,linux/arm64 | ||
|
|
||
| - name: Setup Buildx step | ||
| uses: docker/setup-buildx-action@v4 | ||
| with: | ||
| platforms: linux/amd64,linux/arm64 | ||
|
|
||
| - name: Login to Docker Hub step | ||
| uses: docker/login-action@v4 | ||
| with: | ||
| registry: docker.io | ||
| username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
| password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
|
||
| - name: Create tags and args step | ||
| id: tagsargs | ||
| run: | | ||
| TAGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Tags) }}') | ||
| ARGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Args) }}') | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "$TAGS" | ||
| echo "EOF" | ||
| echo "args<<EOF" | ||
| echo "$ARGS" | ||
| echo "LABEL_VERSION=${{ needs.get-version.outputs.SemVer2 }}" | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Docker build and push step | ||
| uses: docker/build-push-action@v7 | ||
| with: | ||
| push: ${{ inputs.push && (github.ref_name == matrix.images.Branch) }} | ||
| context: Docker | ||
| file: Docker/${{ matrix.images.Name }}.Dockerfile | ||
| platforms: linux/amd64,linux/arm64 | ||
| tags: ${{ steps.tagsargs.outputs.tags }} | ||
| build-args: ${{ steps.tagsargs.outputs.args }} | ||
| cache-from: | | ||
| type=gha,scope=develop-${{ matrix.images.Name }} | ||
| type=gha,scope=main-${{ matrix.images.Name }} | ||
| ${{ github.event.pull_request && format('type=gha,scope=pr-{0}-{1}', github.event.pull_request.number, matrix.images.Name) || '' }} | ||
| cache-to: | | ||
| ${{ (github.ref_name == 'main' || github.ref_name == 'develop') && format('type=gha,mode=min,scope={0}-{1},ignore-error=true', github.ref_name, matrix.images.Name) || '' }} | ||
| ${{ github.event.pull_request && !github.event.pull_request.head.repo.fork && format('type=gha,mode=min,scope=pr-{0}-{1},ignore-error=true', github.event.pull_request.number, matrix.images.Name) || '' }} | ||
| name: Build Docker image task | ||
|
|
||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| push: | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
| # Smoke mode: build a single Ubuntu + single LSIO variant on amd64 | ||
| # only, never push. Used for fast PR feedback in place of the full | ||
| # matrix. See test-pull-request.yml. | ||
| smoke: | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
| # Branch to check out and whose images to build/push. Empty uses the | ||
| # triggering ref (PR context). The publisher passes main and develop | ||
| # so both branches' tags are produced from one scheduled run. | ||
| ref: | ||
| required: false | ||
| type: string | ||
| default: '' | ||
| # When false the caller is expected to have built the base images | ||
| # already (the publisher builds them once, see publish-release.yml), | ||
| # so the internal base build is skipped. | ||
| build_base: | ||
| required: false | ||
| type: boolean | ||
| default: true | ||
|
|
||
| jobs: | ||
|
|
||
| get-matrix: | ||
| name: Get matrix job | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| matrix: ${{ steps.getmatrix.outputs.matrix }} | ||
|
|
||
| steps: | ||
|
|
||
| - name: Checkout step | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| ref: ${{ inputs.ref || github.ref }} | ||
|
|
||
| - name: Load matrix.json step | ||
| id: getmatrix | ||
| env: | ||
| SMOKE: ${{ inputs.smoke }} | ||
| REF: ${{ inputs.ref }} | ||
| run: | | ||
| # $ref below is a jq variable (passed via --arg), not a shell | ||
| # variable, so it must stay single-quoted / unexpanded. | ||
| # shellcheck disable=SC2016 | ||
| if [[ "$SMOKE" == "true" ]]; then | ||
| # One Ubuntu + one LSIO variant per branch exercises the shared | ||
| # Dockerfile build logic and each branch's build args; tags are | ||
| # irrelevant since smoke never pushes. De-dup by name+branch so a | ||
| # PR targeting develop still validates the develop rows. | ||
| FILTER='.Images |= (map(select(.Name == "NxMeta" or .Name == "NxMeta-LSIO")) | unique_by([.Name, .Branch]))' | ||
| elif [[ -n "$REF" ]]; then | ||
| # Publish: build only the rows targeting the branch being built | ||
| # (avoids building the other branch's rows just to discard them). | ||
| # $ref is passed via --arg so the value is never interpolated | ||
| # into the jq program. | ||
| FILTER='.Images |= map(select(.Branch == $ref))' | ||
| else | ||
| FILTER='.' | ||
| fi | ||
| echo "matrix=$(jq --arg ref "$REF" --compact-output "$FILTER" ./Make/Matrix.json)" >> "$GITHUB_OUTPUT" | ||
|
|
||
| get-version: | ||
| name: Get version information job | ||
| uses: ./.github/workflows/get-version-task.yml | ||
| secrets: inherit | ||
| with: | ||
| ref: ${{ inputs.ref }} | ||
|
|
||
| build-base: | ||
| name: Build base image job | ||
| if: ${{ inputs.build_base }} | ||
| uses: ./.github/workflows/build-base-images-task.yml | ||
| with: | ||
| push: ${{ inputs.push }} | ||
| platforms: ${{ inputs.smoke && 'linux/amd64' || 'linux/amd64,linux/arm64' }} | ||
| ref: ${{ inputs.ref }} | ||
| secrets: inherit | ||
|
|
||
| build-docker: | ||
| name: Build Docker image job | ||
| runs-on: ubuntu-latest | ||
| needs: [get-version, get-matrix, build-base] | ||
| # always() so the job still runs when build-base is intentionally | ||
| # skipped (build_base == false); fail only if a prerequisite failed. | ||
| if: >- | ||
| ${{ always() | ||
| && needs.get-version.result == 'success' | ||
| && needs.get-matrix.result == 'success' | ||
| && (needs.build-base.result == 'success' || needs.build-base.result == 'skipped') }} | ||
|
|
||
| strategy: | ||
| max-parallel: 4 | ||
| matrix: | ||
| images: ${{ fromJson(needs.get-matrix.outputs.matrix).Images }} | ||
|
|
||
| steps: | ||
|
|
||
| - name: Checkout step | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| ref: ${{ inputs.ref || github.ref }} | ||
|
|
||
| - name: Setup QEMU step | ||
| uses: docker/setup-qemu-action@v4 | ||
| with: | ||
| platforms: ${{ inputs.smoke && 'linux/amd64' || 'linux/amd64,linux/arm64' }} | ||
|
|
||
| - name: Setup Buildx step | ||
| uses: docker/setup-buildx-action@v4 | ||
| with: | ||
| platforms: ${{ inputs.smoke && 'linux/amd64' || 'linux/amd64,linux/arm64' }} | ||
|
|
||
| # Only needed when pushing; skipped for non-publishing builds (e.g. | ||
| # PR smoke) so they don't depend on Docker Hub secrets (unavailable | ||
| # on forked PRs). The public base image still pulls anonymously. | ||
| - name: Login to Docker Hub step | ||
| if: ${{ inputs.push }} | ||
| uses: docker/login-action@v4 | ||
| with: | ||
| registry: docker.io | ||
| username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
| password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
|
||
| - name: Create tags and args step | ||
| id: tagsargs | ||
| run: | | ||
| TAGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Tags) }}') | ||
| ARGS=$(jq -r '.[]' <<< '${{ toJson(matrix.images.Args) }}') | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "$TAGS" | ||
| echo "EOF" | ||
| echo "args<<EOF" | ||
| echo "$ARGS" | ||
| echo "LABEL_VERSION=${{ needs.get-version.outputs.SemVer2 }}" | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Docker build and push step | ||
| uses: docker/build-push-action@v7 | ||
| with: | ||
| # Matrix is already filtered to the target branch, so push when asked. | ||
| # Smoke callers pass push:false. | ||
| push: ${{ inputs.push }} | ||
| context: Docker | ||
| file: Docker/${{ matrix.images.Name }}.Dockerfile | ||
| platforms: ${{ inputs.smoke && 'linux/amd64' || 'linux/amd64,linux/arm64' }} | ||
| tags: ${{ steps.tagsargs.outputs.tags }} | ||
| build-args: ${{ steps.tagsargs.outputs.args }} | ||
| cache-from: | | ||
| type=gha,scope=develop-${{ matrix.images.Name }} | ||
| type=gha,scope=main-${{ matrix.images.Name }} | ||
| ${{ github.event.pull_request && format('type=gha,scope=pr-{0}-{1}', github.event.pull_request.number, matrix.images.Name) || '' }} | ||
| cache-to: | | ||
| ${{ inputs.push && format('type=gha,mode=min,scope={0}-{1},ignore-error=true', inputs.ref != '' && inputs.ref || github.ref_name, matrix.images.Name) || '' }} | ||
| ${{ github.event.pull_request && !github.event.pull_request.head.repo.fork && format('type=gha,mode=min,scope=pr-{0}-{1},ignore-error=true', github.event.pull_request.number, matrix.images.Name) || '' }} | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.