Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/actions/docker-build-service/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ inputs:
description: Load the built image into the local Docker daemon.
required: false
default: 'false'
use-cache:
description: >-
Import the gha layer cache for this service. Set to 'false' on
retries: BuildKit's "failed to calculate checksum of ref ...:
not found" failure comes from a stale/corrupted gha cache entry,
and cache-from is scoped identically across retries within the
same job, so a retry that reimports it fails identically instead
of getting a fresh build.
required: false
default: 'true'
runs:
using: composite
steps:
Expand Down Expand Up @@ -52,7 +62,7 @@ runs:
tags: lucky-${{ inputs.service }}:ci
provenance: false
sbom: false
cache-from: type=gha,scope=${{ inputs.service }}
cache-from: ${{ inputs.use-cache == 'true' && format('type=gha,scope={0}', inputs.service) || '' }}
cache-to: type=gha,mode=max,scope=${{ inputs.service }}
build-args: |
COMMIT_SHA=${{ github.sha }}
Expand Down
25 changes: 14 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -355,17 +355,18 @@ jobs:
target: ${{ matrix.target }}
load: ${{ matrix.service == 'bot' || matrix.service == 'nginx' }}
# Retry on transient failure (buildx setup and build re-run
# together, matching the previous separate retry steps). Two
# retries (three attempts total) because the failure mode here
# isn't a one-off blip: BuildKit intermittently fails to resolve
# a COPY --from=<stage> reference — "failed to calculate
# checksum of ref ...: not found" — and which specific path it
# hits varies run to run (node_modules, dist, prisma, generated
# — reproduced across all of them). Disk pressure, GHA cache,
# buildx version, and Dockerfile structure were all ruled out
# with direct evidence; this reads as upstream BuildKit/runner
# flakiness with a high enough per-attempt failure rate that a
# single retry isn't reliably enough headroom.
# together, matching the previous separate retry steps). Root
# cause: BuildKit fails to resolve a COPY --from=<stage>
# reference — "failed to calculate checksum of ref ...: not
# found" — because cache-from imports a stale/corrupted gha
# cache entry for that stage. cache-from is scoped identically
# across attempts within the same job, so a retry that reimports
# the same entry fails identically instead of getting a fresh
# build (confirmed: all retries in one job fail on the exact
# same ref hash). use-cache: 'false' skips cache-from on retries
# so they build fresh instead of re-hitting the poisoned cache.
# Two retries (three attempts total) for headroom against the
# ordinary transient case too.
- name: Build ${{ matrix.service }} - retry 1 on transient failure
id: build-retry-1
if: steps.build.outcome == 'failure'
Expand All @@ -376,6 +377,7 @@ jobs:
file: ${{ matrix.file }}
target: ${{ matrix.target }}
load: ${{ matrix.service == 'bot' || matrix.service == 'nginx' }}
use-cache: 'false'
- name: Build ${{ matrix.service }} - retry 2 on transient failure
if: steps.build.outcome == 'failure' && steps.build-retry-1.outcome == 'failure'
uses: ./.github/actions/docker-build-service
Expand All @@ -384,6 +386,7 @@ jobs:
file: ${{ matrix.file }}
target: ${{ matrix.target }}
load: ${{ matrix.service == 'bot' || matrix.service == 'nginx' }}
use-cache: 'false'
# A built image is not a working image: the bot's native addons
# (@discordjs/opus) and the baked Prisma engines only fail at
# require()-time, which `docker build` never reaches. #1735 shipped a
Expand Down
Loading