Feature: Support an externally-cacheable image cache for CI use

[MT-Jacobs]

This issue is copied from docker/cli#3858, though I included it here because the use case is directly tied to a weakness in this Github Action.

Description

When used locally (in my case, via Docker for Mac), Docker will naturally cache images so they don't have to be repeatedly pulled:

> docker pull elasticsearch:8.2.2
8.2.2: Pulling from library/elasticsearch
Digest: sha256:8c666cb1e76650306655b67644a01663f9c7a5422b2c51dd570524267f11ce3d
Status: Image is up to date for elasticsearch:8.2.2
docker.io/library/elasticsearch:8.2.2

(Notably, the example above, Elasticsearch 8.2.2, is a whopping 1.23GB!)

Some of these cached images are very useful to reuse in builds! For example, the Testcontianers project allows various applications to use Docker containers to act as datastores during testing.

However, when using Docker in a CI platform such as Github Actions, there is no way I've found to cache Docker images through client commands. I found some fantastic examples of how to cache individual Docker layers on build here, but you can only use the build-push action's caching options on build, not on pull.

Here is the closest I got using Github Actions as an example - note that the buildx command merely shows the intent of what I'd like to do; it doesn't actually solve the problem I have.

jobs:
  build-test:
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Setup Adopt OpenJDK 11
        uses: actions/setup-java@v3
        with:
          java-version: 11
          distribution: adopt
      - name: Grab Docker registry cache
        id: cache-docker
        uses: actions/cache@v3
        with:
          path: /tmp/docker-registry
          key: docker-registry-mytest # todo this name needs to be smarter
      - name: Start local registry as a pull-through cache
        run: docker run -d -p 5000:5000 --restart=always -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io --name registry -v /tmp/docker-registry:/var/lib/registry registry:2 && npx wait-on tcp:5000
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v2
        with:
          install: true
          # WARNING: Does not work, this registry isn't used with `docker pull` commands
          config-inline: |
            debug = true
            [registry."docker.io"]
              mirrors = ["http://localhost:5000"]
      # Not shown - this build automatically invokes test containers which, essentially, use the Docker CLI
      # to run several `docker pull`s
      - name: Gradle build
        uses: gradle/gradle-build-action@v2
        with:
          arguments: build --no-daemon

In a perfect world, it would be even better if I could use a Docker CLI command to define a local disk registry that all pulled images should be dumped into; that way there would be a very clear place to cache images to/from.

All that said, is this the right project to make this request? I almost feel like this is a core piece of functionality I just haven't found the right documentation for. That or there's some very good security reason that we don't support this type of caching.

[nirdshabo1988]

@MT-Jacobs though its not a solution to the general problem, specific solution for testcontainers is available with the env var TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX
reference
https://java.testcontainers.org/features/image_name_substitution/#automatically-modifying-docker-hub-image-names
https://node.testcontainers.org/features/images/#image-name-substitution

Hope its helps anyone.