[proposal] implement pull-policies (docker build --pull=missing|never|always)

[thaJeztah]

Pull request docker/cli#1498 added a --pull option to docker run and docker create (to address moby/moby#34394), which allows the pull-policy to be configured to use, and has three options:

--pull	description
(not set)	(default) pull the image if it does not exist locally
--pull=missing	same as default above
--pull=never	don't pull the image, but produce an error if the image does not exist
--pull=always	always attempt to pull the image, even if the image exists locally: produce an error if pull failed

docker build also has a --pull flag, but is implemented as a boolean:

--pull	description
(not set)	(default) pull the image if it does not exist locally
--pull=false	same as above
--pull / --pull=true	always attempt to pull the image, even if the image exists locally: produce an error if pull failed. This is the equivalent of --pull=always on docker run

Proposal

I think it would make sense to have docker build support the same options as docker run. For backward-compatibility, --pull would be the equivalent of --pull=always:

--pull	description
(not set)	(default) pull the image if it does not exist locally
--pull=missing	same as default above
--pull=never	don't pull the image, but produce an error if the image does not exist
--pull=always	always attempt to pull the image, even if the image exists locally: produce an error if pull failed
--pull	equivalent to --pull=always. kept for backward compatibility

[thaJeztah]

@tonistiigi wdyt?

[Madhur1997]

Hey @thaJeztah,
I was going through the code flow for this.
We can have this implemented by few modifications at client side itself.

1. --pull=always, --pull are simply equivalent to today's --pull=true.
2. --pull=missing, unset --pull are equivalent to --pull=false.
3. For --pull=never, we can do some sort of
   if imageExistsLocally {
   build(--pull=missing)
   } else {
   log(err)
   }

How does this sound to you?
Please let me know if you have any suggestions.
I would like to take this up.

[thaJeztah]

--pull=missing, unset --pull are equivalent to --pull=false.

actually, the last one is one I didn't look at. Likely a "corner case", but wondering if it would be less surprising if --pull=false would be equivalent to --pull=never. I haven't checked how docker/cli#1498 handles that (and if changing would be a breaking change 🤔)

We can have this implemented by few modifications at client side itself.

We may have to look at docker build with buildkit enabled (DOCKER_BUILDKIT=1), also with multi-stage builds in mind (so there's not a single image to check, and names of images used in specific stages could depend on ARG values). I haven't looked into that yet

@tonistiigi @AkihiroSuda any thoughts?

[felipecrs]

#1889 looks perfect. --pull=never would be a very handy feature for using locally built images as base for another. Especially when using docker buildx bake.

[tbroyer]

Fwiw, this would be a breaking change unless you really carefully implement it as described above.

We've just been bitten by the argument parsing of --pull on the run command, that doesn't validate the values, so our docker run --pull --rm … actually was parsed as --pull=--rm, without any runtime error. This has two side effects:

• the current implementation of the run command only compares the value with always and missing, so any other value is effectivement equivalent to never
• our --rm was eaten silently

(I opened docker/cli#3681 about this)

[tonistiigi]

I'm not sure if the CLI library allows updating this in backward compatible way but assuming it does:

--pull=never | don't pull the image, but produce an error if the image does not exist

This would require BuildKit change. There is no current ResolveMode implementation with these semantics.

What's missing from the list is the "fallback mode" (not sure what would be the correct name) that is the default in buildkitd without docker imagestore. In this mode, you always get the latest version of the image, but you can still use local images as well. If the build fails because there is no remote image with that name or you don't have an active network connection, then it will fall back to using a local version of the image.

[felipecrs]

No matter what I believe it would be nice to have a similar syntax between run and build, while still retaining backwards compatibility, i.e:

--pull and --pull=true translates to --pull=always, --pull=false translates to --pull=missing.

[thaJeztah]

I'm not sure if the CLI library allows updating this in backward compatible way

Honest answer: "close", but not fully.

We did this for docker run, and thought we covered all scenarios (--pull without value is always, --pull=true, --pull=false)

But there's one exception that we overlooked; docker run --pull -it ubuntu works, but docker run -it --pull ubuntu will produce an error, because ubuntu will be set as value for --pull.

[thaJeztah]

In either case, I think this ticket would now better be in the buildx repository (although we may have to implement it for the classic builder as well for parity once it's implemented there). Let me migrate it to there 👍