feat: Introduce update strategy 'digest' (#176)

* feat: Introduce update strategy 'digest'

Signed-off-by: jannfis <[email protected]>

* Update tests

Signed-off-by: jannfis <[email protected]>

* Set dummy digest for update strategy 'digest'

Signed-off-by: jannfis <[email protected]>

* Update docs

Signed-off-by: jannfis <[email protected]>

* Update spelling

Signed-off-by: jannfis <[email protected]>

.github/actions/spelling/allow.txt

@@ -127,6 +127,7 @@ mkdocs
 mktemp
 mutex
 myalias
+myimage
 myimg
 mynamespace
 mysecret
@@ -177,6 +178,7 @@ serviceaccount
 Setenv
 SHAs
 sirupsen
+somelonghashstring
 someparam
 somepassword
 someuser

docs/install/strategies.md

@@ -0,0 +1,194 @@
+# Update strategies
+
+Argo CD Image Updater supports different update strategies for the images that
+are configured to be tracked and updated.
+
+You can configure the strategy to be used for each image individually, with the
+default strategy being `semver` (see below).
+
+Some of the strategies will require additional configuration, or can be tweaked
+with additional parameters. Please have a look at the
+[image configuration](../../configuration/images)
+documentation for more details.
+
+Please note that all update strategies except `digest` assume tags to be
+*immutable* and that new images will be pushed with a new, unique tag. If
+you want to update to *mutable* tags (e.g. the commonly used `latest` tag),
+you should use the `digest` strategy.
+
+## Update to semantic versions
+
+This is the default strategy.
+
+Strategy name: `semver`
+
+```yaml
+argocd-image-updater.argoproj.io/<image>.update-strategy: semver[:<version_constraint>]
+```
+
+The `semver` strategy allows you to track & update images which use tags that
+follow the
+[semantic versioning scheme](https://semver.org). Tag names must contain semver
+compatible identifiers in the format `X.Y.Z`, where `X`, `Y` and `Z` must be
+whole numbers. An optional prefix of `v`, e.g. `vX.Y.Z` is allowed, and both
+variants are treated equal (so, a constraint of `v1.x` would match a tag `1.0`
+and a constraint of `1.x` also matches a tag `v1.0`).
+
+Updating to pre-release versions (e.g. `-rc1`) is supported, but must be 
+explicitly allowed (see below).
+
+This will allow you to update to the latest version of an image within a given
+patch branch or minor release, or just to the latest version that has is tagged
+with a valid semantic version identifier.
+
+To tell Argo CD Image Updater which versions are allowed, simply give a semver
+version as a constraint in the `image-list` annotation. For example, to allow
+updates to the latest patch release within the `1.2` minor release branch, use
+
+```
+argocd-image-updater.argoproj.io/image-list: some/image:1.2.x
+```
+
+The above example would update to any new tag pushed to the registry matching
+this constraint, e.g. `1.2.5`, `1.2.12` etc, but not to a new minor version
+(e.g. `1.3`).
+
+Likewise, to allow updates to any minor release within the major version `1`,
+use
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: some/image:1.x
+```
+
+The above example would update to any new tag pushed to the registry matching
+this constraint, e.g. `1.2.12`, `1.3.0`, `1.15.2` etc, but not to a new major
+version (e.g. `2.0`).
+
+If you also want to allow updates to pre-release versions (e.g. `v2.0-rc1`),
+you need to append the suffix `-0` to the constraint, for example
+
+```
+argocd-image-updater.argoproj.io/image-list: some/image:2.x-0
+```
+
+If no version constraint is specified in the list of allowed images, Argo CD
+Image Updater will pick the highest version number found in the registry.
+
+Argo CD Image Updater will any tags from your registry that do not match a
+semantic version when using the `semver` update strategy.
+
+## Update to the most recently built image
+
+!!!warning
+    As of November 2020, Docker Hub has introduced pull limits for accounts on
+    the free plan and unauthenticated requests. The `latest` update strategy
+    will perform manifest pulls for determining the most recently pushed tags,
+    and these will count into your pull limits. So unless you are not affected
+    by these pull limits, it is **not recommended** to use the `latest` update
+    strategy with images hosted on Docker Hub.
+
+Strategy name: `latest`
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: <alias>=some/image
+argocd-image-updater.argoproj.io/<alias>.update-strategy: latest
+```
+
+Argo CD Image Updater can update to the image that has the most recent build
+date, and is tagged with an arbitrary name (e.g. a Git commit SHA, or even a
+random string). 
+
+It is important to understand, that this strategy will pick the build date of
+the image, and not the date of when the image was pushed to the registry. 
+
+By default, this update strategy will inspect all of the tags it found in the
+image's repository. If you wish to allow only certain tags to be considered
+for update, you will need additional configuration. For example,
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: myimage=some/image
+argocd-image-updater.argoproj.io/myimage.update-strategy: latest
+argocd-image-updater.argoproj.io/myimage.allow-tags: regexp:^[0-9a-f]{7}$
+```
+
+would only consider tags that match a given regular expression for update. In
+this case, the regular expression matches a 7-digit hexadecimal string that
+could represent the short version of a Git commit SHA, so it would match tags
+like `a5fb3d3` or `f7bb2e3`, but not `latest` or `master`.
+
+Likewise, you can ignore a certain list of tags from your repository:
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: myimage=some/image
+argocd-image-updater.argoproj.io/myimage.update-strategy: latest
+argocd-image-updater.argoproj.io/myimage.ignore-tags: latest, master
+```
+
+This would allow for considering all tags found but `latest` and `master`. You
+can read more about filtering tags
+[here](../../configuration/images/#filtering-tags).
+
+## Update to the most recent pushed version of a tag
+
+Strategy name: `digest`
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: <alias>=some/image:<tag_name>
+argocd-image-updater.argoproj.io/<alias>.update-strategy: digest
+```
+
+This update strategy inspects a single tag in the registry for changes, and
+updates the image on any change to the previous state. The tag name to be
+inspected must be specified as a version constraint in the image list.
+
+Use this update strategy if you want to follow a *mutable* tag, such as the
+commonly used `latest` tag. 
+
+Argo CD Image Updater will then update the image when either
+
+* The currently running image has a non-digest specification (e.g. uses a tag),
+  or
+* the currently used digest differs from what is found in the registry
+
+Note that the `digest` update strategy will use image digests for updating
+the image in your applications, so the image running in your application
+will appear as
+
+```
+some/image@sha256:<somelonghashstring>
+```
+
+instead of
+
+```
+some/image:latest
+```
+
+## Update according to lexical sort
+
+Strategy name: `name`
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: <alias>=some/image
+argocd-image-updater.argoproj.io/<alias>.update-strategy: name
+```
+
+This update strategy sorts the tags returned by the registry in a lexical way
+(by name, in descending order) and picks the last tag in the list for update.
+This can be useful if the image you want to track uses `calver` versioning,
+with tags in the format of e.g. `YYYY-MM-DD` or similar lexical sortable
+strings.
+
+By default, this update strategy will inspect all of the tags it found in the
+image's repository. If you wish to allow only certain tags to be considered
+for update, you will need additional configuration. For example,
+
+```yaml
+argocd-image-updater.argoproj.io/image-list: myimage=some/image
+argocd-image-updater.argoproj.io/myimage.update-strategy: name
+argocd-image-updater.argoproj.io/myimage.allow-tags: regexp:^[0-9]{4}-[0-9]{2}[0-9]{2}$
+```
+
+would only consider tags that match a given regular expression for update. In
+this case, only tags matching a date specification of `YYYY-MM-DD` would be
+considered for update.

mkdocs.yml

@@ -26,6 +26,7 @@ nav:
   - Install:
     - Getting Started: install/start.md
     - Running Argo CD Image Updater: install/running.md
+  - Update strategies: install/strategies.md
   - Configuration: 
     - Applications: configuration/applications.md
     - Images: configuration/images.md

pkg/argocd/argocd.go

@@ -365,7 +365,7 @@ func SetHelmImage(app *v1alpha1.Application, newImage *image.ContainerImage) err
 			mergeParams = append(mergeParams, p)
 		}
 		if hpImageTag != "" {
-			p := v1alpha1.HelmParameter{Name: hpImageTag, Value: newImage.ImageTag.TagName, ForceString: true}
+			p := v1alpha1.HelmParameter{Name: hpImageTag, Value: newImage.ImageTag.String(), ForceString: true}
 			mergeParams = append(mergeParams, p)
 		}
 	}

pkg/argocd/update.go

@@ -236,9 +236,20 @@ func UpdateApplication(updateConf *UpdateConfiguration, state *SyncIterationStat
 			continue
 		}
 
+		// If the user has specified digest as update strategy, but the running
+		// image is configured to use a tag and no digest, we need to set an
+		// initial dummy digest, so that tag.Equals() will return false.
+		// TODO: Fix this. This is just a workaround.
+		if vc.SortMode == image.VersionSortDigest {
+			if !updateableImage.ImageTag.IsDigest() {
+				log.Tracef("Setting dummy digest for image %s", updateableImage.GetFullNameWithTag())
+				updateableImage.ImageTag.TagDigest = "dummy"
+			}
+		}
+
 		// If the latest tag does not match image's current tag, it means we have
 		// an update candidate.
-		if updateableImage.ImageTag.TagName != latest.TagName {
+		if !updateableImage.ImageTag.Equals(latest) {
 
 			imgCtx.Infof("Setting new image to %s", updateableImage.WithTag(latest).String())
 			needUpdate = true

pkg/cache/memcache_test.go

@@ -15,7 +15,7 @@ func Test_MemCache(t *testing.T) {
 	imageTag := "v1.0.0"
 	t.Run("Cache hit", func(t *testing.T) {
 		mc := NewMemCache()
-		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0))
+		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0), "")
 		mc.SetTag(imageName, newTag)
 		cachedTag, err := mc.GetTag(imageName, imageTag)
 		require.NoError(t, err)
@@ -26,7 +26,7 @@ func Test_MemCache(t *testing.T) {
 
 	t.Run("Cache miss", func(t *testing.T) {
 		mc := NewMemCache()
-		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0))
+		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0), "")
 		mc.SetTag(imageName, newTag)
 		cachedTag, err := mc.GetTag(imageName, "v1.0.1")
 		require.NoError(t, err)
@@ -36,7 +36,7 @@ func Test_MemCache(t *testing.T) {
 
 	t.Run("Cache clear", func(t *testing.T) {
 		mc := NewMemCache()
-		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0))
+		newTag := tag.NewImageTag(imageTag, time.Unix(0, 0), "")
 		mc.SetTag(imageName, newTag)
 		cachedTag, err := mc.GetTag(imageName, imageTag)
 		require.NoError(t, err)

pkg/image/image.go

@@ -35,14 +35,7 @@ func (img *ContainerImage) String() string {
 		str += img.ImageAlias
 		str += "="
 	}
-	if img.RegistryURL != "" {
-		str += img.RegistryURL + "/"
-	}
-	str += img.ImageName
-	if img.ImageTag != nil {
-		str += ":"
-		str += img.ImageTag.TagName
-	}
+	str += img.GetFullNameWithTag()
 	return str
 }
 
@@ -55,15 +48,22 @@ func (img *ContainerImage) GetFullNameWithoutTag() string {
 	return str
 }
 
+// GetFullNameWithTag returns the complete image slug, including the registry
+// and any tag digest or tag name set for the image.
 func (img *ContainerImage) GetFullNameWithTag() string {
 	str := ""
 	if img.RegistryURL != "" {
 		str += img.RegistryURL + "/"
 	}
 	str += img.ImageName
 	if img.ImageTag != nil {
-		str += ":"
-		str += img.ImageTag.TagName
+		if img.ImageTag.TagDigest != "" {
+			str += "@"
+			str += img.ImageTag.TagDigest
+		} else if img.ImageTag.TagName != "" {
+			str += ":"
+			str += img.ImageTag.TagName
+		}
 	}
 	return str
 }
@@ -149,10 +149,26 @@ func getImageTagFromIdentifier(identifier string) (string, string, *tag.ImageTag
 		imageString = strings.Join(comp[1:], "/")
 	}
 
-	comp = strings.SplitN(imageString, ":", 2)
-	if len(comp) != 2 {
-		return sourceName, imageString, nil
+	// We can either have a tag name or a digest reference
+	if strings.Contains(imageString, "@") {
+		comp = strings.SplitN(imageString, "@", 2)
+		return sourceName, comp[0], tag.NewImageTag("", time.Unix(0, 0), comp[1])
+	} else {
+		comp = strings.SplitN(imageString, ":", 2)
+		if len(comp) != 2 {
+			return sourceName, imageString, nil
+		} else {
+			tagName, tagDigest := getImageDigestFromTag(comp[1])
+			return sourceName, comp[0], tag.NewImageTag(tagName, time.Unix(0, 0), tagDigest)
+		}
+	}
+}
+
+func getImageDigestFromTag(tagStr string) (string, string) {
+	a := strings.Split(tagStr, "@")
+	if len(a) != 2 {
+		return tagStr, ""
 	} else {
-		return sourceName, comp[0], tag.NewImageTag(comp[1], time.Unix(0, 0))
+		return a[0], a[1]
 	}
 }

pkg/image/image_test.go

@@ -33,6 +33,18 @@ func Test_ParseImageTags(t *testing.T) {
 		assert.Equal(t, "gcr.io/jannfis/test-image", image.GetFullNameWithoutTag())
 	})
 
+	t.Run("Parse valid image name with digest tag", func(t *testing.T) {
+		image := NewFromIdentifier("gcr.io/jannfis/test-image@sha256:abcde")
+		assert.Equal(t, "gcr.io", image.RegistryURL)
+		assert.Empty(t, image.ImageAlias)
+		assert.Equal(t, "jannfis/test-image", image.ImageName)
+		require.NotNil(t, image.ImageTag)
+		assert.Empty(t, image.ImageTag.TagName)
+		assert.Equal(t, "sha256:abcde", image.ImageTag.TagDigest)
+		assert.Equal(t, "gcr.io/jannfis/test-image@sha256:abcde", image.GetFullNameWithTag())
+		assert.Equal(t, "gcr.io/jannfis/test-image", image.GetFullNameWithoutTag())
+	})
+
 	t.Run("Parse valid image name with source name and registry info", func(t *testing.T) {
 		image := NewFromIdentifier("jannfis/orig-image=gcr.io/jannfis/test-image:0.1")
 		assert.Equal(t, "gcr.io", image.RegistryURL)
@@ -79,7 +91,7 @@ func Test_WithTag(t *testing.T) {
 		imageName := "jannfis/argocd=jannfis/orig-image:0.1"
 		nimageName := "jannfis/argocd=jannfis/orig-image:0.2"
 		oImg := NewFromIdentifier(imageName)
-		nImg := oImg.WithTag(tag.NewImageTag("0.2", time.Unix(0, 0)))
+		nImg := oImg.WithTag(tag.NewImageTag("0.2", time.Unix(0, 0), ""))
 		assert.Equal(t, nimageName, nImg.String())
 	})
 }

pkg/image/options.go

@@ -75,6 +75,8 @@ func ParseUpdateStrategy(val string) VersionSortMode {
 		return VersionSortLatest
 	case "name":
 		return VersionSortName
+	case "digest":
+		return VersionSortDigest
 	default:
 		log.Warnf("Unknown sort option %s -- using semver", val)
 		return VersionSortSemVer