Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: image updater first class status proposal #11787

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

jaideepr97
Copy link
Contributor

Signed-off-by: Jaideep Rao [email protected]

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).

@codecov
Copy link

codecov bot commented Dec 20, 2022

Codecov Report

Patch and project coverage have no change.

Comparison is base (064c8da) 49.14% compared to head (a5771f7) 49.15%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #11787   +/-   ##
=======================================
  Coverage   49.14%   49.15%           
=======================================
  Files         248      248           
  Lines       42891    42891           
=======================================
+ Hits        21079    21082    +3     
+ Misses      19693    19691    -2     
+ Partials     2119     2118    -1     

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jaideepr97 jaideepr97 marked this pull request as ready for review December 21, 2022 20:25
@jaideepr97
Copy link
Contributor Author

jaideepr97 commented Dec 21, 2022

Hi @jannfis @wtam2018 @alexmt @jessesuen @leoluz @crenshaw-dev @daniel-codefresh

This is a follow up proposal for making Image Updater a first class citizen in Argo CD. It discusses how we envision Image Updater can fit into Argo CD by leveraging the Application CR fields and how this would set it up for future development while providing value to the users and community at large.

All comments/feedback are much appreciated, TIA!

@jaideepr97
Copy link
Contributor Author

jaideepr97 commented Jan 5, 2023

@leoluz @jannfis
Following up on the discussion we had during the contributor meeting, I touch upon the question about image updater as a separate controller within argo-cd vs integrated into the application-controller in the proposal a little bit here. This is how I see it

Benefits of merging image-updater into application-controller:

  • Avoids addition of a new/separate component that needs to be enabled/disabled/managed within an Argo-cd deployment
  • Eliminates concern outlined in Open Questions as there would only be a single component reading/writing to the application resource
  • Provides Image Updater functionality out of the box for all Argo CD installations even more natively
  • Image updater would receive native awareness of all applications in control-plane and non-control plane namespaces

Drawbacks:

  • Loss of separation of concerns between application resource management and image updation that currently exists
  • Adds additional complexity to the application-controller that would now have to interact with image-registries & perform git write-backs as well (alternatively, git write-back functionality may potentially need to be extracted out and added to the repo-server instead)
  • Might impact maintenance/reliability of application-controller code
  • Separate controller would be consistent with how projects like appset and notifications controller have been merged into argo-cd in the past

Please add any other aspects I might have missed or been unaware of that could help better inform this discussion, Thanks!

@jaideepr97 jaideepr97 force-pushed the image-updater-first-class-support-proposal branch from 8f5807f to cb8cfdd Compare January 19, 2023 19:10
Copy link
Collaborator

@leoluz leoluz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the proposal Jaideep! Please consider my comments.

docs/proposals/image-updater-first-class-support.md Outdated Show resolved Hide resolved
docs/proposals/image-updater-first-class-support.md Outdated Show resolved Hide resolved
docs/proposals/image-updater-first-class-support.md Outdated Show resolved Hide resolved
docs/proposals/image-updater-first-class-support.md Outdated Show resolved Hide resolved
Comment on lines 114 to 119
helm:
imageName: <name of helm parameter to set for the image name> # for git write back
imageTag: <name of helm parameter to set for the image tag>
imageSpec: <name of helm parameter to set for canonical name of image>
kustomize:
imageName: <original_image_name> # for git write back
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems redundant with the image values already provided in the imageList.path

Copy link
Contributor Author

@jaideepr97 jaideepr97 Feb 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would need this if users already have specific, non-standard field names for their helm/kustomize parameters right?

Copy link
Contributor Author

@jaideepr97 jaideepr97 Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some additional context: helm.imageTag here is the name of the helm parameter that should hold the value for the updated image tag in spec.source.helm.parameters. So if someone were to set helm.imageTag: someImage.version image updater would know to set spec.source.helm.parameters.someImage.version=<updated_version> in the application spec


```

There is a need to introduce state by tracking the status of image Updates in the application CR. This is mainly important to honor application rollbacks, avoid repeated image updates in case of failure, as well as to support future use cases such as enabling image-updater to create pull requests against the source repositories. The above detailed `.status.imageUpdates` field could be extended to store metadata regarding pull request creation that would serve to avoid duplicate/excessive number of pull requests being opened, for example.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As suggested above, I think it would be great to consider a solution where we don't have to commit back to Git at all in order to have the image update functionality. We could have something similar to how the ignoreDifference configuration works today but used for updating images and defining the current tag.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Git write backs have the advantage of reproducibility and version controlling a crucial component of applications upgrades

docs/proposals/image-updater-first-class-support.md Outdated Show resolved Hide resolved
path: "../../base"

# (optional) credentials for git write-back
secret:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to re-use ArgoCD repo auth (e.g using GitHub App)? or is this something in the roadmap?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I think the goal is to re-use Argo CD provided authentication mechanism by default (and we do it right now, too). Currently, we are lagging a little behind Argo CD's authentication mechanisms, though. But yes, we definitely want to streamline authentication to Git between Image Updater and Argo CD.

forceUpdate: true/false

# specify strategy to be followed when determining which images have qualifying updates
updateStrategy: semver/digest/lexical/most-recently-built
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should most-recently-built match the existing value latest?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

latest will be deprecated in Image Updater by the next iteration and is to be replaced by most-recently-build.

env: DOCKER_HUB_CREDDS

# or use an external script mounted to image-updater FS to generate credentials
ext: <path/to/script>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include credsexpire here as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Copy link
Contributor Author

@jaideepr97 jaideepr97 Apr 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mubarak-j thanks for the feedback!
You mean just a timestamp to know when the credentials expire right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added, thanks for the suggestion!


# (optional) specify helm parameter names to be used for image update write back
helm:
imageName: someImage.image
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is the existing config, but in the future can we consider not requiring helm.imageName if a user is updating only one image?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is that we don't know (or can make an informed guess on) the name of the parameter to set. There is a default that we use, that probably works with a couple of off-the-shelve charts.

- image: quay.io/some/image
lastTransitionTime: <update_timestamp>
oldTag: v1.0.0
newTag: v1.1.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is nice and would help us with more accurate notifications for image updates. currently notifications pickup both tags () during rollout/sync.

- <pattern 1>
- <pattern 2>
forceUpdate: true/false
updateStrategy: semver/digest/lexical/most-recently-built
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should most-recently-built match the existing value latest?

Signed-off-by: Jaideep Rao <[email protected]>
@jaideepr97
Copy link
Contributor Author

@mubarak-j Thanks for taking a look at this!
Just to update here - I have raised a PR to implement the proposed fields in the application API in #12645


2. Change Image updater operational model to become webhook aware - At present the updater polls the configured image registries periodically to detect new available versions of images. In the future Image Updater would need to be able to support webhooks so that it can move to a pub/sub model and updates can be triggered as soon as a qualifying image version is found.

3. Supporting container verification - As described earlier, this could be a useful feature to reduce security risks introduced into Argo CD while providing all the benefits of having automatic image updates available.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: add better support for apps in mono-repo which is currently blocked by argoproj-labs/argocd-image-updater#475 (comment)
I believe mono-repo is a common use case with ArgoCD and this support is crucial for adoption similar to other controllers in ArgoCD.


This section highlights some of the key aspects of Image Updater's present design constraints and behaviors in order to provide context for the upcoming sections of the proposal to people who may not be familiar with the project. Full details may be found in the Image Updater documentation (https://argocd-image-updater.readthedocs.io/en/stable/)

- Image Updater can only update container images for applications whose manifests are rendered using either Kustomize or Helm and - especially in the case of Helm - the templates need to support specifying the image's tag (and possibly name) using a parameter (i.e. image.tag).
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it also support setting the image registry via a separate parameter? There are widely used Helm charts in the wild that build the repository name not just from repository and tag but also registry. An example is prometheus-operator.

@aqeelat
Copy link

aqeelat commented Mar 12, 2024

I glanced over the code reviews and most of them are enhancements that do not alter the spirit of the proposal, and they're not backward incompatible.

  1. Wouldn't it be more productive if we approve the proposal as is and then enhance it in another proposal? This project is way too important to be held up for over a year.
  2. Shouldn't this PR replace docs: proposal to merge image-updater into argo-cd #10447 in the roadmap? I think we should add it there and add a clear target version. I suggest we target 2.11 for finalizing the proposal and 2.13 (or 3) for implementing the proposal.
  3. If the proposal will not be merged soon (within 6 months), why not elect community maintainers to the image updater repo? There are good discussions/PRs happening there and it would be great if these PRs get merged.

@jannfis @jaideepr97 I really appreciate your work and I hate for it to become irrelevant/obsolete.

@idurgakalyan
Copy link

Argo image updater needs support, Can we agree to the above mentioned comment and make a call on roadmap or elect community maintainers to the image updater repo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants