Support for Custom Annotations on Linkerd-Viz Prometheus Deployment

[cemenson]

Support for Custom Annotations on Linkerd-Viz Prometheus Deployment

Currently, adding custom annotations to the prometheus deployment isn't supported. This can't be done either with --set helm flag or by adding relevant config to values.yaml.

By adding additional helm values by way of a with keyword within the prometheus deployment yaml, we can support custom annotations. Setting the value of the reverenced value (.Values.prometheusAnnotations) to empty will ensure this doesn't break or modify any existing deployments.

Once merged, setting prometheusAnnotations in the values.yaml or via a --set helm flag will result in specified annotations being included in the prometheus deployment and pod spec.

Fixes #11365

I agree to the DCO for all the commits in this PR.

[cemenson]

@alpeb is this good to merge?

[alpeb]

Thanks @cemenson, this looks good to me.

[cemenson]

@adleong could I trouble you for a review of this?

[adleong]

Hi @cemenson, can you tell me more about the use case? What kinds of annotations do you need to add to the Prometheus deployment and why? I'll note that we already have a podAnnotations value which can be used to set annotations on the pods themselves. It would be great to understand in what scenarios annotations need to be added to the deployement vs added to the pod template.

[cemenson]

Hi @cemenson, can you tell me more about the use case? What kinds of annotations do you need to add to the Prometheus deployment and why? I'll note that we already have a podAnnotations value which can be used to set annotations on the pods themselves. It would be great to understand in what scenarios annotations need to be added to the deployement vs added to the pod template.

Hi @adleong. For our specific case, we need to set an annotation config.linkerd.io/skip-inbound-ports on the prometheus pod. There is no way to do this as part of the helm install, other than as you said setting the podAnnotations value. Whilst this would work, it would also apply (as I understand from the chart) the annotation to all the other pods that are part of the linkerd-viz chart (tap, web, etc.), which is not desired.

[adleong]

thanks @cemenson, that context is super helpful!

How would you feel about renaming the value to prometheus.podAnnotations and having it apply just to the prometheus pod template and not the deployment itself?

[cemenson]

thanks @cemenson, that context is super helpful!

How would you feel about renaming the value to prometheus.podAnnotations and having it apply just to the prometheus pod template and not the deployment itself?

Yes that's fine @adleong. I've made those changes now.

[adleong]

Thanks, @cemenson. I'm seeing some weird CI failures that I can't reproduce locally. Could you try pulling the latest main and merging it into this branch?

Edit: I've merged main, let's see if that resolves the CI issues.

[adleong]

🎉 🎉 🎉