Enhance Helm with an option to not generate a new IngressClass object

[brianehlert]

One upgrade pattern for services in Kubernetes is a blue/green. We tend to focus on this pattern for backend services behind the ingress controller. However, this is a valid upgrade pattern for the ingress controller itself.

This is currently not possible using the Helm chart provided with this project.
What the Helm chart enforces is that each ingress controller deployment has a unique IngressClass.
While this is true to the IngressClass intention of being a unique management construct, it blocks the possibility of using the blue/green pattern to upgrade the ingress controller service itself.
The additional impact of this behavior is that the customer is not able to generate an IngressClass out of band with the Helm chart, such as part of some automation workflow prior to deploying the ingress controller using the Helm chart.

This is believed to be possible with manifest (not validated) however that does not meet the automation needs when a customer has expressly instrumented the Helm chart.

What I am proposing is some option of the Helm chart to:

• not generate a new ingress class when using the Helm chart to deploy ingress controller
• support adding an additional deployment of ingress controller to an existing IngressClass

[brianehlert]

This article describes the blue/green pattern for an ingress controller starting at Step 1
https://medium.com/codecademy-engineering/kubernetes-nginx-and-zero-downtime-in-production-2c910c6a5ed8
The core customer objective is that a secondary ingress controller deployment is created and both will exist for a short period of time. The customer is not concerned with the problem of two deployments writing back status to the same set of configuration objects. In fact, it is the ability to have one set of configuration objects that both deployments are sharing that makes this pattern compelling in the first place.

ingress-nginx Helm has a way to disable the creation of the IngressClass resource: https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/templates/controller-ingressclass.yaml#L1
Does that address the problem of using one IngressClass for two deployments?

[coolbry95]

We do something similar with the lifecycle.preStop hooks.

    terminationGracePeriodSeconds: 330
    lifecycle:
      preStop:
        exec:
          command: 
            - "/bin/sh"
            - "-c"
            - "sleep 235 && /usr/sbin/nginx -s reload && sleep 90"

Being able to do the blue/green without changing DNS is definitely a helpful feature that I can see us using in the future. I would like to see blue/green in the same namespace without having to change DNS. Using the same service and/or using a new service would also be beneficial. We are migrating from nlb instance targeting to nlb ip targeting and requires recreating the service. We are doing something similar to the DNS change described in your link.

How do you envision having multiple deployments? Would there be multiple configmaps? Are we just deploying the helm chart multiple times in the same namespace?

[brianehlert]

The conversation this came from is based on using a single ingressClass for two deployments and thus also those two deployments sharing all configuration objects. Essentially, duplicates of each other.
The case is specifically about an alternate to rolling / in-place upgrades.

[coolbry95]

I understand not creating the ingressClass and sharing one. How do you propose the second deployment would work though? Is it deployed as another helm release? Like helm install releaseName1; helm install releaseName2?

[brianehlert]

The reference article begins here: https://medium.com/codecademy-engineering/kubernetes-nginx-and-zero-downtime-in-production-2c910c6a5ed8#:~:text=Step%201%3A%20Create%20a%20Second%20Ingress%20Controller

it is a second deployment, in its own namespace, sharing the same class.
I am not a Helm expert. But the reference article is performing helm upgrade --install - I am not deep enough with Helm to know if this is different behavior from helm install

[vepatel]

• Ability to install IC without requiring/creating an ingressClass.
• Attach an existing ingressClass to an IC spec

[brianehlert]

This was delivered: #4333