Feature Request: Ability to use a custom TLS certificate with the Ingress addon

[cpaika]

Environment:
Minikube on Mac OS X

What happened:

I am trying to load NGINX-Ingress with a custom tls certificate, in this case a trusted TLS certificate I generated on my machine. I start minikube, then enable the ingress-nginx addon which deploys nginx to the kube-system namespace successfully. I create my certificate in the kube-system namespace with the title mkcert.

I then modified the nginx deployment to have the arg --default-ssl-certificate=kube-system/mkcert. NGINX restarts, and successfully loads the new certificate as I see this in the logs:
6 backend_ssl.go:66] Adding Secret "kube-system/mkcert" to the local store

I then deploy my application which contains an ingress resource, and the Nginx pods restart. When the pods restart, the deployment somehow removes the --default-ssl-certificate=kube-system/mkcert flag and nginx logs show that it is self generating a certificate.

When I try to access my application, it's now using the fake certificate.

If I then deploy the real certificate to the cluster after my application is deployed, it works successfully and I can access my application.

What you expected to happen:
The Minikube control loop does not reset the default TLS certificate

Somehow Minikube is resetting the deployment object to its initial state and losing an argument but I have no idea how.

How to reproduce it:
The following steps require mkcert and minikube:

minikube start --vm-driver=virtualbox
minikube addons enable ingress
mkcert -key-file key.pem -cert-file cert.pem '*.local.ctct.net'
kubectl -n kube-system create secret tls mkcert --key key.pem --cert cert.pem
rm key.pem cert.pem
kubectl -n kube-system patch deployment ingress-nginx-controller --type "json" -p "[{\"op\":\"add\",\"path\":\"/spec/template/sp

Then deploy a test application to the cluster with an ingress resource and nginx will drop the default certificate.

Is there any interest in making custom TLS certificates possible in Minikube with the ingress add on?

[cpaika]

@tstromberg I could actually take this on if you think it's a good first issue. I'm new to the codebase and am still learning Go, any tips on how I can get started here?

I know I need to conditionally add an entry here to point to a TLS certificate: https://github.com/kubernetes/minikube/blob/master/deploy/addons/ingress/ingress-dp.yaml.tmpl#L59

I'm a little unsure of the best approach - since the file is a .tmpl I'm assuming is the best way to add some go templating and then fill in the value when the file gets built?

[tstromberg]

@cpaika - I don't know enough about ingress & custom TLS certs to say, but I think you are on the right path. That's the file you'll need to edit to alter ingress behavior, anyways.

For an example of an addon that has advanced configuration options, see cmd/minikube/cmd/config/configure.go, for example, the support for the metallb addon. Spoiler alert, it isn't pretty, but it is straight-forward.

If you happen to need to dynamically pass data into the template, see GenerateTemplateData in pkg/minikube/assets/addons.go.

[cpaika]

@tstromberg Awesome, thanks for the pointers! I'll get started on this. If I have any future questions, what's the best place to ask in this thread or in slack?

[msedzins]

@cpaika Hi there! are you still working on this? if no - I can take this on.

[cpaika]

@msedzins Yeah I'm not, I wanted to take this on but life got in the way. Its all yours!

[msedzins]

@cpaika Pull request was merged. I think we can close this issue.

[cpaika]

@msedzins Awesome, sounds good!