Implement KEP753 initContainer restartPolicy=Always sidecar

[teejaded]

What problem are you trying to solve?

Now that we have kubernetes/enhancements#753 (aka native sidecar containers https://kubernetes.io/blog/2023/08/25/native-sidecar-containers/) can the proxy injector be updated to support it when it is available or via flag?

I would comment in #1869, but it is locked.

This would greatly simplify adopting linkerd with jobs.

How should the problem be solved?

Implement support for the new sidecar pattern into the proxy-injector process and linkerd cli.

Any alternatives you've considered?

Adding the shutdown hook call to all jobs is not easy. This is especially true when you're consuming public helm charts.

How would users interact with this feature?

Either opt-in or opt-out of the new sidecar behavior with a flag gated on a k8s api version check.

Would you like to work on this feature?

maybe

[teejaded]

My initial results just moving the proxy container into initContainers looks good so far!

https://github.com/teejaded/linkerd2/commits/native-sidecar

❯ kubectl get pod nginx-5bf8965ddf-948qs -o yaml | yq '.status.initContainerStatuses[1]'
containerID: containerd://18d6acc27364f8c9118a7fa27dd8bd28f437c65cda3b05f5fdb9e41ec76a9c3d
image: cr.l5d.io/linkerd/proxy:stable-2.14.0
imageID: cr.l5d.io/linkerd/proxy@sha256:8bdf507a00c154b3fa0916acc98d80bfdcb0fb8ca1fa3d1e12cbba82604ef23e
lastState: {}
name: linkerd-proxy
ready: true
restartCount: 0
started: true
state:
  running:
    startedAt: "2023-10-06T19:17:18Z"

[aquam8]

Can't wait for it!