Support credentials forwarding in Kubernetes Access

[tigrato]

This PR introduces a new process for forwarding the user's identity for Kubernetes Access.

The identity forwarding is only enabled if all the kube_servers support the new impersonation mechanism and if the request isn't forwarded into a remote cluster. Remote clusters still use mTLS where the user identity is embedded in a proxy-cached certificate.

Only proxies are allowed to impersonate users.

In Teleport 14 we will remove all the mTLS code and we will support impersonation only.

The following benchmarks show the improvement between old and new versions.

Old: mTLS benchmark

Histogram

Percentile Response Duration 
---------- ----------------- 
25         31 ms             
50         32 ms             
75         33 ms             
90         33 ms             
95         37 ms             
99         44 ms             
100        88 ms   

New: Impersonation benchmark

Histogram

Percentile Response Duration 
---------- ----------------- 
25         10 ms             
50         11 ms             
75         12 ms             
90         15 ms             
95         15 ms             
99         20 ms             
100        23 ms     

Part of #22533
Closes #21609

[espadolini]

I think it might be legal for a v14 cluster to have a v13 leaf with v12 proxies, so maybe we should delete this in v15?

[tigrato]

It won't be a problem.

the root proxy will dial to the leaf proxy using the new impersonated method. Leaf proxy will dial to the kubernetes service using the old method since kubernetes service does not support it.

[espadolini]

The issue is that I don't think that we specify how old the components of the leaf cluster can be; if we interpret "leaf cluster version" to mean both auth and proxy (which is the sensible interpretation, I think) then you're right, we can delete it in v14.

The alternative would be that the leaf auth is one version behind and then the leaf proxy is one version behind its auth, but that would mean that the remote cluster reverse tunnel needs to stay compatible for two major versions of the proxy, which is kind of a tall order.