Handle secret read clusterroles

[mikkeloscar]

Improves handling of secret read cluster roles in EKS clusters

The implementation is described as follows:

1. Extend admission-controller with a clusterrole admitter which detects clusterroles that defined secret read permissions ({ApiGroups: [""], Resources:["secrets"],Verbs:["get|list|watch"]}). If detected, it extracts the permissions from the role and moves them to an annotation: clusterroles.admission-controller.zalando.org/secret-read-verbs: "get,list,watch". This achieves two things 1) Users can't deploy clusterroles with secret read permissions because those permissions will be removed during admission. 2) The role gets annotated such that it can be detected if the users intended to have read secret permissions. This is implemented in admission-controller: Update to version master-244 #8987
2. Extend the role-sync-controller to look for clusterroles withs the annotation: clusterroles.admission-controller.zalando.org/secret-read-verbs: "get,list,watch" and resolve the clusterrolebindings binding to those roles. With the bindings we get a list of subjects that want read secret permissions at cluster level and those subjects can be added to the per namespace secrets role bindings. This is implemented in role-sync-controller: Update to version main-7 #8986
3. Allow read secret permissions at the cluster level for poweruser/CDP. This effectively allows users to apply a clusterrole with read secret permissions, but because of 1 and 2, the clusterrole is not applied in that way but the subject(s) binding to the clusterrole will instead bind to the per namespace secret reader roles and the users get effectively the same permissions as in non-EKS, without changing their manifests.
4. Lastly, to prevent powerusers from reading from kube-system we remove secret read permissions from the cluster-role and only attach those permissions to the cdp/deployment-service service accounts. This means a poweruser, user, e.g. Manual access, can't create a cluster-role with read secret permissions. This is however still possible via CDP.

TODO

• Add e2e covering this feature.

[demonCoder95]

Add e2e covering this feature

Don't the admission-controller unit tests cover this already? What kind of coverage are you thinking about for e2e tests? 🤔

[mikkeloscar]

Add e2e covering this feature

Don't the admission-controller unit tests cover this already? What kind of coverage are you thinking about for e2e tests? 🤔

Nvm, the role-sync-controller was not working correctly making some e2e fail. This was fixed in #9005

There is one thing which is not covered by e2e, will track it in an internal issue:

With this change the cdp and deployment-service can read secrets from kube-system because they need this access to allow users to deploy read secret cluster roles. This can not be defined as a generic test as it would fail on non-eks where it's not allowed.

[mikkeloscar]

👍

[linki]

👍

[mikkeloscar]

👍