[KEP-3104] Update KEP with credential plugin allowlist

[pmengelbert]

• One-line PR description: Update kuberc KEP with credential plugin allowlist

• Issue link: Separate kubectl user preferences from cluster configs #3104

• Other comments:

Currently, the kubeconfig may specify arbitrary binaries to run as
client-go credential plugins. Due to the fact that kubeconfig files are
often generated, and because they contain a lot of noise, there is
significant friction in manually inspecting the kubeconfig for
suspicious binaries after it is generated.

To encourage secure behavior, we want to introduce an allowlist to the
kuberc, which will describe the conditions under which a binary plugin
may be run. Currently the only condition is the name (absolute path or
basename of a binary found in PATH).

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: pmengelbert / name: Peter Engelbert (536c62b)

[k8s-ci-robot]

Welcome @pmengelbert!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @pmengelbert. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[enj]

/sig auth

[enj]

This LGTM from a SIG Auth perspective (I added us as a participating SIG).

[enj]

/ok-to-test

[ardaguclu]

Overall looks good to me. I'll defer the final approval from SIG-CLI POV to @soltysh

[ardaguclu]

is there any other command that will be exceptional like kubectl config view?

[pmengelbert]

Off the top of my head, there are
kubectl config (any subcommand)
kubectl version
kubectl plugin list

Should I update the KEP with a full list?

[liggitt]

hmm... I think it's going to be hard to enumerate every specific command that won't actually use the client. There's lots of commands that only run locally with specific options (e.g. all kubectl create ... --dry-run=client commands, or kubectl {label,annotate,set} --local)

I think the implementation should be structured so that it intercepts attempts to use a non-allowlisted credential plugin and rejects them at time of use, so commands that don't even need the plugin don't start failing for no reason.

[ardaguclu]

I think the implementation should be structured so that it intercepts attempts to use a non-allowlisted credential plugin and rejects them at time of use, so commands that don't even need the plugin don't start failing for no reason.

So that significantly changes the implementation. Maybe we just need to get the list of plugins (allowed, disallowed, etc.) in kubectl/cmd and pass it to client-go (or cli-runtime) to accept/reject this credential exec plugin, when there is a request to API Server?

[soltysh]

I don't think this will change the implementation that much. Per my understanding, that was how it was going to work from start.

[pmengelbert]

I think the implementation should be structured so that it intercepts attempts to use a non-allowlisted credential plugin and rejects them at time of use, so commands that don't even need the plugin don't start failing for no reason.

So that significantly changes the implementation. Maybe we just need to get the list of plugins (allowed, disallowed, etc.) in kubectl/cmd and pass it to client-go (or cli-runtime) to accept/reject this credential exec plugin, when there is a request to API Server?

Per recent discussion in the sig-api-machinery biweekly, they are in agreement with the following:

1. The allowlist can be specified in kuberc as described by this KEP so that kubectl may benefit from it.
2. The allowlist will be parsed by the kuberc parser.
3. The allowlist parsing logic should be made public if it is not already, such that non-kubectl clients can use it to read allowlists configured there.
4. client-go will be modified to permit a hook accepting a parsed allowlist on client creation.
5. Frontends other than kubectl may use this hook to use the allowlist
6. Go clients can use the hook as they wish
7. The allowlist will be plumbed to the point of plugin execution and will be consulted before the exec call.

[ardaguclu]

That sounds like a reasonable plan to me.

[soltysh]

This all sounds exactly what I was imagining it would, thank you!

[ardaguclu]

I think, this PR needs a rebase to remove the invalid-commit label. Otherwise, this PR can't be merged.

[pmengelbert]

go ahead and squash down commits and take out the @mention in the commit message tripping up the presubmit check

@liggitt done

[pmengelbert]

The threat is from downloaded kubeconfigs. A user implicitly trusts the tools (ex the aws, az, etc cli tools, or kind) that generate kubeconfigs -- if they are malicious then there is already a malicious process running on the user's system and the kuberc configuration shouldn't be expected to protect the user.

I'm not sure what this use case looks like TBH, I have only ever locally generated with eg gcloud, kind, minikube, eksctl, ...

An example: a CI pipeline creates a cluster on-demand and generates a kubeconfig using bespoke tools. That generated kubeconfig is produced as an artifact for a cluster user to download. Now the bespoke tooling and the pipeline are targets because they generate a kubeconfig; the allowlist protects the user of that kubeconfig from compromised pipelines or tooling.

I think we should be clear about how we expect it to be used safely though.

Agreed, I will add language clarifying this.

Maybe SIG Security folks can take a look.

/sig security
(Does that work?)

[pmengelbert]

All comments have been addressed and this is ready for another round of review.

[liggitt]

/lgtm
/approve

[pmengelbert]

/assign mpuckett159
pinging @mpuckett159 for review

Not sure if I should wait until there's a review from someone in sig-cli-leads to request your review, but given the looming enhancements freeze I figured better safe than sorry.

[ardaguclu]

I'm approving from sig-cli POV
/approve
I'm also adding hold to give a chance to @soltysh for one last check
/hold

[soltysh]

I left there one comment we can discuss during implementation, not blocking this document, though.

/lgtm
/approve

[soltysh]

Configuration error should error out during parsing not during execution. We already have a separate validation step which I'd expect to be extended with validation for this new addition.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ardaguclu, liggitt, pmengelbert, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/sig-cli/OWNERS [ardaguclu,soltysh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[soltysh]

/hold cancel