update KEP 740 #4852

HarshalNeelkamal · 2024-09-13T20:32:47Z

One-line PR description: Update KEP for External ServiceAccount JWT signing
Issue link: API for external signing of Service Account tokens #740
Other comments:
- The KEP was marked provisional in KEP-740: fix incorrect status #3653 due to being stale for over 3 years.
- This PR attempts to bring the KEP up to speed with recent changes.
- The intention is that underlying implementation can be picked back up.
- A consensus regarding external JWT signing was achieved in the community (as documented here)

k8s-ci-robot · 2024-09-13T20:32:53Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: HarshalNeelkamal
Once this PR has been reviewed and has the lgtm label, please assign mikedanese for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

keps/sig-auth/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-09-13T20:32:55Z

Welcome @HarshalNeelkamal!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2024-09-13T20:32:56Z

Hi @HarshalNeelkamal. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ahmedtd

One high-level point of discussion... does this need any feature gates? It seems like it is explicitly controlled by a kube-apiserver flag, so there's no point in additionally adding a feature gate.

keps/sig-auth/740-service-account-external-signing/README.md

HarshalNeelkamal · 2024-09-17T22:32:03Z

/cc @liggitt @enj @micahhausler

liggitt · 2024-09-18T17:03:28Z

keps/sig-auth/740-service-account-external-signing/kep.yaml

  - "@liggit"
  - "@tallclair"
 approvers:
-  - "@mikedanese"
+  - "@taahm"
  - "@liggit"


typo: liggitt

liggitt · 2024-09-18T17:03:45Z

keps/sig-auth/740-service-account-external-signing/kep.yaml

 owning-sig: sig-auth
 participating-sigs: []
 reviewers:
-  - "@mikedanese"
+  - "@taahm"
  - "@liggit"
  - "@tallclair"


add @enj @micahhausler as reviewers

liggitt · 2024-09-18T17:06:25Z

keps/sig-auth/740-service-account-external-signing/README.md

-* Initial PR: kubernetes/kubernetes#73110
+Initial PRs: 
+- kubernetes/kubernetes#73110
+- kubernetes/kubernetes#125177


copy all missing sections / questions from https://raw.githubusercontent.com/kubernetes/enhancements/master/keps/NNNN-kep-template/README.md so they're all here, even if they're optional for alpha and we leave them as TODO for now

liggitt · 2024-09-18T17:06:46Z

keps/sig-auth/740-service-account-external-signing/README.md

  - [New API](#new-api)
-  - [Implementation Details/Notes/Constraints](#implementation-detailsnotesconstraints)
+  - [Plugins] (#plugins)
+  - [Updates for token generation](#updates-for-token-generation)


(can run https://github.com/kubernetes/enhancements/blob/master/hack/update-toc.sh to automatically update this)

liggitt · 2024-09-18T17:11:50Z

keps/sig-auth/740-service-account-external-signing/README.md

+Enable integration with alternate key management solutions for service account keys. While at it, make the association between kube-apiserver and service account keys more agile.
+
+Benefits are 2 fold:


can simplify to just list item 1 and 2 alone and drop the preamble

liggitt · 2024-09-18T17:41:17Z

keps/sig-auth/740-service-account-external-signing/README.md

-message SignPayloadRequest {
-  // payload is the content to be signed. JWT headers must be included by the caller
+message SignJWTRequest {
+  // The raw payload to be signed.  A serialized JSON object, no base64 wrapping.


let's make this already be base64 encoded to remove a possible mistake in the external signer encoding (and symmetric with the "Already wrapped in URL-safe base64" fields in the response)

liggitt · 2024-09-18T17:42:13Z

keps/sig-auth/740-service-account-external-signing/README.md

-message SignPayloadRequest {
-  // payload is the content to be signed. JWT headers must be included by the caller
+message SignJWTRequest {
+  // The raw payload to be signed.  A serialized JSON object, no base64 wrapping.
  bytes payload = 1;


let's call this claims to make it clear which bit of the jwt this is

Suggested change

bytes payload = 1;

bytes claims = 1;

liggitt · 2024-09-18T17:43:40Z

keps/sig-auth/740-service-account-external-signing/README.md

+
+message SignJWTResponse {
+  // The header for the JWT.  Already wrapped in URL-safe base64, valid for
+  // directly inserting into the final JWT.


hoist in the docs from https://docs.google.com/document/d/1QVtBX8J2Tk70toefDegXMXigMFzrdQfezjlhBrKR29Y/edit?tab=t.0#bookmark=id.xf2j5nbnuczz with the restrictions kube-apiserver will put on the returned header

// header must contain alg, kid, typ claims. // type must be “JWT”. // alg must be one of the algorithms supported by kube-apiserver (currently RS256, ES256, ES384, ES512).

liggitt · 2024-09-18T17:48:16Z

keps/sig-auth/740-service-account-external-signing/README.md

-  string algorithm = 4;
+  // The timestamp for when it's appropriate to refetch verification keys to 
+  // pick changes if any.
+  google.protobuf.Timestamp suggested_next_fetch_at = 3;


is there a minimum we'd accept here? (what would kube-apiserver do if this is before time.Now()? make sure we don't hot-loop or do something crazy if skew is encountered like that)

is this what informs the GetCacheAgeMaxSeconds() implementation for PublicKeysGetter? something like suggested_next_fetch_at.Sub(time.Now())/time.Second? that would mean the OIDC docs would return shrinking cache maxAge headers as we got closer and closer to the internal refresh time

Need to document how to use this and/or the metrics to safely add new keys and promote them to signing keys in an external signer

liggitt · 2024-09-18T17:58:48Z

keps/sig-auth/740-service-account-external-signing/README.md

+
+- `<package>`: `<date>` - `<test coverage>`
+ -->
+


add an integration test section describing wiring up a real working kube-apiserver configured to run with this signer (KMS KEP might have a good testing section you can mimic)

dims · 2024-09-18T18:02:58Z

/ok-to-test

k8s-ci-robot · 2024-09-18T18:06:07Z

@HarshalNeelkamal: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-enhancements-verify	`fda0a65`	link	true	`/test pull-enhancements-verify`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

micahhausler · 2024-09-18T18:10:28Z

keps/sig-auth/740-service-account-external-signing/README.md

 - [Implementation History](#implementation-history)
 <!-- /toc -->

 ## Summary

-The Kubernetes API server has always read service account keys from disk as the process starts, and kept them in memory for the duration of the server's lifetime. As the API server can now verify and issue projected volume tokens, it would be advantageous to support external signing and verifying of token data over an API, as well as reading public keys from an API.
+Service account keys are used by kube-apiserver for JWT signing and authentication. Keys are stored on the disk and are loaded by kube-apiserver during process start time. Key management for service account keys is entirely within the cluster and is less flexible. This KEP proposes updates that will allow Kubernetes distributions to integrate with key management solutions of their choice (eg: HSMs, cloud KMSes).  


nit:

"... and is less flexible."

Less flexible than what? It might be better to either compare it to something, or just state what it can't/doesn't do (ex: keys are exportable)

micahhausler · 2024-09-18T18:17:13Z

keps/sig-auth/740-service-account-external-signing/README.md

+
+### Risks and Mitigations
+
+- New token generation and validation could suffer a performance difference when reading over a socket, as an external process will be signing data.


nit: the performance change is not really because a separate process is signing material (indeed, the external signer could be optimized and sign in even less CPU cycles than kube-api), its the communication over a socket (and potentially a remote RPC call, but depends on the signer implementation)

update KEP 740

02f5e3e

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 13, 2024

k8s-ci-robot added kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/auth Categorizes an issue or PR as relevant to SIG Auth. labels Sep 13, 2024

k8s-ci-robot requested review from deads2k and mikedanese September 13, 2024 20:32

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 13, 2024

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 13, 2024

ahmedtd reviewed Sep 13, 2024

View reviewed changes

HarshalNeelkamal added 2 commits September 17, 2024 00:07

Address comments on updates to KEP-740: Iteration 1

7a8842a

Address comments on updates to KEP-740: Iteration 2

fda0a65

k8s-ci-robot requested review from enj, liggitt and micahhausler September 17, 2024 22:32

liggitt reviewed Sep 18, 2024

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 18, 2024

micahhausler reviewed Sep 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update KEP 740 #4852

update KEP 740 #4852

HarshalNeelkamal commented Sep 13, 2024 •

edited

Loading

k8s-ci-robot commented Sep 13, 2024

k8s-ci-robot commented Sep 13, 2024

k8s-ci-robot commented Sep 13, 2024

ahmedtd left a comment

HarshalNeelkamal commented Sep 17, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

liggitt Sep 18, 2024

dims commented Sep 18, 2024

k8s-ci-robot commented Sep 18, 2024

micahhausler Sep 18, 2024

micahhausler Sep 18, 2024

		Enable integration with alternate key management solutions for service account keys. While at it, make the association between kube-apiserver and service account keys more agile.

		Benefits are 2 fold:


		### Risks and Mitigations

		- New token generation and validation could suffer a performance difference when reading over a socket, as an external process will be signing data.

update KEP 740 #4852

Are you sure you want to change the base?

update KEP 740 #4852

Conversation

HarshalNeelkamal commented Sep 13, 2024 • edited Loading

k8s-ci-robot commented Sep 13, 2024

k8s-ci-robot commented Sep 13, 2024

k8s-ci-robot commented Sep 13, 2024

ahmedtd left a comment

Choose a reason for hiding this comment

HarshalNeelkamal commented Sep 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dims commented Sep 18, 2024

k8s-ci-robot commented Sep 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HarshalNeelkamal commented Sep 13, 2024 •

edited

Loading