-
Couldn't load subscription status.
- Fork 15.1k
Refactor Certificates Documentation #51487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
👷 Deploy Preview for kubernetes-io-vnext-staging processing.
|
✅ Pull request preview available for checkingBuilt without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
/retitle [WIP] Document Pod certificate requests |
79b343d to
6ce8b70
Compare
|
/assign stlaz |
f35bb0a to
8363ffd
Compare
|
Hello @ahmedtd 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday July 29th 2025 18:00 PST. Thank you! |
4268852 to
9a8f68c
Compare
3400449 to
8ba16fc
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/remove-language es hi id ja pt ru uk zh |
|
/label refactor |
| override_link_text: "CSR v1" | ||
| - apiVersion: "certificates.k8s.io/v1beta1" | ||
| kind: "ClusterTrustBundle" | ||
| - apiVersion: "certificates.k8s.io/v1alpha1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For a PR targeting main we can't accept this yet.
Docs are continuously deployed and main should document the latest (released) minor release, ie v1.33 at time of writing.
/hold
We can't merge this yet but we can do everything bar that.
| client certificate. | ||
|
|
||
| <!-- body --> | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here, let's do a stub that mentions two common certificate tasks, and signposts people to go there:
- set up TLS for your workload, with a valid cert
- rotate the certs your existing cluster uses - see issue Document requirements/recommended process for updating cluster TLS certs/keys #30575
A stub is all it needs.
| this 15-minute limit. | ||
|
|
||
|
|
||
| ## ClusterTrustBundles {#cluster-trust-bundles} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would introduce these before the other API kinds in this document. They are in away useful without any of the other concepts: you can use them to distribute trust anchors to your workloads. That's a common need.
|
|
||
| All ClusterTrustBundle objects have strong validation on the contents of their | ||
| `trustBundle` field. That field must contain one or more X.509 certificates, | ||
| DER-serialized, each wrapped in a PEM `CERTIFICATE` block. The certificates |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: could hyperlink to the RFC that standardized certs in PEM format.
| kubectl get clustertrustbundles --as='system:serviceaccount:mynamespace:default' | ||
| ``` | ||
|
|
||
| ### Signer-linked ClusterTrustBundles {#ctb-signer-linked} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can have this even if we haven't fully explained what signers are. An initial outline and are hyperlink could be enough.
| See the [clusterTrustBundle projected volume source](/docs/concepts/storage/projected-volumes#clustertrustbundle) for more details. | ||
|
|
||
|
|
||
| ## Kubernetes built-in signers {#kubernetes-signers} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would have this list of signers in the reference section of the docs.
| With the `AuthorizeWithSelectors` feature enabled, field and label selectors in the request | ||
| are passed to the authorization webhook. The webhook can make authorization decisions | ||
| When calling out to an authorization webhook, Kubernetes passes | ||
| label and field selectors in the request to the authorization webhook. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: optionally, make "selectors" a glossary_tooltip
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the biggest thing missing in the projected volume docs is details on private key generation. Users who are regulated or just care about PKI will want to know what process is generating it, where its located, and any options on key size/algo/if you can use hardware generation/etc
| --- | ||
| reviewers: | ||
| - liggitt | ||
| - mikedanese |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this autogenerated based on OWNERS? I'm happy to be a reviewer here instead of Mike
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's been copy and pasted. There's no automatic maintenance that I know of.
| * CertificateSigningRequest: An API type that lets you request an arbitrary | ||
| certificate from a signer. | ||
| * PodCertificateRequest: (alpha) An API type that lets `kubelet` request | ||
| certificates to inject into a Pod. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
| certificates to inject into a Pod. | |
| certificates to mount into a Pod. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think inject is better, actually. Today it's only supported using a mount, but in a future minor release we might offer other ways to use the cert and private key (example: your sandbox sees it in a kernel-provided keyring).
|
|
||
| {{< feature-state feature_gate_name="ClusterTrustBundleProjection" >}} | ||
|
|
||
| The contents of ClusterTrustBundles can be injected into the container filesystem, similar to ConfigMaps and Secrets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| The contents of ClusterTrustBundles can be injected into the container filesystem, similar to ConfigMaps and Secrets. | |
| The contents of ClusterTrustBundles can be mounted into the container filesystem, similar to ConfigMaps and Secrets. |
| * ClusterTrustBundle: (beta) A group of certificates (typically CA root | ||
| certificates) that should be used as roots of trust for verifying a server or | ||
| client certificate. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| * ClusterTrustBundle: (beta) A group of certificates (typically CA root | |
| certificates) that should be used as roots of trust for verifying a server or | |
| client certificate. | |
| * ClusterTrustBundle: (beta) A list of certificates (typically CA root | |
| certificates) that can be used as a root of trust for verifying a server or | |
| client certificate. |
| example, `kubernetes.io/foo`). Each signer uniquely identifies a category of | ||
| certificates that can be issued within the scope of a particular cluster. | ||
| Kubernetes offers some [built-in signers](#kubernetes-signers), and third-party | ||
| signers may also be freely installed into a cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: 'installed' typically implies a pod running in the cluster, but a signer just needs to be an API client, and can be hosted externally
| signers may also be freely installed into a cluster. | |
| signers may also be used with a cluster. |
| signer-linked ClusterTrustBundles or ConfigMaps | ||
|
|
||
| The Kubernetes project provides a set of [built-in signers](#kubernetes-signers) | ||
| that are available in every cluster. It is also possible to install (or build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| that are available in every cluster. It is also possible to install (or build | |
| that are available in every cluster. It is also possible to use (or build |
|
|
||
| Once created, a CSR must be approved before it can be signed. Approval is | ||
| indicated by adding an `Approved` entry to `status.conditions`. Denial is | ||
| indicated by adding a `Denied` entry to `status.conditions`. Both of these |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: My understanding is that you wouldn't have 2 contradictory conditions in the same list, I'd suggest making it "either" or just "These conditions..."
| indicated by adding a `Denied` entry to `status.conditions`. Both of these | |
| indicated by adding a `Denied` entry to `status.conditions`. These |
|
|
||
| To ensure that terminal PodCertificateRequests do not build up in the cluster, a | ||
| `kube-controller-manager` controller deletes all PodCertificateRequests older | ||
| than 15 minutes. All certificate issuance flows are expected to complete within |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might want to specify how to configure this. I can think of very large clusters where this timeout might be an issue
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIRC this currently cannot be configured.
|
|
||
| ClusterTrustBundle objects should be considered world-readable within the | ||
| cluster. If your cluster uses [RBAC](/docs/reference/access-authn-authz/rbac/) | ||
| authorization, all ServiceAccounts have a default grant that allows them to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😬 Is this a removable RBAC permission?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can edit the clusterrolebinding and opt-out of its auto-reconcile per https://kubernetes.io/docs/reference/access-authn-authz/rbac/#auto-reconciliation.
Just curious - what's the use case? Are you trying to prevent undesired trust, perhaps?
|
|
||
| There is no built-in auto-approval controller for this signer; all certificates | ||
| must be manually approved, or a third-party auto-approval controller must be | ||
| installed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| installed. | |
| used. |
| * CertificateSigningRequest: An API type that lets you request an arbitrary | ||
| certificate from a signer. | ||
| * PodCertificateRequest: (alpha) An API type that lets `kubelet` request | ||
| certificates to inject into a Pod. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think inject is better, actually. Today it's only supported using a mount, but in a future minor release we might offer other ways to use the cert and private key (example: your sandbox sees it in a kernel-provided keyring).
| {{< note >}} | ||
| When encoded in JSON or YAML, `status.certificate` field is base-64 encoded, so | ||
| it is not directly readable. This is handled transparently when using a client | ||
| library. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that depends on exactly which client library you use, and we can't confidently assume every user has picked the one we were thinking of.
| The three API types for requesting and trusting certificates are: | ||
| * CertificateSigningRequest: An API type that lets you request an arbitrary | ||
| certificate from a signer. | ||
| * PodCertificateRequest: (alpha) An API type that lets `kubelet` request |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't hardcode feature maturity, find a way to incorporate {{< feature-state feature_gate_name="NAME_HERE" >}}. This is otherwise hard to find and replace when the feature matures further.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can go too far with making maintenance low toil. Here's how I'd strike a balance.
- keep this wording; leave in “(alpha)”
- just afterwards, add an HTML comment that mentions the feature gate name (which is
PodCertificateRequests, plural) - edit
content/en/docs/reference/command-line-tools-reference/feature-gates/PodCertificateRequests.mdwith another HTML comment to search for the feature gates in the English docs and tidy up any matches, on graduation
That's enough of a hint to make maintenance easy, but also makes live comfortable for readers. For the K8s docs, we have many many more readers than writers, so we should prefer choices that impose lower costs on readers, provided the effort to do so is reasonable.
HTML comments are an easily-understood way to leave notes for contributors, that don't harm the end user experience. And they work in Markdown.
Bear in mind that my suggestion won't 100% work out, yet, because this PR is a work in progress (either it targets the wrong branch, or it shouldn't mention PodCertificateRequest yet). But you get the idea.
| Kubernetes certificate and trust bundle APIs enable automation of | ||
| [X.509](https://www.itu.int/rec/T-REC-X.509) credential provisioning by | ||
| providing a programmatic interface for clients of the Kubernetes API to request | ||
| and obtain X.509 {{< glossary_tooltip term_id="certificate" text="certificates" | ||
| >}} from a Certificate Authority (CA). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This paragraph is a little odd. In short it reads as "Kubernetes cert/trust bundle APIs enable credential provisioning by providing APIs for clients of these APIs". Try to rephrase somehow, please.
| The CSR flow begins when a requester creates a CSR object: | ||
| * `spec.signerName` says which signer should process the CSR. | ||
| * `spec.request` contains a PEM-wrapped PKCS#10 signing request. | ||
| * `spec.expirationSeconds` (optional) allows the requester to ask for a specific | ||
| lifetime for the issued certificate. | ||
| * `spec.usages` contains the key usages that the requester wants on the issued | ||
| certificate. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we leave specific fields docs here to the API reference and only mention what the CSR object is capable of? Alternatively mention that this list is not exhaustive and ref the docs from above for further details.
| be approved. Broadly speaking: | ||
| * Some signers automatically approve conforming CSRs, and will refuse to issue a | ||
| certificate to a non-conforming CSR. These signers will move an approved, but | ||
| non-conforming CSR into the `Failed` terminal state, rather than issuing it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably mention the "Failed" condition, otherwise the terminal state is not properly defined.
| {{< /note >}} | ||
|
|
||
| Signer implementations are encouraged to: | ||
| * Document whether or not they consider manual approval decisions on |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This?
| * Document whether or not they consider manual approval decisions on | |
| * Document whether or not they require manual approval decisions on |
They would always consider manual approvals, right? Maybe unless they approve and issue in a single step?
| The `podCertificate` projected volumes source securely provisions a private key | ||
| and X.509 certificate chain for pod to use as client or server credentials. | ||
| Kubelet will then handle refreshing the private key and certificate chain when | ||
| they get close to expiration. The application just has to make sure that it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit
| they get close to expiration. The application just has to make sure that it | |
| they get close to expiration. The application needs to be able to react to these file changes |
| You can also introduce your own custom signer, which should have a similar | ||
| prefixed name but using your own domain name. For example, if you represent an |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is its own page, the "similar prefixed name" gets confusing as you don't know what it should be similar to.
| You can also introduce your own custom signer, which should have a similar | |
| prefixed name but using your own domain name. For example, if you represent an | |
| Kubernetes lets you introduce your own custom signer that should have a similar (?) | |
| prefixed name but using your own domain name. For example, if you represent an |
|
|
||
| #### CertificateSigningRequests {#useful-cluster-roles-csr} | ||
|
|
||
| CertificateSigningRequests have three roles — requesters, approvers, and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| CertificateSigningRequests have three roles — requesters, approvers, and | |
| There are three roles to consider with CertificateSigningRequests — requesters, approvers, and |
| Allowing broad read access to CSRs is not a security issue, because the CSRs | ||
| only contain public (not private) keys. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not on us to decide what is and isn't a security issue. Some might even consider being able to see who requests which certs as a security issue. Think leaking service hostnames in a multi-tenant cluster, for example.
| {{< mermaid >}} | ||
| stateDiagram-v2 | ||
|
|
||
| direction LR | ||
|
|
||
| Initial --> Wait | ||
| Wait --> Fresh | ||
| Wait --> Failed | ||
| Wait --> Denied | ||
| Fresh --> WaitRefresh | ||
| WaitRefresh --> Failed | ||
| WaitRefresh --> Denied | ||
|
|
||
| {{< /mermaid >}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we want to highlight start/final states? Does that work in our docs?
This comment was marked as off-topic.
This comment was marked as off-topic.
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
|
||
| {{< feature-state feature_gate_name="ClusterTrustBundle" >}} | ||
|
|
||
| This signer does not issue any certificates. It purely serves to distribute the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It still issues certificates but that is all done externally, not automated by the vanilla platform.
|
|
||
| This signer does not issue any certificates. It purely serves to distribute the | ||
| trust anchors that allow workloads within the cluster to verify the API server's | ||
| server certificate when they connect to `kubernetes.default.svc`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kubernetes.defaults.svc is only one of the ways to connect to the kube-apiserver. See https://github.com/kubernetes/website/pull/48492/files#diff-6d12239777c4a9144a2fb9051645d7ace1f7d2a4a79f28e819c1dafb4bde0abeR219-R220
| trust anchors that allow workloads within the cluster to verify the API server's | ||
| server certificate when they connect to `kubernetes.default.svc`. | ||
|
|
||
| There are three built-in distribution mechanisms for these trust anchors: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should mention all the other expectations that we have on the certs signed by this signer. While they are all not necessarily enforced, they could still be considered best practice, which is worth documenting. See what I had in #48492, I believe most of the doc there was agreed on when I was updating the KEP.
The "Certificates" page is creaking under its own weight, and is due to be split up. This is being pursued in a separate PR: kubernetes#51487
This change breaks apart the "Certificates" reference documentation (which currently lives under the API Access, Authentication, and Authorization section of the reference documentation. It becomes: