diff --git a/content/en/docs/tutorials/_index.md b/content/en/docs/tutorials/_index.md index fdc62e11fb76c..0f1181e2cf930 100644 --- a/content/en/docs/tutorials/_index.md +++ b/content/en/docs/tutorials/_index.md @@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the * [Using Source IP](/docs/tutorials/services/source-ip/) +## Security + +* [Apply Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/) +* [Apply Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/) + ## {{% heading "whatsnext" %}} If you would like to write a tutorial, see diff --git a/content/en/docs/tutorials/security/_index.md b/content/en/docs/tutorials/security/_index.md new file mode 100644 index 0000000000000..fbb8140dfadf6 --- /dev/null +++ b/content/en/docs/tutorials/security/_index.md @@ -0,0 +1,5 @@ +--- +title: "Security" +weight: 40 +--- + diff --git a/content/en/docs/tutorials/security/cluster-level-pss.md b/content/en/docs/tutorials/security/cluster-level-pss.md new file mode 100644 index 0000000000000..e2d37b765a653 --- /dev/null +++ b/content/en/docs/tutorials/security/cluster-level-pss.md @@ -0,0 +1,324 @@ +--- +title: Apply Pod Security Standards at the Cluster Level +content_type: tutorial +weight: 10 +--- + +{{% alert title="Note" %}} +This tutorial applies only for new clusters. +{{% /alert %}} + +Pod Security admission (PSA) is enabled by default in v1.23 and later, as it has +[graduated to beta](/blog/2021/12/09/pod-security-admission-beta/). +Pod Security +is an admission controller that carries out checks against the Kubernetes +[Pod Security Standards](docs/concepts/security/pod-security-standards/) when new pods are +created. This tutorial shows you how to enforce the `baseline` Pod Security +Standard at the cluster level which applies a standard configuration +to all namespaces in a cluster. + +To apply Pod Security Standards to specific namespaces, refer to [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss). + +## {{% heading "prerequisites" %}} + +Install the following on your workstation: + +- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) + +## Choose the right Pod Security Standard to apply + +[Pod Security Admission](/docs/concepts/security/pod-security-admission/) +lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/) +with the following modes: `enforce`, `audit`, and `warn`. + +To gather information that helps you to choose the Pod Security Standards +that are most appropriate for your configuration, do the following: + +1. Create a cluster with no Pod Security Standards applied: + + ```shell + kind create cluster --name psa-wo-cluster-pss --image kindest/node:v1.23.0 + ``` + The output is similar to this: + ``` + Creating cluster "psa-wo-cluster-pss" ... + ✓ Ensuring node image (kindest/node:v1.23.0) đŸ–ŧ + ✓ Preparing nodes đŸ“Ļ + ✓ Writing configuration 📜 + ✓ Starting control-plane 🕹ī¸ + ✓ Installing CNI 🔌 + ✓ Installing StorageClass 💾 + Set kubectl context to "kind-psa-wo-cluster-pss" + You can now use your cluster with: + + kubectl cluster-info --context kind-psa-wo-cluster-pss + + Thanks for using kind! 😊 + + ``` + +1. Set the kubectl context to the new cluster: + + ```shell + kubectl cluster-info --context kind-psa-wo-cluster-pss + ``` + The output is similar to this: + + ``` + Kubernetes control plane is running at https://127.0.0.1:61350 + + CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy + + To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. + ``` + +1. Get a list of namespaces in the cluster: + + ```shell + kubectl get ns + ``` + The output is similar to this: + ``` + NAME STATUS AGE + default Active 9m30s + kube-node-lease Active 9m32s + kube-public Active 9m32s + kube-system Active 9m32s + local-path-storage Active 9m26s + ``` + +1. Use `--dry-run=server` to understand what happens when different Pod Security Standards + are applied: + + 1. Privileged + ```shell + kubectl label --dry-run=server --overwrite ns --all \ + pod-security.kubernetes.io/enforce=privileged + ``` + The output is similar to this: + ``` + namespace/default labeled + namespace/kube-node-lease labeled + namespace/kube-public labeled + namespace/kube-system labeled + namespace/local-path-storage labeled + ``` + 2. Baseline + ```shell + kubectl label --dry-run=server --overwrite ns --all \ + pod-security.kubernetes.io/enforce=baseline + ``` + The output is similar to this: + ``` + namespace/default labeled + namespace/kube-node-lease labeled + namespace/kube-public labeled + Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest" + Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes + Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes + Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged + namespace/kube-system labeled + namespace/local-path-storage labeled + ``` + + 3. Restricted + ```shell + kubectl label --dry-run=server --overwrite ns --all \ + pod-security.kubernetes.io/enforce=restricted + ``` + The output is similar to this: + ``` + namespace/default labeled + namespace/kube-node-lease labeled + namespace/kube-public labeled + Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest" + Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile + Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true + Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile + Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile + namespace/kube-system labeled + Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest" + Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile + namespace/local-path-storage labeled + ``` + +From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings +for any namespaces. However, `baseline` and `restricted` standards both have +warnings, specifically in the `kube-system` namespace. + +## Set modes, versions and standards + +In this section, you apply the following Pod Security Standards to the `latest` version: + +* `baseline` standard in `enforce` mode. +* `restricted` standard in `warn` and `audit` mode. + +The `baseline` Pod Security Standard provides a convenient +middle ground that allows keeping the exemption list short and prevents known +privilege escalations. + +Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace +from having Pod Security Standards applied. + +When you implement Pod Security Admission in your own environment, consider the +following: + +1. Based on the risk posture applied to a cluster, a stricter Pod Security + Standard like `restricted` might be a better choice. +1. Exempting the `kube-system` namespace allows pods to run as + `privileged` in this namespace. For real world use, the Kubernetes project + strongly recommends that you apply strict RBAC + policies that limit access to `kube-system`, following the principle of least + privilege. + To implement the preceding standards, do the following: +1. Create a configuration file that can be consumed by the Pod Security + Admission Controller to implement these Pod Security Standards: + + ``` + mkdir -p /tmp/pss + cat < /tmp/pss/cluster-level-pss.yaml + apiVersion: apiserver.config.k8s.io/v1 + kind: AdmissionConfiguration + plugins: + - name: PodSecurity + configuration: + apiVersion: pod-security.admission.config.k8s.io/v1beta1 + kind: PodSecurityConfiguration + defaults: + enforce: "baseline" + enforce-version: "latest" + audit: "restricted" + audit-version: "latest" + warn: "restricted" + warn-version: "latest" + exemptions: + usernames: [] + runtimeClasses: [] + namespaces: [kube-system] + EOF + ``` + +1. Configure the API server to consume this file during cluster creation: + + ``` + cat < /tmp/pss/cluster-config.yaml + kind: Cluster + apiVersion: kind.x-k8s.io/v1alpha4 + nodes: + - role: control-plane + kubeadmConfigPatches: + - | + kind: ClusterConfiguration + apiServer: + extraArgs: + admission-control-config-file: /etc/config/cluster-level-pss.yaml + extraVolumes: + - name: accf + hostPath: /etc/config + mountPath: /etc/config + readOnly: false + pathType: "DirectoryOrCreate" + extraMounts: + - hostPath: /tmp/pss + containerPath: /etc/config + # optional: if set, the mount is read-only. + # default false + readOnly: false + # optional: if set, the mount needs SELinux relabeling. + # default false + selinuxRelabel: false + # optional: set propagation mode (None, HostToContainer or Bidirectional) + # see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation + # default None + propagation: None + EOF + ``` + + {{}} + If you use Docker Desktop with KinD on macOS, you can + add `/tmp` as a Shared Directory under the menu item + **Preferences > Resources > File Sharing**. + {{}} + +1. Create a cluster that uses Pod Security Admission to apply + these Pod Security Standards: + + ```shell + kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml + ``` + The output is similar to this: + ``` + Creating cluster "psa-with-cluster-pss" ... + ✓ Ensuring node image (kindest/node:v1.23.0) đŸ–ŧ + ✓ Preparing nodes đŸ“Ļ + ✓ Writing configuration 📜 + ✓ Starting control-plane 🕹ī¸ + ✓ Installing CNI 🔌 + ✓ Installing StorageClass 💾 + Set kubectl context to "kind-psa-with-cluster-pss" + You can now use your cluster with: + + kubectl cluster-info --context kind-psa-with-cluster-pss + + Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂 + ``` + +1. Point kubectl to the cluster + ```shell + kubectl cluster-info --context kind-psa-with-cluster-pss + ``` + The output is similar to this: + ``` + Kubernetes control plane is running at https://127.0.0.1:63855 + CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy + + To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. + ``` +1. Create the following Pod specification for a minimal configuration in the default namespace: + + ``` + cat < /tmp/pss/nginx-pod.yaml + apiVersion: v1 + kind: Pod + metadata: + name: nginx + spec: + containers: + - image: nginx + name: nginx + ports: + - containerPort: 80 + EOF + ``` +1. Create the Pod in the cluster: + + ```shell + kubectl apply -f /tmp/pss/nginx-pod.yaml + ``` + The output is similar to this: + ``` + Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") + pod/nginx created + ``` + +## Clean up + +Run `kind delete cluster -name psa-with-cluster-pss` and +`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you +created. + +## {{% heading "whatsnext" %}} + +- Run a + [shell script](/examples/security/kind-with-cluster-level-baseline-pod-security.sh) + to perform all the preceding steps at once: + 1. Create a Pod Security Standards based cluster level Configuration + 2. Create a file to let API server consumes this configuration + 3. Create a cluster that creates an API server with this configuration + 4. Set kubectl context to this new cluster + 5. Create a minimal pod yaml file + 6. Apply this file to create a Pod in the new cluster +- [Pod Security Admission](/docs/concepts/security/pod-security-admission/) +- [Pod Security Standards](/docs/concepts/security/pod-security-standards/) +- [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/) diff --git a/content/en/docs/tutorials/security/ns-level-pss.md b/content/en/docs/tutorials/security/ns-level-pss.md new file mode 100644 index 0000000000000..119c1411e7b29 --- /dev/null +++ b/content/en/docs/tutorials/security/ns-level-pss.md @@ -0,0 +1,159 @@ +--- +title: Apply Pod Security Standards at the Namespace Level +content_type: tutorial +weight: 10 +--- + +{{% alert title="Note" %}} +This tutorial applies only for new clusters. +{{% /alert %}} + +Pod Security admission (PSA) is enabled by default in v1.23 and later, as it [graduated +to beta](/blog/2021/12/09/pod-security-admission-beta/). Pod Security Admission +is an admission controller that applies +[Pod Security Standards](docs/concepts/security/pod-security-standards/) +when pods are created. In this tutorial, you will enforce the `baseline` Pod Security Standard, +one namespace at a time. + +You can also apply Pod Security Standards to multiple namespaces at once at the cluster +level. For instructions, refer to [Apply Pod Security Standards at the cluster level](/docs/tutorials/security/cluster-level-pss). +## {{% heading "prerequisites" %}} + +Install the following on your workstation: + +- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) + +## Create cluster + +1. Create a `KinD` cluster as follows: + + ```shell + kind create cluster --name psa-ns-level --image kindest/node:v1.23.0 + ``` + The output is similar to this: + ``` + Creating cluster "psa-ns-level" ... + ✓ Ensuring node image (kindest/node:v1.23.0) đŸ–ŧ + ✓ Preparing nodes đŸ“Ļ + ✓ Writing configuration 📜 + ✓ Starting control-plane 🕹ī¸ + ✓ Installing CNI 🔌 + ✓ Installing StorageClass 💾 + Set kubectl context to "kind-psa-ns-level" + You can now use your cluster with: + + kubectl cluster-info --context kind-psa-ns-level + + Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/ + ``` + +1. Set the kubectl context to the new cluster: + ```shell + kubectl cluster-info --context kind-psa-ns-level + ``` + The output is similar to this: + ``` + Kubernetes control plane is running at https://127.0.0.1:50996 + CoreDNS is running at https://127.0.0.1:50996/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy + + To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. + ``` + +## Create a namespace + +Create a new namespace called `example`: + +```shell +kubectl create ns example +``` +The output is similar to this: +``` +namespace/example created +``` + +## Apply Pod Security Standards + +1. Enable Pod Security Standards on this namespace using labels supported by + built-in Pod Security Admission. In this step we will warn on baseline pod + security standard as per the latest version (default value) + + ```shell + kubectl label --overwrite ns example \ + pod-security.kubernetes.io/warn=baseline \ + pod-security.kubernetes.io/warn-version=latest + ``` + +2. Multiple pod security standards can be enabled on any namespace, using labels. + Following command will `enforce` the `baseline` Pod Security Standard, but + `warn` and `audit` for `restricted` Pod Security Standards as per the latest + version (default value) + + ``` + kubectl label --overwrite ns example \ + pod-security.kubernetes.io/enforce=baseline \ + pod-security.kubernetes.io/enforce-version=latest \ + pod-security.kubernetes.io/warn=restricted \ + pod-security.kubernetes.io/warn-version=latest \ + pod-security.kubernetes.io/audit=restricted \ + pod-security.kubernetes.io/audit-version=latest + ``` + +## Verify the Pod Security Standards + +1. Create a minimal pod in `example` namespace: + + ```shell + cat < /tmp/pss/nginx-pod.yaml + apiVersion: v1 + kind: Pod + metadata: + name: nginx + spec: + containers: + - image: nginx + name: nginx + ports: + - containerPort: 80 + EOF + ``` +1. Apply the pod spec to the cluster in `example` namespace: + ```shell + kubectl apply -n example -f /tmp/pss/nginx-pod.yaml + ``` + The output is similar to this: + ``` + Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") + pod/nginx created + ``` + +1. Apply the pod spec to the cluster in `default` namespace: + ```shell + kubectl apply -n default -f /tmp/pss/nginx-pod.yaml + ``` + Output is similar to this: + ``` + pod/nginx created + ``` + +The Pod Security Standards were applied only to the `example` +namespace. You could create the same Pod in the `default` namespace +with no warnings. + +## Clean up + +Run `kind delete cluster -name psa-ns-level` to delete the cluster created. + +## {{% heading "whatsnext" %}} + +- Run a + [shell script](/examples/security/kind-with-namespace-level-baseline-pod-security.sh) + to perform all the preceding steps all at once. + 1. Create KinD cluster + 2. Create new namespace + 3. Apply `baseline` Pod Security Standard in `enforce` mode while applying + `restricted` Pod Security Standard also in `warn` and `audit` mode. + 4. Create a new pod with the following pod security standards applied +- [Pod Security Admission](/docs/concepts/security/pod-security-admission/) +- [Pod Security Standards](/docs/concepts/security/pod-security-standards/) +- [Apply Pod Security Standards at the cluster level](/docs/tutorials/security/cluster-level-pss/) \ No newline at end of file diff --git a/content/en/examples/security/kind-with-cluster-level-baseline-pod-security.sh b/content/en/examples/security/kind-with-cluster-level-baseline-pod-security.sh new file mode 100644 index 0000000000000..690b33344314e --- /dev/null +++ b/content/en/examples/security/kind-with-cluster-level-baseline-pod-security.sh @@ -0,0 +1,70 @@ +#!/bin/sh +mkdir -p /tmp/pss +cat < /tmp/pss/cluster-level-pss.yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AdmissionConfiguration +plugins: +- name: PodSecurity + configuration: + apiVersion: pod-security.admission.config.k8s.io/v1beta1 + kind: PodSecurityConfiguration + defaults: + enforce: "baseline" + enforce-version: "latest" + audit: "restricted" + audit-version: "latest" + warn: "restricted" + warn-version: "latest" + exemptions: + usernames: [] + runtimeClasses: [] + namespaces: [kube-system] +EOF +cat < /tmp/pss/cluster-config.yaml +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: +- role: control-plane + kubeadmConfigPatches: + - | + kind: ClusterConfiguration + apiServer: + extraArgs: + admission-control-config-file: /etc/config/cluster-level-pss.yaml + extraVolumes: + - name: accf + hostPath: /etc/config + mountPath: /etc/config + readOnly: false + pathType: "DirectoryOrCreate" + extraMounts: + - hostPath: /tmp/pss + containerPath: /etc/config + # optional: if set, the mount is read-only. + # default false + readOnly: false + # optional: if set, the mount needs SELinux relabeling. + # default false + selinuxRelabel: false + # optional: set propagation mode (None, HostToContainer or Bidirectional) + # see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation + # default None + propagation: None +EOF +kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml +kubectl cluster-info --context kind-psa-with-cluster-pss +# Wait for 15 seconds (arbitrary) ServiceAccount Admission Controller to be available +sleep 15 +cat < /tmp/pss/nginx-pod.yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - image: nginx + name: nginx + ports: + - containerPort: 80 +EOF +kubectl apply -f /tmp/pss/nginx-pod.yaml diff --git a/content/en/examples/security/kind-with-namespace-level-baseline-pod-security.sh b/content/en/examples/security/kind-with-namespace-level-baseline-pod-security.sh new file mode 100644 index 0000000000000..2081de7c14c25 --- /dev/null +++ b/content/en/examples/security/kind-with-namespace-level-baseline-pod-security.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# Until v1.23 is released, kind node image needs to be built from k/k master branch +# Ref: https://kind.sigs.k8s.io/docs/user/quick-start/#building-images +kind create cluster --name psa-ns-level --image kindest/node:v1.23.0 +kubectl cluster-info --context kind-psa-ns-level +# Wait for 15 seconds (arbitrary) ServiceAccount Admission Controller to be available +sleep 15 +kubectl create ns example +kubectl label --overwrite ns example \ + pod-security.kubernetes.io/enforce=baseline \ + pod-security.kubernetes.io/enforce-version=latest \ + pod-security.kubernetes.io/warn=restricted \ + pod-security.kubernetes.io/warn-version=latest \ + pod-security.kubernetes.io/audit=restricted \ + pod-security.kubernetes.io/audit-version=latest +cat < /tmp/pss/nginx-pod.yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - image: nginx + name: nginx + ports: + - containerPort: 80 +EOF +kubectl apply -n example -f /tmp/pss/nginx-pod.yaml