Tutorials for Pod Security Admission

content/en/docs/tutorials/_index.md

@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
 
 * [Using Source IP](/docs/tutorials/services/source-ip/)
 
+## Security
+
+* [Apply Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/)
+* [Apply Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/)
+
 ## {{% heading "whatsnext" %}}
 
 If you would like to write a tutorial, see

content/en/docs/tutorials/security/_index.md

@@ -0,0 +1,5 @@
+---
+title: "Security"
+weight: 40
+---
+

content/en/docs/tutorials/security/cluster-level-pss.md

@@ -0,0 +1,324 @@
+---
+title: Apply Pod Security Standards at the Cluster Level
+content_type: tutorial
+weight: 10
+---
+
+{{% alert title="Note" %}}
+This tutorial applies only for new clusters.
+{{% /alert %}}
+
+Pod Security admission (PSA) is enabled by default in v1.23 and later, as it has
+[graduated to beta](/blog/2021/12/09/pod-security-admission-beta/).
+Pod Security
+is an admission controller that carries out checks against the Kubernetes
+[Pod Security Standards](docs/concepts/security/pod-security-standards/) when new pods are
+created. This tutorial shows you how to enforce the `baseline` Pod Security
+Standard at the cluster level which applies a standard configuration
+to all namespaces in a cluster.
+
+To apply Pod Security Standards to specific namespaces, refer to [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss).
+
+## {{% heading "prerequisites" %}}
+
+Install the following on your workstation:
+
+- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
+- [kubectl](https://kubernetes.io/docs/tasks/tools/)
+
+## Choose the right Pod Security Standard to apply
+
+[Pod Security Admission](/docs/concepts/security/pod-security-admission/)
+lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
+with the following modes: `enforce`, `audit`, and `warn`.
+
+To gather information that helps you to choose the Pod Security Standards
+that are most appropriate for your configuration, do the following:
+
+1. Create a cluster with no Pod Security Standards applied:
+
+    ```shell
+    kind create cluster --name psa-wo-cluster-pss --image kindest/node:v1.23.0
+    ```
+   The output is similar to this:
+    ```
+    Creating cluster "psa-wo-cluster-pss" ...
+    ✓ Ensuring node image (kindest/node:v1.23.0) 🖼
+    ✓ Preparing nodes 📦  
+    ✓ Writing configuration 📜
+    ✓ Starting control-plane 🕹️
+    ✓ Installing CNI 🔌
+    ✓ Installing StorageClass 💾
+    Set kubectl context to "kind-psa-wo-cluster-pss"
+    You can now use your cluster with:
+
+    kubectl cluster-info --context kind-psa-wo-cluster-pss
+
+    Thanks for using kind! 😊
+
+    ```
+
+1. Set the kubectl context to the new cluster:
+
+    ```shell
+    kubectl cluster-info --context kind-psa-wo-cluster-pss
+    ```
+   The output is similar to this:
+
+    ```
+     Kubernetes control plane is running at https://127.0.0.1:61350
+
+    CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
+
+    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+    ```
+
+1.  Get a list of namespaces in the cluster:
+
+    ```shell
+    kubectl get ns
+    ```
+    The output is similar to this:
+    ```      
+    NAME                 STATUS   AGE
+    default              Active   9m30s
+    kube-node-lease      Active   9m32s
+    kube-public          Active   9m32s
+    kube-system          Active   9m32s
+    local-path-storage   Active   9m26s
+    ```
+
+1. Use `--dry-run=server` to understand what happens when different Pod Security Standards
+   are applied:
+
+   1. Privileged
+      ```shell
+      kubectl label --dry-run=server --overwrite ns --all \                    
+      pod-security.kubernetes.io/enforce=privileged
+      ```
+     The output is similar to this:
+      ```      
+      namespace/default labeled
+      namespace/kube-node-lease labeled
+      namespace/kube-public labeled
+      namespace/kube-system labeled
+      namespace/local-path-storage labeled
+      ```
+   2. Baseline
+      ```shell    
+      kubectl label --dry-run=server --overwrite ns --all \
+      pod-security.kubernetes.io/enforce=baseline
+      ```
+     The output is similar to this:
+      ```   
+      namespace/default labeled
+      namespace/kube-node-lease labeled
+      namespace/kube-public labeled
+      Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
+      Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
+      Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
+      Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
+      namespace/kube-system labeled
+      namespace/local-path-storage labeled
+      ```   
+
+   3. Restricted
+     ```shell
+      kubectl label --dry-run=server --overwrite ns --all \
+      pod-security.kubernetes.io/enforce=restricted
+      ```
+     The output is similar to this:
+      ```   
+      namespace/default labeled
+      namespace/kube-node-lease labeled
+      namespace/kube-public labeled
+      Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
+      Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
+      Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
+      Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
+      Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
+      namespace/kube-system labeled
+      Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
+      Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
+      namespace/local-path-storage labeled
+      ```
+
+From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings
+for any namespaces. However, `baseline` and `restricted` standards both have
+warnings, specifically in the `kube-system` namespace.
+
+## Set modes, versions and standards
+
+In this section, you apply the following Pod Security Standards to the `latest` version:
+
+* `baseline` standard in `enforce` mode.
+* `restricted` standard in `warn` and `audit` mode.
+
+The `baseline` Pod Security Standard provides a convenient
+middle ground that allows keeping the exemption list short and prevents known
+privilege escalations.
+
+Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace
+from having Pod Security Standards applied.
+
+When you implement Pod Security Admission in your own environment, consider the
+following:
+
+1. Based on the risk posture applied to a cluster, a stricter Pod Security
+   Standard like `restricted` might be a better choice.
+1. Exempting the `kube-system` namespace allows pods to run as
+   `privileged` in this namespace. For real world use, the Kubernetes project
+   strongly recommends that you apply strict RBAC
+   policies that limit access to `kube-system`, following the principle of least
+   privilege.
+   To implement the preceding standards, do the following:
+1. Create a configuration file that can be consumed by the Pod Security
+   Admission Controller to implement these Pod Security Standards:
+
+    ```
+    mkdir -p /tmp/pss
+    cat <<EOF > /tmp/pss/cluster-level-pss.yaml 
+    apiVersion: apiserver.config.k8s.io/v1
+    kind: AdmissionConfiguration
+    plugins:
+    - name: PodSecurity
+      configuration:
+        apiVersion: pod-security.admission.config.k8s.io/v1beta1
+        kind: PodSecurityConfiguration
+        defaults:
+          enforce: "baseline"
+          enforce-version: "latest"
+          audit: "restricted"
+          audit-version: "latest"
+          warn: "restricted"
+          warn-version: "latest"
+        exemptions:
+          usernames: []
+          runtimeClasses: []
+          namespaces: [kube-system]
+    EOF
+    ```
+
+1. Configure the API server to consume this file during cluster creation:
+
+    ```
+    cat <<EOF > /tmp/pss/cluster-config.yaml 
+    kind: Cluster
+    apiVersion: kind.x-k8s.io/v1alpha4
+    nodes:
+    - role: control-plane
+      kubeadmConfigPatches:
+      - |
+        kind: ClusterConfiguration
+        apiServer:
+            extraArgs:
+              admission-control-config-file: /etc/config/cluster-level-pss.yaml
+            extraVolumes:
+              - name: accf
+                hostPath: /etc/config
+                mountPath: /etc/config
+                readOnly: false
+                pathType: "DirectoryOrCreate"
+      extraMounts:
+      - hostPath: /tmp/pss
+        containerPath: /etc/config
+        # optional: if set, the mount is read-only.
+        # default false
+        readOnly: false
+        # optional: if set, the mount needs SELinux relabeling.
+        # default false
+        selinuxRelabel: false
+        # optional: set propagation mode (None, HostToContainer or Bidirectional)
+        # see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
+        # default None
+        propagation: None
+    EOF
+    ```
+
+   {{<note>}}
+   If you use Docker Desktop with KinD on macOS, you can
+   add `/tmp` as a Shared Directory under the menu item
+   **Preferences > Resources > File Sharing**.
+   {{</note>}}
+
+1. Create a cluster that uses Pod Security Admission to apply
+   these Pod Security Standards:
+
+   ```shell
+    kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml
+   ```
+   The output is similar to this:
+   ```
+    Creating cluster "psa-with-cluster-pss" ...
+     ✓ Ensuring node image (kindest/node:v1.23.0) 🖼 
+     ✓ Preparing nodes 📦  
+     ✓ Writing configuration 📜 
+     ✓ Starting control-plane 🕹️ 
+     ✓ Installing CNI 🔌 
+     ✓ Installing StorageClass 💾 
+    Set kubectl context to "kind-psa-with-cluster-pss"
+    You can now use your cluster with:
+
+    kubectl cluster-info --context kind-psa-with-cluster-pss
+
+    Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
+    ```
+
+1. Point kubectl to the cluster
+   ```shell
+    kubectl cluster-info --context kind-psa-with-cluster-pss
+    ```
+   The output is similar to this:
+    ```
+     Kubernetes control plane is running at https://127.0.0.1:63855
+     CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
+
+     To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+    ```
+1. Create the following Pod specification for a minimal configuration in the default namespace:
+
+    ```
+    cat <<EOF > /tmp/pss/nginx-pod.yaml
+    apiVersion: v1
+    kind: Pod
+    metadata:
+      name: nginx
+    spec:
+      containers:
+        - image: nginx
+          name: nginx
+          ports:
+            - containerPort: 80
+    EOF
+    ```
+1. Create the Pod in the cluster:
+
+   ```shell
+    kubectl apply -f /tmp/pss/nginx-pod.yaml
+   ```
+   The output is similar to this:
+   ```
+    Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
+    pod/nginx created
+   ```
+
+## Clean up
+
+Run `kind delete cluster -name psa-with-cluster-pss` and
+`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you
+created.
+
+## {{% heading "whatsnext" %}}
+
+- Run a
+  [shell script](/examples/security/kind-with-cluster-level-baseline-pod-security.sh)
+  to perform all the preceding steps at once:
+  1. Create a Pod Security Standards based cluster level Configuration
+  2. Create a file to let API server consumes this configuration
+  3. Create a cluster that creates an API server with this configuration
+  4. Set kubectl context to this new cluster
+  5. Create a minimal pod yaml file
+  6. Apply this file to create a Pod in the new cluster
+- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
+- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
+- [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/)