Kyverno Demo

Setup

devcontainer

This demo shows how to use Kyverno to enforce policies on Kubernetes resources. You can use the devcontainer following this repo. The devcontainer has all the tools installed to run the demo.

Setting up the demo without the devcontainer

First, you will need to have kind installed. You can find instructions on how to install kind here.

Create a cluster

To create a cluster, run the following command:

kind create cluster --name kyverno-demo

Get the kubeconfig

kind get kubeconfig --name kyverno-demo > ~/.kube/config

Install Kyverno

Using Helm

Following the instructions here to install Kyverno using Helm.

Using kubectl

To install Kyverno, run the following command:

kubectl apply -f https://raw.githubusercontent.com/kyverno/kyverno/main/definitions/release/install.yaml

K8s policy Demo

kubectl apply -f policies/require-tech-lead-label.yaml
kubectl apply -f demo-pods/missing-tech-lead-label.yaml

We should then see the following output:

    Error from server: error when creating "demo-pods/missing-tech-lead-label.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

    policy Pod/demo-pods/failing-demo for resource violation: 

    require-labels:
    check-for-labels: 'validation error: The label `tech-lead` is required. rule check-for-labels
        failed at path /metadata/labels/tech-lead/'

Report

To get a report, run the following command:

kubectl describe backgroundscanreports.kyverno.io {a-backgroundscanreport-name}

Example output(reduced for brevity):

Name:         194e4165-e135-49ad-8585-1c397b1cd773
Namespace:    kyverno
Labels:       app.kubernetes.io/managed-by=kyverno
              audit.kyverno.io/resource.hash=42a4837ede0daaa4e04e5a2a26ce3300
              audit.kyverno.io/resource.uid=194e4165-e135-49ad-8585-1c397b1cd773
              cpol.kyverno.io/require-labels=24562
Annotations:  audit.kyverno.io/last-scan-time: 2023-03-25T23:46:40Z
API Version:  kyverno.io/v1alpha2
Kind:         BackgroundScanReport
Metadata:
  Creation Timestamp:  2023-03-25T21:33:52Z
  Generation:          3
  Owner References:
    API Version:     v1
    Kind:            Pod
    Name:            kyverno-cleanup-controller-89d978b7c-wr544
Spec:
  Results:
    Category:  Best Practices
    Message:   validation error: The label `tech-lead` is required. rule check-for-labels failed at path /metadata/labels/tech-lead/
    Policy:    require-labels
    Result:    fail
    Rule:      check-for-labels
    Scored:    true
    Severity:  medium
    Source:    kyverno
    Timestamp:
      Nanos:    0
      Seconds:  1679788006
  Summary:
    Error:  0
    Fail:   1
    Pass:   0
    Skip:   0
    Warn:   0
Events:     <none>

Kyverno CLI Demo

The kyverno cli is installed as part of this codespace. Please see the Kyverno CLI documentation if you want to install it locally.

Validate a policy

kyverno test .

The result

codespace ➜ /workspaces/kyverno-demo (main) $ kyverno test .

Executing require tech label...
applying 2 policies to 3 resources... 
I0331 16:56:19.873155   12360 evaluate.go:63] EngineValidate "msg"="a condition failed in 'all' block" "kind"="Pod" "name"="failing-demo" "namespace"="tech-lead" "policy"="valid-image" "rule"="valid-image" "condition"={"key":["nginx"],"operator":"AnyNotIn","value":["traefik","nginx"]}
I0331 16:56:19.874060   12360 evaluate.go:63] EngineValidate "msg"="a condition failed in 'all' block" "kind"="Pod" "name"="valid-tech-lead-demo" "namespace"="tech-lead" "policy"="valid-image" "rule"="valid-image" "condition"={"key":["nginx"],"operator":"AnyNotIn","value":["traefik","nginx"]}

│───│─────────────────────────│─────────────────────│────────────────────────────────────│────────│
│ # │ POLICY                  │ RULE                │ RESOURCE                           │ RESULT │
│───│─────────────────────────│─────────────────────│────────────────────────────────────│────────│
│ 1 │ require-tech-lead-label │ check-for-tech-lead │ tech-lead/Pod/failing-demo         │ Pass   │
│ 2 │ require-tech-lead-label │ check-for-tech-lead │ tech-lead/Pod/valid-tech-lead-demo │ Pass   │
│ 3 │ valid-image             │ valid-image         │ valid-image-demo/Pod/invalid-image │ Pass   │
│───│─────────────────────────│─────────────────────│────────────────────────────────────│────────│

Test Summary: 3 tests passed and 0 tests failed

Pod Security Standards

Pod Security Standards define 3 different levels of security. The Kyverno Pod Security Standard policies are available here.

Privileged

Lets apply the restricted policy:

kubectl apply -f pod-security-standard/policies/pod-security-standard-restricted.yaml

The output should be:

k apply -f pod-security-standard/demo-pods/testing-restricted.yaml
Error from server: error when creating "pod-security-standard/demo-pods/testing-restricted.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

policy Pod/demo-pods/valid-tech-lead-demo for resource violations: 

podsecurity-subrule-restricted:
  restricted: |
    Validation rule 'restricted' failed. It violates PodSecurity "restricted:latest": ({Allowed:false ForbiddenReason:allowPrivilegeEscalation != false ForbiddenDetail:container "failing-demo" must set securityContext.allowPrivilegeEscalation=false})
    ({Allowed:false ForbiddenReason:unrestricted capabilities ForbiddenDetail:container "failing-demo" must set securityContext.capabilities.drop=["ALL"]})
    ({Allowed:false ForbiddenReason:runAsNonRoot != true ForbiddenDetail:pod or container "failing-demo" must set securityContext.runAsNonRoot=true})
    ({Allowed:false ForbiddenReason:seccompProfile ForbiddenDetail:pod or container "failing-demo" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost"})
require-labels:
  check-for-labels: 'validation error: The label `tech-lead` is required. rule check-for-labels
    failed at path /metadata/labels/tech-lead/'

Now lets test opting the pod out of the policy:

kubectl apply -f pod-security-standard/policies/pod-security-standard-restricted-opt-out.yaml

We should then be able to apply the previously failing pod:

k apply -f pod-security-standard/demo-pods/testing-restricted.yaml 
pod/valid-tech-lead-demo created

Rapporting

First start cleaning up the previous demo:

k delete -f pod-security-standard/policies/pod-security-standard-restricted.yaml
k delete -f pod-security-standard/policies/pod-security-standard-restricted-opt-out.yaml

Add a pod that violates a future policy:

k apply -f pod-security-standard/demo-pods/testing-restricted.yaml

Now lets apply the policy:

k apply -f pod-security-standard/policies/pod-security-standard-restricted.yaml

You can now get a rapport

k get policyreports.wgpolicyk8s.io cpol-podsecurity-subrule-restricted -n demo-pods
NAME                                  PASS   FAIL   WARN   ERROR   SKIP   AGE
cpol-podsecurity-subrule-restricted   0      1      0      0       0      72s

Mutate policy demo

In addition to validation, Kyverno can also mutate resources. For a full list of its features see https://kyverno.io/docs/writing-policies/mutate/ Lets apply a mutate policy:

k apply -f policies/mutate/set-image-pull-policy.yaml

Now add a pod that is missing imagePullPolicy:

k apply -f demo-pods/mutate/missing-imagePullPolicy.yaml