feat: add CcTolerations field to support custom tolerations in ds

[rayanebel]

Description

👋 Team,

In our infrastructure, we need to configure tolerations for each pod we deploy. Currently, there is no option to add tolerations to the underlying DaemonSets, which prevents us from deploying Confidential Containers.

Solution

The solution is straightforward: introduce a new field called ccTolerations, allowing us to specify custom tolerations within the DaemonSets.

apiVersion: confidentialcontainers.org/v1beta1
kind: CcRuntime
metadata:
  name: ccruntime
spec:
  ccNodeSelector:
    matchLabels:
      node.kubernetes.io/worker: ""
  ccTolerations:
  - effect: NoSchedule
    key: metal
    operator: Exists
  - effect: NoSchedule
    key: protocol
    value: test
    operator: Equal

I tested this solution in our cluster, and it works as expected.

NAME                             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                                    AGE
cc-operator-daemon-install       1         1         1       1            1           node.kubernetes.io/worker=""                     36m
cc-operator-daemon-uninstall     0         0         0       0            0           confidentialcontainers.org/startuninstall=true   62m
cc-operator-pre-install-daemon   1         1         1       1            1           node.kubernetes.io/worker=""                     62m

kubectl get ds cc-operator-pre-install-daemon -o yaml | yq '.spec.template.spec.tolerations'

- effect: NoSchedule
  key: metal
  operator: Exists
- effect: NoSchedule
  key: protocol
  operator: Equal
  value: test