kbs/config/kubernetes: changes on base/policy.rego don't take effects on deployment

[wainersm]

Describe the bug

Context

I've been using kbs/config/kubernetes to deploy KBS in coco-play tool. Until CoCo v0.12.0, by overwriting kbs/config/kubernetes/base/policy.rego I was able to set a default resources policy of type "allow all". Something has changed on v0.12.0 so overwriting kbs/config/kubernetes/base/policy.rego takes no effect anymore.

Problem

Unless I'm not using the right kbs image, overwriting kbs/config/kubernetes/base/policy.rego doesn't take any effect.

How to reproduce

• Case 1 (not working):

Checkout main, updated base/kustomization.yaml to use ghcr.io/confidential-containers/staged-images/kbs:latest, and overwrote base/policy.rego.

As can be seen below, /opt/confidential-containers/kbs/policy.rego in the kbs container doesn't correspond to the base/policy.rego file's content:

xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ git diff
diff --git a/kbs/config/kubernetes/base/kustomization.yaml b/kbs/config/kubernetes/base/kustomization.yaml
index 5446b3b..da5f71b 100644
--- a/kbs/config/kubernetes/base/kustomization.yaml
+++ b/kbs/config/kubernetes/base/kustomization.yaml
@@ -4,8 +4,8 @@ namespace: coco-tenant
 
 images:
 - name: kbs-container-image
-  newName: ghcr.io/confidential-containers/key-broker-service
-  newTag: built-in-as-v0.10.1
+  newName: ghcr.io/confidential-containers/staged-images/kbs
+  newTag: latest
 
 resources:
 - namespace.yaml
diff --git a/kbs/config/kubernetes/base/policy.rego b/kbs/config/kubernetes/base/policy.rego
index 4d9b339..2a379fc 100644
--- a/kbs/config/kubernetes/base/policy.rego
+++ b/kbs/config/kubernetes/base/policy.rego
@@ -1,40 +1,5 @@
-# Resource Policy
-# ---------------
-#
-# The resource policy of KBS is to make a strategic decision on
-# whether the requester has access to resources based on the
-# input Attestation Claims (including tee-pubkey, tcb-status, and other information)
-# and KBS Resource Path.
-#
-# The format of the resource path data is:
-# ```
-# {
-#        "resource-path": <PATH>
-# }
-# ```
-#
-# The <PATH> variable is a KBS resource path,
-# which is required to be a string in three segment path format:<TOP>/<MIDDLE>/<TAIL>,
-# for example: "repo/License/key".
-#
-# The format of Attestation Claims Input is defined by the attestation service,
-# and its format may look like the following:
-# ```
-# {
-#     "tee-pubkey": "",
-#     "tcb-status": {
-#         "productId": “”,
-#         "svn": “”,
-#                ……
-#     }
-#        ……
-# }
-# ```
 
 package policy
 
-default allow = false
+default allow = true
 
-allow {
-       not input["submods"]["cpu"]["ear.veraison.annotated-evidence"]["sample"]
-} 
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ export DEPLOYMENT_DIR=nodeport
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ echo "somesecret" > overlays/key.bin
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ ./deploy-kbs.sh 
namespace/coco-tenant created
configmap/kbs-config-d655m9k8gk created
configmap/policy-config-t6mb856fbh created
secret/kbs-auth-public-key-g668mc72dg created
secret/keys-gmf2g75547 created
service/kbs created
deployment.apps/kbs created
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ kubectl get pods -n coco-tenant
NAME                   READY   STATUS    RESTARTS   AGE
kbs-64c8f67f74-q24s7   1/1     Running   0          15m
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ kubectl exec kbs-64c8f67f74-q24s7 -c kbs -n coco-tenant -- cat /opt/confidential-containers/kbs/policy.rego
# Resource Policy
# ---------------
#
# The resource policy of KBS is to make a strategic decision on
# whether the requester has access to resources based on the
# input Attestation Claims (including tee-pubkey, tcb-status, and other information)
# and KBS Resource Path.
#
# The format of the resource path data is:
# ```
# {
#         "resource-path": <PATH>
# }
# ```
#
# The <PATH> variable is a KBS resource path,
# which is required to be a string in three segment path format:<TOP>/<MIDDLE>/<TAIL>,
# for example: "repo/License/key".
#
# The format of Attestation Claims Input is defined by the attestation service,
# and its format may look like the following:
# ```
# {
#     "tee-pubkey": "",
#     "tcb-status": {
#         "productId": “”,
#         "svn": “”,
#                 ……
#     }
#         ……
# }
# ```

package policy

default allow = false

allow {
        not input["submods"]["cpu"]["ear.veraison.annotated-evidence"]["sample"]
}

Case 2 (working):

Checkout v0.10.1 (the version tested on CoCo 0.11.0), kept base/kustomization.yaml intact to use ghcr.io/confidential-containers/key-broker-service:built-in-as-v0.10.1, and overwrote base/policy.rego.

Omitted deployment commands for brevity, below you can see the /opt/confidential-containers/kbs/policy.rego file is correct:

xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ git diff
diff --git a/kbs/config/kubernetes/base/policy.rego b/kbs/config/kubernetes/base/policy.rego
index d9f9eb5..2a379fc 100644
--- a/kbs/config/kubernetes/base/policy.rego
+++ b/kbs/config/kubernetes/base/policy.rego
@@ -1,40 +1,5 @@
-# Resource Policy
-# ---------------
-#
-# The resource policy of KBS is to make a strategic decision on
-# whether the requester has access to resources based on the
-# input Attestation Claims (including tee-pubkey, tcb-status, and other information)
-# and KBS Resource Path.
-#
-# The format of the resource path data is:
-# ```
-# {
-#        "resource-path": <PATH>
-# }
-# ```
-#
-# The <PATH> variable is a KBS resource path,
-# which is required to be a string in three segment path format:<TOP>/<MIDDLE>/<TAIL>,
-# for example: "my'repo/License/key".
-#
-# The format of Attestation Claims Input is defined by the attestation service,
-# and its format may look like the following:
-# ```
-# {
-#     "tee-pubkey": "",
-#     "tcb-status": {
-#         "productId": “”,
-#         "svn": “”,
-#                ……
-#     }
-#        ……
-# }
-# ```
 
 package policy
 
-default allow = false
+default allow = true
 
-allow {
-       input["tee"] != "sample"
-}
xxx:~/src/github.com/confidential-containers/trustee/kbs/config/kubernetes$ kubectl exec kbs-56b4c75998-4svrx -c kbs -n coco-tenant -- cat /opa
/confidential-containers/kbs/policy.rego

package policy

default allow = true

CoCo version information

trustee latest

What TEE are you seeing the problem on

None

Failing command and relevant log output

[wainersm]

I've found the bug. It's writing the policy file to the old location (/opa/confidential-containers/kbs). Gonna send a fix soon.

[fitzthum]

Sorry got here too late.