/var/lib/rancher/k3s/storage should not be world-readable.

[BoleynSu]

/var/lib/rancher/k3s/storage should not be world-readable.

[brandond]

This will probably need to be resolved via an issue in https://github.com/rancher/local-path-provisioner

[MikaelSmith]

An issue was filed at rancher/local-path-provisioner#182. I'm not sure if the permission of the PVC directory itself would impact whether non-root containers are able to use it.

I'm pretty sure ensuring /var/lib/rancher/k3s/storage (and maybe /var/lib/rancher/k3s/data?) have permissions 700 would prevent non-root users from accessing the volumes while allowing them to be used by containers (no matter what user the container runs as).

[rancher-max]

Validated on master branch commit a4c992ce521110ba90d077717b93dab926c7cae8

Only changes were made to items within the storage directory.

• /var/lib/rancher/k3s/storage directory has 0755 permissions
• Items within storage directory correctly have 0700 permissions

$ ls -la /var/lib/rancher/k3s/storage/
total 16
drwxr-xr-x 4 root root 4096 Jul  9 17:56 .
drwxr-xr-x 6 root root 4096 Jul  9 17:50 ..
drwx------ 2 root root 4096 Jul  9 17:56 pvc-435bf9d1-8dd8-45d6-aee7-17f17798a9d0_default_other-path-pvc
drwx------ 2 root root 4096 Jul  9 17:50 pvc-614bcf34-9f1c-4c17-8af5-7010134e5509_default_local-path-pvc

$ stat -c %a /var/lib/rancher/k3s/storage/
755

$ stat -c %a /var/lib/rancher/k3s/storage/pvc-435bf9d1-8dd8-45d6-aee7-17f17798a9d0_default_other-path-pvc/
700

$ stat -c %a /var/lib/rancher/k3s/storage/pvc-614bcf34-9f1c-4c17-8af5-7010134e5509_default_local-path-pvc/
700

[flokli]

@rancher-max this is not fixed, and in fact broke all volumes for containers running not as root. See #3704, which reported this as a k3s regression.

From rancher/local-path-provisioner#182 (comment), /var/lib/rancher/k3s/storage shouln't be world-readable (but restricted to root), and individual volumes should be chmod 777:

So volume itself is 0777, but the parent directory secured with 0700 and accessible by root only.

[brandond]

@flokli I see that you've found the linked issue that tracks fixing the regression, is there any reason in particular that you're adding comments to the original PR and this closed issue? Do you need anything else?

[flokli]

@brandond I guess I was mostly confused about where the chmods happen, and whether k3s did workaround a change done in local-path-provisioner, or if these changes (and regressions) all appeared in the k3s local-storage manifest.

Maybe it might make sense to update rancher/local-path-provisioner#182 (comment) to set things clear.

[brandond]

As you can see from the PRs, we don't use the sample deployment yaml that is in the local-path-provisioner repo, so we haven't changed anything on that side.

[endawkins]

Validated on branch 1.27 with commit 91c5e0d / version 1.27

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-13-52 5.19.0-1025-aws #26~22.04.1-Ubuntu SMP Mon Apr 24 01:58:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Cluster Configuration:

1 server

Config.yaml:

cluster-init: true
write-kubeconfig-mode: 644
token: test

Additional files

pvc.yaml:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-path-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: local-path
  resources:
    requests:
      storage: 2Gi

pod.yaml:
apiVersion: v1
kind: Pod
metadata:
  name: volume-test
  namespace: default
spec:
  containers:
  - name: volume-test
    image: nginx:stable-alpine
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: volv
      mountPath: /data
    ports:
    - containerPort: 80
  volumes:
  - name: volv
    persistentVolumeClaim:
      claimName: local-path-pvc

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s
3. Run the following commands:

$ kubectl create -f pvc.yaml
$ kubectl create -f pod.yaml
$ kubectl get pv
$ kubectl get pic
$ kubectl get nodes, pods -A - o wide
$ stat -c %a /var/lib/rancher/k3s/storage/

Replication Results:

• k3s version used for replication:

k3s -v
k3s version v1.24.13+k3s1 (3f79b289)
go version go1.19.8

k3s -v
k3s version v1.25.9+k3s1 (7502ce6a)
go version go1.19.8

k3s -v
k3s version v1.26.4+k3s1 (8d0255af)
go version go1.19.8

k3s -v
k3s version v1.27.1+k3s1 (bc5b42c2)
go version go1.20.3

stat -c %a /var/lib/rancher/k3s/storage/
701

Validation Results:

• k3s version used for validation:

k3s -v
k3s version v1.24.14-rc2+k3s1 (313a3b99)
go version go1.19.9

k3s -v
k3s version v1.25.10-rc2+k3s1 (613a3bc8)
go version go1.19.9

k3s -v
k3s version v1.26.5-rc2+k3s1 (6cd54032)
go version go1.19.9

k3s -v
k3s version v1.27.2-rc2+k3s1 (91c5e0d7)
go version go1.20.4

stat -c %a /var/lib/rancher/k3s/storage/
700

Additional context / logs:

N/A