Localpath provisioner regression v1.21.3: Non-root users can't write to volume

[nsmith5]

Environmental Info:
K3s Version:

• Broken on v1.21.3+k3s1
• Workingo on v1.20.9+k3s1

Node(s) CPU architecture, OS, and Version:

• kernel and arch: Linux 5.12.19-300.fc34.x86_64 x86_64 x86_64 x86_64 GNU/Linux
• OS: Fedora CoreOS 34
• Selinux disabled completely

Cluster Configuration:

• Single standalone node (server + agent all in one)

Describe the bug:
Permissions appear to be incorrect on PVC's created with local path provisioner. On v1.21.3 I see the PVC directories under /var/lib/rancher/k3s/storage created with 0700 and owned by root. On v1.20.9 I see the PVC directories under /var/lib/rancher/k3s/storage created with 0777. The 0700 permission forbids non-root users from writing any data to these PVC directories when they're mounted into containers where as the old 0777 permissions allowed non-root users to write immediately without any further configuration

Steps To Reproduce:

• Install k3s binary for v1.21.3 in /usr/local/bin and run k3s systemd unit as follows:

Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-/etc/systemd/system/k3s.service.env
ExecStart=/usr/local/bin/k3s server
KillMode=process
Delegate=yes
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

Expected behavior:
Should have same behaviour as 1.20.9 (non-root users should be able to write to a PVC created by local-path provisioner without any additional work)

Actual behavior:
Not possible in 1.21.3. Need to chmod the PVC directory as 0777 before that is possible (or chown with the container user id)

Additional context / logs:

Backporting

• Needs backporting to older releases

[ChristianCiach]

Duplicate of #3704

[nsmith5]

Closing as a duplicate (thanks @ChristianCiach !)