vz: podman "Permission denied" on bind mounts

[stac47]

Description

Hello,
I did some tests to run podman against a VM spinned by lima with Apple hypervisor On my M1 machine MacOs Ventura 13.6.

Darwin Kernel Version 22.6.0: Fri Sep 15 13:41:28 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_ARM64_T6000 arm64 arm Darwin

I took the podman template model and adapted it to use vz and mounted everything with virtiofs:

vmType: "vz"

images:
- location: "https://mirror.servaxnet.com/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.raw.xz"
  arch: "x86_64"
  digest: "sha256:5fe502369737c77e8ed074803ea0e6b34365ed274701f47327fd1f5eb45474ce"
- location: "https://mirror.servaxnet.com/fedora/linux/releases/38/Cloud/aarch64/images/Fedora-Cloud-Base-38-1.6.aarch64.raw.xz"
  arch: "aarch64"
  digest: "sha256:58ea3280f8fcf8adea81ec2d7d26afcc75989ad4021d4e1de22a88e9714477a8"

mountType: "virtiofs"
mounts:
- location: "~"
  writable: true
- location: "/tmp/lima"
  writable: true

containerd:
  system: false
  user: false

provision:
- mode: system
  script: |
    #!/bin/bash
    set -eux -o pipefail
    command -v podman >/dev/null 2>&1 && exit 0
    dnf -y install podman
- mode: user
  script: |
    #!/bin/bash
    set -eux -o pipefail
    systemctl --user enable --now podman.socket

probes:
- script: |
    #!/bin/bash
    set -eux -o pipefail
    if ! timeout 30s bash -c "until command -v podman >/dev/null 2>&1; do sleep 3; done"; then
      echo >&2 "podman is not installed yet"
      exit 1
    fi
  hint: See "/var/log/cloud-init-output.log" in the guest

portForwards:
- guestSocket: "/run/user/{{.UID}}/podman/podman.sock"
  hostSocket: "{{.Dir}}/sock/podman.sock"

message: |
  To run `podman` on the host (assumes podman-remote is installed), run the following commands:
  ------
  podman system connection add lima-{{.Name}} "unix://{{.Dir}}/sock/podman.sock"
  podman system connection default lima-{{.Name}}
  podman{{if eq .HostOS "linux"}} --remote{{end}} run quay.io/podman/hello
  ------

Then, starting this and opening a shell inside this VM:

% limactl create --name podman ~/perso/podman-vm/podman.yaml
% limactl start podman
% limactl shell podman
[stac@lima-podman /]$ podman --version
podman version 4.7.0

I tried to mount /Users inside a container:

[stac@lima-podman /]$ podman run -it --rm -v /Users/:/toto debian bash
root@afeec1ece533:/# ls /toto
ls: cannot open directory '/toto': Permission denied
root@afeec1ece533:/# ls -l / |grep toto
drwxr-xr-x.   1 nobody nogroup  14 Oct  4 15:07 toto

When I do exact same with the official template works fine:

% limactl create --name podman-template template://podman
% limactl start podman-template
% limactl shell podman-template
[stac@lima-podman-template /]$ podman --version
podman version 4.7.0
[stac@lima-podman-template /]$ podman run -it --rm -v /Users/:/toto debian bash
root@c2b11bc38c30:/# ls -l / | grep toto
drwxr-xr-x.   1 nobody nogroup  14 Oct  4 15:19 toto
root@c2b11bc38c30:/# ls /toto
stac

I spent a couple of hours on this and was wondering whether I missed something obvious or if it was a bug.
Thanks in advance for your help and kudo for lima.

[afbjorklund]

So besides all the cosmetic changes and the writable home directory, you changed two lines

vmType: "vz"
mountType: "virtiofs"

So it seems indeed to be a difference either from the the vmType, or from the mountType...

Does it make a difference if you use the mounted home directory only, and not all of /Users ?

[afbjorklund]

Since this is Fedora, you might also want to see if there is any difference coming from SELinux

setenforce 0 (or use Z mounts)

[balajiv113]

Looks like its related to SELinux policy and also a limitation from virtiofs

As mentioned in this thread i could confirm it works when using --security-opt label=disable
containers/podman#3683 (comment)

I can confirm sshfs working fine (without security-opt) with vz driver

[stac47]

Thanks both for your comments. Yes it is linked to SELinux so I can close this ticket which is not related to Lima.

[AkihiroSuda]

This should be reported to https://github.com/containers/container-selinux/issues ?

[afbjorklund]

Or Podman "known issues" ?

https://github.com/containers/podman

[hasan4791]

We need to mount the host volumes with context parameter when using virtiofs, Something similar to this we need to perform.

#1920

If you all agree, shall we reopen this issue. I can take a look.

[hasan4791]

I have gone though the code flow and seems the mount points for these are set through the userdata instead of script(incase of rosetta). So it is impossible to know the correct distro before hand, we cant set the secontext parameters. Though this can be worked around by maintaining a list of selinux supported distros, it wont come very handy as we need to update the file whenever a new distro comes up.!!! One possible workaround is we can unmount and remount those mountpoints using the secontext options. This wont be a straight fix but get the job done.

Looking for suggestions
@afbjorklund @AkihiroSuda @balajiv113

It should look like this

[root@lima-fedora ~]# ls -lZ /Users/chasan/
total 0
drwx------.  7 root root system_u:object_r:container_file_t:s0  224 Oct 17 05:02 Desktop
drwx------.  3 root root system_u:object_r:container_file_t:s0   96 Oct 12 07:08 Documents
drwx------. 13 root root system_u:object_r:container_file_t:s0  416 Oct 17 07:25 Downloads
drwx------. 88 root root system_u:object_r:container_file_t:s0 2816 Oct 12 17:57 Library
drwx------.  4 root root system_u:object_r:container_file_t:s0  128 Oct 12 07:10 Movies
drwx------.  4 root root system_u:object_r:container_file_t:s0  128 Oct 12 07:11 Music
drwx------.  4 root root system_u:object_r:container_file_t:s0  128 Oct 12 07:08 Pictures
drwxr-xr-x.  4 root root system_u:object_r:container_file_t:s0  128 Oct 12 07:08 Public
drwxr-xr-x.  8 root root system_u:object_r:container_file_t:s0  256 Oct 16 08:55 Work
[root@lima-fedora ~]#