Host folder mount: inconsistent permissions behavior (permissions spontaneously change during session)

[metal450]

Description

Using Docker Desktop with WSL2, I'm mounting a host folder via a relative path. However, the file permissions in that shared folder (as viewed by the container) seem extremely unpredictable/buggy - to the point where the permissions literally seem to change from one moment to the next. Full repro steps are below, but as a quick example, here's a screenshot where I used Docker Desktop's "exec" tab to navigate to the shared folder, and did ls -la twice in a row (with no actions taken in between). You can see that the first time the files have permissions -----, and the second time they have rwxrwxrwx. I literally just did ls -la twice back-to-back:

Reproduce

1. git clone https://github.com/cytopia/devilbox (it's a docker-compose LAMP stack)
2. cd devilbox
3. git checkout v2.4.0
4. Copy the included env-example file to .env
5. The only changes I made: HTTPD_SERVER=apache-2.4, HOST_PORT_MYSQL=33060, HOST_PORT_BIND=45013
6. Devilbox automatically creates vhosts for whatever folders you stick in /shared/httpd. Create a docker-compose.override.yml file containing:

services:   
  httpd:
    volumes:
      - ../../../projects/SOME_FOLDER_WITH_PHP_FILES:/shared/httpd/dev/htdocs:rw
  php:
    volumes:
      - ../../../projects/SOME_FOLDER_WITH_PHP_FILES:/shared/httpd/dev/htdocs:rw

7. Make sure you have some php at those paths on the host (obviously). i.e. download https://wordpress.org & unzip it there.
8. docker-compose up -d mysql
9. Visit http://localhost. You'll see the Devilbox dashboard. (If you visit the "Virtual Hosts" page, you can also see that it autocreated a vhost "dev.loc", based on the fact that it found directory "dev" in /shared/httpd).
10. Edit your Windows hosts file to add 127.0.0.1 dev.loc
11. Visit https://dev.loc You'll see "403 Forbidden".
12. Go into Docker Desktop -> the "php" container-> Exec tab. cd /shared/httpd/dev/htdocs. ls -la. It shows all the files with permissions ----, as in the screenshot above.
13. Immediately ls -la again. Now it suddenly shows rwxrwxrwx

Additional notes:

• I've found that simply listing the files via Docker Desktop actually changes the behavior of the server. i.e. I'll visit the URL in a browser, and it'll show "forbidden." Then I'll ls -la. I'll refresh the browser. Then it will successfully load scripts in the folder that I listed - but if they include/require any scripts in other folders, which I haven't "looked at" from Docker desktop, it will fail to include those due to access denied errors.
• The above Devilbox setup works perfectly with Docker Toolbox (which I was using previously for years).
• My machine is Windows-Linux dual boot. If I reboot to Linux, I can also run the exact same Devilbox instance as setup above. It works properly. Even the project files are the exact same (i.e. I mount the same NTFS data partition to Linux).

So to summarize, only Docker Desktop/wsl is behaving strangely - the same containers, config, & data works properly with both Docker Engine on Linux and Docker Toolbox.

Why does Docker Desktop it show --- one minute, then rwx the next, and why does simply listing the files from within Docker Desktop change the behavior of the server?

Expected behavior

It should be able to access the host folder properly, as it can in both Docker Engine and Docker Toolbox

docker version

Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:17:17 2024
 OS/Arch:           windows/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.34.2 (167172)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:15 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.2.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2-desktop.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     C:\Program Files\Docker\cli-plugins\docker-desktop.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     C:\Program Files\Docker\cli-plugins\docker-feedback.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.13.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 12
 Server Version: 27.2.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.15.153.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 19.36GiB
 Name: docker-desktop
 ID: 28e3f3fe-b8d7-4dd2-bae9-dacdcc80eb23
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile

Diagnostics ID

52FB3D9B-25A6-442E-A07D-0B5E6D120234/20240929033802

Additional Info

No response

[metal450]

Ok...after many hours, I finally just figured it out. Edit: Nevermind, not resolved- see follow-up below

NTFS permissions. Evidently giving "Full Access" to "Everyone" is not sufficient - you have to explicitly give "Full Access" to "SYSTEM" as well.

[metal450]

....Nevermind, I take it back. It seemed to start working when I applied those ntfs permissions, & it worked for about 15 minutes, but then spontaneously stopped working again. So the broken/inconsistent behavior remains.

[metal450]

Ok, more clarity:

• Let's say it's initially working (i.e. I can access the php vhost in a browser with no errors)
• Stop/remove the container (docker-compose down)
• Reboot Windows
• Start Docker Desktop
• Start the container docker-compose up -d mysql
• Now, after the reboot, it won't work (accessing in a browser shows 403 forbidden, or "file not found", or PHP errors. The exact error will depend on the nature of your PHP, but the root cause is always the same: the container can't access the php files).
• In Windows Explorer, go to the source files & Get Properties on the containing folder. Do nothing else other than get properties.
• Refresh browser. Now it mysteriously works.

Observations:

• Recall my original observation/screenshot: the first ls -l showed no permissions, then ls -l immediately after showed rwx. It looks like just "reading from the files" after the container is started makes them accessible, as if reading from them once causes Docker to properly "evaluate" the permissions. However, it's weird that I have to manually do this - why does my manually reading the files "fix" the permissions but the webhost process itself accessing them does not?
• This explains why I previously thought that setting NTFS permissions fixed it: when setting permissions, I first accessed the "Properties" dialog. So the permissions were just a red herring, it was going into that dialog that did it.
• I can repeat the above & its behavior is consistent. i.e every time I stop+remove the container, reboot Windows, then start the container, it's broken. Then by doing Get Properties after the container is started, it fixes it.
  • Occasionally "Get Properties" doesn't resolve it on the very first try - just do it once or twice more, that always fixes it.
  • Occasionally just stopping & restarting the container (without rebooting Windows) is sufficient to reintroduce the error, but not always. Fully rebooting Windows reproduces the error 100% of the time.
• If I Get Properties before starting the container itself, it does NOT fix it. (i.e. reboot -> start Docker Desktop -> get properties -> start the container -> 403 Forbidden. Reboot -> Start Docker Desktop -> Start the container -> Get properties -> OK).