Artifacts are installed with wrong permissions

[maleadt]

I'm having an issue with a rootfs artifact (i.e. containing a Linux OS) being extracted with the wrong permissions. The image in question is used in PkgEval.jl, and the problematic file is a lock whose u+w permissions are dropped.

First of all, here's the tarball and the permissions the lock file should have:

❯ wget https://github.com/JuliaCI/PkgEval.jl/releases/download/v0.1/debian-bullseye-20220818.tar.xz
❯ tar -tvf debian-bullseye-20220818.tar.xz | grep var/lib/dpkg/lock-frontend
-rw-r-----  0 tim    tim         0 Aug 18 16:19 ./var/lib/dpkg/lock-frontend

If I download this tarball using Pkg's download_verify_unpack, all seems well (an o+r permission is added, which isn't great, but also not terrible):

julia> Pkg.download_verify_unpack("https://github.com/JuliaCI/PkgEval.jl/releases/download/v0.1/debian-bullseye-20220818.tar.xz", nothing, mktempdir(); verbose=true, ignore_existence=true)
[ Info: Unpacking /var/folders/q0/wbdz2dpd0q9gtjk3wkj1fcnh0000gn/T/jl_ti7eJ2RSRC-download.xz into /var/folders/q0/wbdz2dpd0q9gtjk3wkj1fcnh0000gn/T/jl_EajM2J...
true

shell> ls -la /var/folders/q0/wbdz2dpd0q9gtjk3wkj1fcnh0000gn/T/jl_EajM2J/var/lib/dpkg/lock-frontend
-rw-r--r--  1 tim  staff  0 Dec  2 10:07 /var/folders/q0/wbdz2dpd0q9gtjk3wkj1fcnh0000gn/T/jl_EajM2J/var/lib/dpkg/lock-frontend

However, if I pack this up in an Artifacts.toml, I get the following:

julia> ArtifactUtils.add_artifact!("Artifacts.toml", "debian", "https://github.com/JuliaCI/PkgEval.jl/releases/download/v0.1/debian-bullseye-20220818.tar.xz"; lazy=true)

❯ cat Artifacts.toml
[debian]
git-tree-sha1 = "e5d2359bc94f9d9ddc771c2274c83324ca8cbbcc"
lazy = true
    [[debian.download]]
    sha256 = "072392d8635847de70665af512551038b04baca99b38e271d0eef1ea36d2833a"
    url = "https://github.com/JuliaCI/PkgEval.jl/releases/download/v0.1/debian-bullseye-20220818.tar.xz"

julia> using LazyArtifacts

julia> artifact"debian"
"/Users/tim/.julia/artifacts/e5d2359bc94f9d9ddc771c2274c83324ca8cbbcc"

shell> ls -la /Users/tim/.julia/artifacts/e5d2359bc94f9d9ddc771c2274c83324ca8cbbcc/var/lib/dpkg/lock-frontend
-r--r--r--  1 tim  staff  0 Dec  2 10:08 /Users/tim/.julia/artifacts/e5d2359bc94f9d9ddc771c2274c83324ca8cbbcc/var/lib/dpkg/lock-frontend

... and our u+w permission is gone. That's bad, here breaking dpkg in the containers spawned using this rootfs because it can't take the lock.

Tested on 1.8.2, reproduces on macOS and Linux. This artifact also seems to trigger an unrelated git tree hash change between 1.7 (d41ccdb32c924682293a4fe525b9986b7debb11d) and 1.8 (e5d2359bc94f9d9ddc771c2274c83324ca8cbbcc).

[giordano]

Is that because Tar.jl normalises permissions?

[maleadt]

I don't think so, because in JuliaCI/PkgEval.jl#189 I'm now using Tar.jl directly to unpack this tarball, and permissions are fine.

[staticfloat]

The issue is that Pkg likes to set things to be read-only:

Pkg.jl/src/Artifacts.jl

Line 79 in 6dd0e7c

set_readonly(new_path)

I think the reason the areas of the ecosystem that I administer don't run into this is that I'm usually running as root on the inside of containers, and dpkg likely ignores read-only permissions on lock files if its run as root:

julia> using Sandbox
       config = SandboxConfig(
           Dict("/" => Sandbox.debian_rootfs());
           stdin, stdout, stderr,
       )
       with_executor() do exe
           run(exe, config, `/bin/bash -c 'apt update; apt install make'`)
       end
Get:1 https://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 https://deb.debian.org/debian bullseye/main amd64 Packages [8068 kB]
Get:3 https://deb.debian.org/debian bullseye/main Translation-en [6236 kB]
Fetched 14.4 MB in 2s (7687 kB/s)                           
Reading package lists... Done
Building dependency tree... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: No sandbox user '_apt' on the system, can not drop privileges
Reading package lists... Done
Building dependency tree... Done
Suggested packages:
  make-doc
The following NEW packages will be installed:
  make
0 upgraded, 1 newly installed, 0 to remove and 20 not upgraded.
Need to get 396 kB of archives.
After this operation, 1630 kB of additional disk space will be used.
Get:1 https://deb.debian.org/debian bullseye/main amd64 make amd64 4.3-4.1 [396 kB]
Fetched 396 kB in 0s (1996 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package make.
(Reading database ... 6972 files and directories currently installed.)
Preparing to unpack .../make_4.3-4.1_amd64.deb ...
Unpacking make (4.3-4.1) ...
Setting up make (4.3-4.1) ...
W: No sandbox user '_apt' on the system, can not drop privileges
Process(`/usr/bin/docker run --privileged -i --label org.julialang.sandbox.jl=7utA1BMVq3 -t -w / --user 0:0 sandbox_rootfs:a507efe7-0-0 /bin/bash -c 'apt update; apt install make'`, ProcessExited(0))

julia> run(`ls -la $(joinpath(Sandbox.debian_rootfs(), "var", "lib", "dpkg", "lock"))`)
-r--r--r-- 1 sabae sabae 0 Aug  8  2023 /home/sabae/.julia/artifacts/16738e2d713323e608ff891cb66de38d9d667d45/var/lib/dpkg/lock

[maleadt]

I think the reason the areas of the ecosystem that I administer don't run into this is that I'm usually running as root on the inside of containers

Makes sense, but for PkgEval I don't want to run as root in order to spot packages doing questionable things (e.g., using su or sudo).