Data not copying up to volumes for containers created but not started

[adelton]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When volume and container with this volume are created by docker, that volume gets populated (copied up from the image) and can then be consumed by another container using --volumes-from. It does not seem to work with podman, in spite of #3945 having been resolved via #3961.

Steps to reproduce the issue:

1. podman volume create test-copy-up
2. podman create --name test-copy -v test-copy-up:/etc/yum.repos.d registry.fedoraproject.org/fedora noop
3. podman run --rm --volumes-from test-copy docker.io/library/busybox ls -la /etc/yum.repos.d

Describe the results you received:

total 8
drwxr-xr-x    2 root     root          4096 Oct 16 08:43 .
drwxr-xr-x    4 root     root          4096 Oct 16 08:44 ..

Describe the results you expected:

total 40
drwxr-xr-x    2 root     root          4096 Oct 16 08:42 .
drwxr-xr-x    1 root     root          4096 Oct 16 08:43 ..
-rw-r--r--    1 root     root           728 Aug 25 20:31 fedora-cisco-openh264.repo
-rw-r--r--    1 root     root          1303 Aug 25 20:31 fedora-modular.repo
-rw-r--r--    1 root     root          1349 Aug 25 20:31 fedora-updates-modular.repo
-rw-r--r--    1 root     root          1391 Aug 25 20:31 fedora-updates-testing-modular.repo
-rw-r--r--    1 root     root          1344 Aug 25 20:31 fedora-updates-testing.repo
-rw-r--r--    1 root     root          1286 Aug 25 20:31 fedora-updates.repo
-rw-r--r--    1 root     root          1239 Aug 25 20:31 fedora.repo

Additional information you deem important (e.g. issue happens only occasionally):

I get the same results both with rootless podman and under sudo podman.

The expected result is from the same commands, just replaced podman with docker, under moby-engine-19.03.11-1.ce.git42e35e6.fc32.x86_64.

Output of podman version:

Version:      2.1.1
API Version:  2.0.0
Go Version:   go1.14.9
Built:        Wed Sep 30 21:31:11 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.16.1
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.21-2.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: 81d18b6c3ffc266abdef7ca94c1450e669a6a388'
  cpus: 8
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: journald
  hostname: test.machine.test
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.8.13-200.fc32.x86_64
  linkmode: dynamic
  memFree: 1252122624
  memTotal: 16553324544
  ociRuntime:
    name: runc
    package: runc-1.0.0-144.dev.gite6555cc.fc32.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10+dev
      commit: fbdbaf85ecbc0e077f336c03062710435607dbf1
      spec: 1.0.1-dev
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.4-1.fc32.x86_64
    version: |-
      slirp4netns version 1.1.4
      commit: b66ffa8e262507e37fca689822d23430f3357fe8
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 2
  swapFree: 4266692608
  swapTotal: 4294963200
  uptime: 188h 19m 16.7s (Approximately 7.83 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/test/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 0
    stopped: 2
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.1.2-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /home/test/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 12
  runRoot: /run/user/1000/containers
  volumePath: /home/test/.local/share/containers/storage/volumes
version:
  APIVersion: 2.0.0
  Built: 1601494271
  BuiltTime: Wed Sep 30 21:31:11 2020
  GitCommit: ""
  GoVersion: go1.14.9
  OsArch: linux/amd64
  Version: 2.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

podman-2.1.1-7.fc32.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

No

Additional environment details (AWS, VirtualBox, physical, etc.):

This is on physical Fedora 32 machine.

[adelton]

The reason I only create the container but not run it is that in real use-case, the image serves only as a distribution mechanism for the data to be used as volume and there are no binaries on it to run. The end goal of this exercise is to then copy the data from the volume to real directory on the host, with something like

docker run --volumes-from container-with-volume-populated-at-data -v /path/we-want-the-data-here:/data-out:z --rm busybox sh -c 'cd /data && cp -a . /data-out'

I also looked at skopeo but it only seems to manipulate the whole images, and the task of exploding the layers would need to be done manually.

[mheon]

This is one place where I don't know if we can copy Docker. Our create operation is very lightweight and deliberately does not mount the container (so we can't perform the copy-up during it). Changing this wouldn't be that hard but does have significant performance implications for commands like podman run.

You should be able to do a podman init and then podman container cleanup on the container to force it to prepare the mounts by copying up into them - would that be sufficient?

[adelton]

Running podman init I get

Error: container_linux.go:349: starting container process caused "exec: \"noop\": executable file not found in $PATH": OCI runtime command not found error

So for a container image which has no executable binary to run, that does not seem to do the trick.

[mheon]

Hm, damn - didn't expect it would do that check before starting the container.

…

On Fri, Oct 16, 2020, 10:27 Jan Pazdziora ***@***.***> wrote: Running podman init I get Error: container_linux.go:349: starting container process caused "exec: \"noop\": executable file not found in $PATH": OCI runtime command not found error So for a container image which has no executable binary to run, that does not seem to do the trick. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8041 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3AOCEXATHVIWI7VBRU7X3SLBJ4LANCNFSM4STBWIJA> .

[Luap99]

podman init does error but the volume got the files from the container:

$ ll /home/paul/.local/share/containers/storage/volumes/test-copy-up/_data
total 28
-rw-r--r--. 1 paul paul  728 Jun  4 16:59 fedora-cisco-openh264.repo
-rw-r--r--. 1 paul paul 1303 Jun  4 16:59 fedora-modular.repo
-rw-r--r--. 1 paul paul 1349 Jun  4 16:59 fedora-updates-modular.repo
-rw-r--r--. 1 paul paul 1391 Jun  4 16:59 fedora-updates-testing-modular.repo
-rw-r--r--. 1 paul paul 1344 Jun  4 16:59 fedora-updates-testing.repo
-rw-r--r--. 1 paul paul 1286 Jun  4 16:59 fedora-updates.repo
-rw-r--r--. 1 paul paul 1239 Jun  4 16:59 fedora.repo

[adelton]

Ah, you are right. Running podman init and ignoring the exit status 125 indeed does what I need.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@adelton Can we close this issue, since you have a work around?

[adelton]

Yes. Thank you.

[TomSweeneyRedHat]

Closing, thanks for the confirmation @adelton