Ubernerd: Portable install of nerdctl (without modifying host rootfs)

[Jip-Hop]

Hi all, I'm developing an experimental install script which allows starting containerd and running nerdctl without modifying the host rootfs. I'm trying to reduce the possibility for conflicts as much as possible. Looking forward to your feedback. 🙂

[fahedouch]

Hi @Jip-Hop,

thanks for you effort. It would be nice if the portable install of nerdctl/containerd could provide more isolation such as a container isolation ( e.g by mounting config and data folder in a container including containerd sock ) , this may help many people asking for containerd-in-containerd #2008 #1383

I gave you the first star ;)

[fahedouch]

Is this what you meant?

Yes. but I expected a lot less of container mounts

. Slight mystery still as to why /run/containerd is still being used (in addition to /run/ubernerd/containerd). I thought I changed all the relevant config to /run/ubernerd/containerd...

strange, are you facing some errors without /run/containerd ?

Regarding installing ubernerd and running nerdctl inside a container (without connecting to the containerd socket on the host). It's only working for me when I specify --snapshotter=native. Otherwise I get:

IMO ubernerd should focus on mount host containerd sock provided by ubernerd.sh setup. We already provider a containerd setup inside a docker image

[Jip-Hop]

Without mounting /run/containerd I get:

time="2023-09-01T07:36:20Z" level=fatal msg="failed to create shim task: failed to start io pipe copy: unable to copy pipes: containerd-shim: opening file "/run/containerd/fifo/3297564989/a1084e5bef3f95e71a08a0d3b9e397534dd50b6d2773f7c2865d2e1f350c5d76-stdout" failed: open /run/containerd/fifo/3297564989/a1084e5bef3f95e71a08a0d3b9e397534dd50b6d2773f7c2865d2e1f350c5d76-stdout: no such file or directory: unknown"

Did I miss some config to change this path from /run/containerd to /run/ubernerd/containerd or is it not possible to change this path?

[Jip-Hop]

Yes. but I expected a lot less of container mounts

If this fifo dir is configurable I think I can make it work with a single mount.

[Jip-Hop]

P.S. Running ubernerd.sh inside a docker container now works without --snapshotter=native. The issue I ran into was due to using a bind mounted directory from macOS into the docker container.

[philrhc]

i have used this to run containerd-in-containerd connecting to the host containerd.sock, the script uses the version from https://api.github.com/repos/containerd/nerdctl/releases/latest which is 1.7.6

@Jip-Hop this is really good work, thank you

@fahedouch

this may help many people asking for containerd-in-containerd #2008 #1383

i also think this will help people

[ningziwen]

Thanks for the effort! Gave the 3rd star :)

Going through the scripts and documentation, the use case seems pretty similar to Finch Linux (tracking issue).

Looking forward to the potential collaboration if the use cases can align.

[Jip-Hop]

Looks interesting! But idk if Finch Linux would be as portable as I'd like ubernerd to be. Not sure if that's the intention for Fine Linux, but I definitely don't want to run nerdctl inside a VM.

[ningziwen]

Yeah. Finch macOS and Windows have to use VM, but whether to use VM in Finch Linux is still open. We have not discussed much about it yet.

Considering the nested VM issue of running Finch Linux in cloud instance such as EC2, there are definitely reasons supporting no-VM portable Finch Linux.

Feel free to share your thoughts in Finch Linux issue. The input from you or other community members will contribute to the decision of whether to go.