Per Container User Namespaces in Pod Quadlets?

[connelhooley]

Feature request description

I have 2 containers in a pod but I'd like to have one container map UID 33 to HUID 1000 and another container map UID 999 to HUID 1000.

If I try set UIDMap=+999:@%U in a .container file that is in a pod, I'm told I can't do that:

Error: cannot specify a new uid/gid map when entering a pod with an infra container: invalid argument

I can't specify this at a pod level as the mappings conflict.

I can see there is an option in podman pod create to create a pod that doesn't have an infra container. If I specify this in my .container quadlet, like this: PodmanArgs=--infra=false the setting is ignored. I'm assuming this is by design and not a bug, but if it isn't I can raise a bug instead of a feature request.

I can also see that there is an option in podman pod create to specify which namespaces to share. If I specify that in my .container quadlet like this PodmanArgs=--share="net" I still get the same error referring to the infra container above (even though I'm trying to opt out of the shared user namespace). The share option is a little strange in that it only supports these namespaces "cgroup, ipc, net, pid, uts". User isn't mentioned.

Another semi-related gotcha with the current design is that ShmSize is ignored when set on a container that is in a pod. This isn't a big deal but was unintuitive and it would be great to have more control over what is shared in a pod and what isn't.

Suggest potential solution

Users should be able to specify per container user namespaces in pods.

When specifying PodmanArgs=--infra=false the value should either be applied correctly or the user should be given an error message explaining that quadlets don't support it and ideally why.

When specifying ShmSize=xxx for a container that is in a pod, the value should either be applied correctly at a container level or the user should be given an error message explaining that containers must have the value set at the pod level.

Have you considered any alternatives?

I think the alternative is to share a network instead of being in a pod and then create a service that depends on the containers so that I can start and stop the containers as a group. The downside of this approach is that I lose a lot of the benefits of pods like the UI in Cockpit (I like the hardware usage overview of each pod).

It would feel like I'm recreating some of pod's functionality manually just to have a bit more control of host user ids.

Additional context

I'm still very new to Podman and Linux namespaces so maybe what I'm asking for makes no sense given how pods actually work but the option is there to turn off the infra container and configure the shared namespaces so hopefully something is possible.

[mheon]

This is a kernel-level limitation. User namespaces own all the namespaces under them - including things like the network and IPC namespaces, which we share through all containers in a pod. User namespaces can only use namespaces that they directly own - AFAIK, a security measure to ensure that host namespaces can't accidentally leak into a user namespace. If you have two different user namespaces (necessitated by the fact that you want different mappings for each pod), then we can't share the namespaces between them anymore because one user namespace won't have permissions on them.

Your alternative of sharing a network is probably the best option - each container will have a separate namespace, just tying into a shared network on the host.

[connelhooley]

@mheon that's fair, thanks for taking the time to explain it to me. I have what I need set up via networks and it's working really well.

Maybe the error message could be a little better for the uneducated like myself as it only refers to the infra container which seems slightly misleading based on your reply.

[mheon]

Yeah, we should probably catch this sort of thing at container creation time - user-specified custom user namespace configuration for containers in pods should trigger a check against the pod to verify it's different from the pod namespace, then we can print a helpful error message instead.

Tagging good first issue as this shouldn't be particularly difficult.

[Luap99]

The logic how this is implemented is also kinda broken right now, compare #26889.

I think for 6.0 we need fixup the logic how we check this user namesapce and return proper errors, i.e. consider the code does this

if c.Flag("pod").Changed && !strings.HasPrefix(c.Flag("pod").Value.String(), "new:") && c.Flag("userns").Changed {
			return vals, errors.New("--userns and --pod cannot be set together")
		}

But that gets bypassed by quadlet as it uses --pod-id-file instead. Such validations must be correctly applied in specgen and not on the cli and then consistently enforced for all user namesapces modes

[ninja-quokka]

I think for 6.0 we need fixup the logic how we check this user namesapce and return proper errors, i.e. consider the code does this

@Luap99 Should we add the 6.0 tag to this issue?

[Luap99]

I mean fixing the error message to be more descriptive and improving docs can always be done.

As for the condition I linked I would say that one should be dropped in favor of proper user namesapce checks in the specgen level that cover all paths. Maybe it could be done right now when I look at it there are some ways to create a sort of "working" pod with a container that has a different userns which then coould count as regession even though the permissions never really worked. So I rather wait with that or 6.0.

[eshentials]

Can I work on this issue

[mheon]

Sure, feel free