poudriere doesn't work in a jail

[bapt]

It is not possible to start poudriere bulk within a jail:

# poudriere bulk -f /usr/local/etc/poudriere.d/ports-lists/Kunden-Ports -j Kunden

[00:00:00] ====>> Creating the reference jail... done
[00:01:04] ====>> Mounting system devices for Kunden-default
[00:01:04] ====>> Mounting ports/packages/distfiles
[00:01:04] ====>> Using packages from previously failed build
[00:01:04] ====>> Mounting packages from: /poudriere/data/packages/Kunden-default
[00:01:04] ====>> Mounting /var/db/ports from: /usr/local/etc/poudriere.d/Kunden-options
[00:01:04] ====>> Appending to make.conf: /usr/local/etc/poudriere.d/Kunden-make.conf
/etc/resolv.conf -> /poudriere/data/.m/Kunden-default/ref/etc/resolv.conf
[00:01:04] ====>> Starting jail Kunden-default
jail: jail_set: Operation not permitted
[00:01:04] ====>> Cleaning up
[00:01:04] ====>> Umounting file systems

I've followed the wiki-entry as much as possible, but i do not use ZFS. Therefore my FreeBSD 10 config for the jail looks like:

poudriere {
  path="/usr/local/jail/poudriere";
  host.hostname="poudriere";
  ip6.addr="2a01:4f8:150:50a5::12/64";
        persist;
        children.max=99;
        allow.mount;
        allow.mount.devfs;
        allow.mount.procfs;
        allow.mount.zfs;
        allow.mount.nullfs;
        allow.mount.tmpfs;
        allow.raw_sockets;
        allow.socket_af;
        allow.sysvipc;
        allow.chflags;
        mount.devfs;
        enforce_statfs=1; 
  interface=re0;
  exec.start = "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";

}

The error message indicates that it is not possible to create a jail within a jail. Therefore i tried it manually - i'm able to create a jail within a jail- poudriere not.

If you have IPv6 it is possible to give you access to the jail for further testing.

[bapt]

It is probably failing due to not having an 'ip4' entry. Can you run 'poudriere -x bulk ... 2>log' and either upload the log file or host it somewhere that I can look at?

[bapt]

You can see the log here:
http://pkg.toco-domains.de/poudriere-in-jail-error.log

I tried you suggestion with "ip4=inherit;ip6=inherit;". You are right - after setting this options, it works :)

[bapt]

With "it works" i mean:
The error message is gone. I could not test the building of ports, because i have no IPv4 address to use. poudriere fails to fetch the ports - it seems to force download via IPv4?

[bapt]

I have a look at the log. The error occurs in this line:

jail -c persist name=Kunden-default path=/poudriere/data/.m/Kunden-default/ref host.hostname=Kunden-default ip4.addr=127.0.0.1 ip6.addr=::1 allow.socket_af allow.raw_sockets allow.chflags allow.sysvipc

If a change the line to:

jail -c persist name=Kunden-default path=/mnt/ host.hostname=Kunden-default ip6.addr=2a01:4f8:150:50a5::12 allow.socket_af allow.raw_sockets allow.chflags allow.sysvipc

The jail is created successfull. Also i can work with it. I hope this helps.

Thanks for all your work :)

[bapt]

After setting the IPv6 address hardcoded, the building of a port starts.

But it stops right after compiling the pkg-package. Error Message:

=== Start ===

====> Compressing man pages (compress-man)

=======================<phase: package >============================
===> Building package for pkg-1.3.5.1
pkg-static: Cannot open "/var/run/ld-elf.so.hints": No such file or directory
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
make: stopped in /usr/ports/ports-mgmt/pkg
====>> Cleaning up wrkdir
===> Cleaning for pkg-1.3.5.1
build of /usr/ports/ports-mgmt/pkg ended at Mon Aug 11 08:35:21 UTC 2014
build time: 00:00:30
!!! build failure encountered !!!
[root@poudriere /]# ls -lah /var/run/ld-elf.so.hints

=== End ===

The file exists:

ls -lah /var/run/ld-elf.so.hints

-r--r--r-- 1 root wheel 199B Aug 6 13:58 /var/run/ld-elf.so.hints

So i believe it is not copied correctly?

[bapt]

After some debugging, i found the problem with the elf-hints. When creating the reference jail, the path to the jail is used for copy.

In my case this is /usr/local/jail/poudriere/var/run -> and this path didn't exists from within the jail.
As a workaround for me i hardcoded the copying of /var/run. This seems to work :)

[gynter]

The problem described in this issue is not related to the Poudriere, but to the design of FreeBSD jails. Since this issue report is the first one which pops up in search engines I'll put the solution to here.

Issue/error

[00:00:00] ====>> Creating the reference jail... done
[00:00:04] ====>> Mounting system devices for 10-1R_amd64-default
[00:00:04] ====>> Mounting ports/packages/distfiles
[00:00:04] ====>> Using packages from previously failed build
[00:00:04] ====>> Mounting packages from: /poudriere/data/packages/10-1R_amd64-default
[00:00:04] ====>> Appending to make.conf: /usr/local/etc/poudriere.d/10-1R_amd64-make.conf
/etc/resolv.conf -> /poudriere/data/.m/10-1R_amd64-default/ref/etc/resolv.conf
[00:00:04] ====>> Starting jail 10-1R_amd64-default
jail: jail_set: Operation not permitted
[00:00:04] ====>> Cleaning up
[00:00:04] ====>> Umounting file systems

Description

From man jail:

Jailed processes are not allowed to confer greater permissions than they
themselves are given, e.g., if a jail is created with allow.nomount, it
is not able to create a jail with allow.mount set. Similarly, such
restrictions as ip4.addr and securelevel may not be bypassed in child
jails.

Therefore one must define IP addresses 127.0.0.1 and ::1 for
loopback interface lo0 in the super jail system (Poudriere jail). Same goes with any other IP address, You can only use the IP's for hierarchical jails which are set for the super jail.

Test (on 10.1-STABLE)

When running, this one gets the error stated above:

# jail -c persist 'name=10-1R_amd64-default' \ 
'path=/poudriere/data/.m/10-1R_amd64-default/ref' \ 
'host.hostname=10-1R_amd64-default' \ 
'ip4.addr=127.0.0.1'  'ip6.addr=::1' \ 
allow.socket_af allow.raw_sockets allow.chflags allow.sysvipc

Runnig the code below goes without issues (You can see that
'ip4.addr=127.0.0.1' 'ip6.addr=::1' params are removed):

# jail -c persist 'name=10-1R_amd64-default' \ 
'path=/poudriere/data/.m/10-1R_amd64-default/ref' \ 
'host.hostname=10-1R_amd64-default' \ 
allow.socket_af allow.raw_sockets allow.chflags allow.sysvipc

The test can be reproduced by removing ${network} variable from line
252 of /usr/local/share/poudriere/common.sh.

Solution (for ezjail only, but same method can be applied to /etc/jail.conf)

Append 127.0.0.1,::1 to the jail IP list in
/usr/local/etc/ezjail/myjail_example_org:

export jail_myjail_example_org_ip="em0|192.168.3.6,lo0|127.0.0.1,lo0|::1"

[bdrewery]

Super old bug squashed!

c756e18 also helps.