[macOS] Mounted directory is not accessible in the chroot environment

[mohitmv]

Steps to reproduce this issue in macOS.

1. Create /tmp/new_root
2. Copy paste required stuff from "/bin" to "/tmp/new_root/bin" and "/usr/lib" to "/tmp/new_root/usr/lib".
3. "chroot /tmp/new_root".. It will work fine. "ls", "cp", "cat" commands will work fine.
4. Create a union mount from "/usr/bin=RO:/tmp/tmp_write=RW --> /tmp/new_root/usr/bin". [In a new terminal outside chroot environment].. Union directory works fine.
5. shift to chroot terminal (having "/tmp/new_root" as root).
6. Directory "/usr/bin" exists but when you run "ls /usr/bin".. It will raise error : "No such file or directory"

---

• macOS: 10.13.6
• Build of this tool - origin/master (as of 23 Dec 2020).
• UnionFs linked against - /usr/local/lib/libfuse.2.dylib in my macOS.

[bsbernd]

So summary, /tmp/new_root/usr/bin can be listed outside of the chroot, but not from the inside? Also, is 'ls' itself failing or does 'readdir' or 'stat' fail? I.e. could you please provide 'strace' output (I have that macOS has strace...) for your 'ls /usr/bin' command?
My guess is that it fails because the *at calls are not used (like openat). These are posix in the mean time, but I have no idea if macOS supports them.

[mohitmv]

Yes - summary is correct.
stat is working for the existing files as well as folders.
ls is failing. In fact ls filename works but ls dir_name fails. cat filename fails.

strace ? did you mean, stderr output of this tool ? (with -o debug option ?)
When I hit ls /usr/bin, I get following debug info printed:

unique: 3, opcode: GETATTR (3), nodeid: 16, insize: 56, pid: 27190
getattr /mohit
   unique: 3, success, outsize: 136
unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 27190
getattr /
   unique: 2, success, outsize: 136

https://pastebin.com/6t4uwfuq

[bsbernd]

I mean output of 'strace -f ls /usr/bin'. Or if 'cat' also failing, 'strace -f cat /usr/bin/{some_failing_file}'. Debug output from 'unionfs -d -o debug_file=<some_log_file>' would also help,

[mohitmv]

Actually strace is linux specific. I found that macOS have dtrace. Which was also not working out for me, I tried do what they suggested (sudo chmod u+s /usr/sbin/dtrace ) or to directly use sudo dtrace -f ls... however it didn't work for me (even outside the chroot).
PS: "They" = https://opensourcehacker.com/2011/12/02/osx-strace-equivalent-dtruss-seeing-inside-applications-what-they-do-and-why-they-hang

[binary-person]

for anyone landing here, i managed to workaround the issue by doing a bindfs mount on the unionfs mount

originally you would do:

unionfs -o cow mychanges=RW:myreaddir=RO jail/mount
# then chroot jail

with the workaround:

unionfs -o cow upper=RW:lower=RO mount
mkdir jail/mount
bindfs mount jail/mount
# then chroot jail

then when you unmount, you unmount the bindfs first

umount jail/mount
umount mount