Buildroot 2020.02

[afbjorklund]

The good news is that it actually compiles (at last)

179M out/minikube.iso

The bad news is that it doesn't boot, but hangs...

The upstream Buildroot 2020.02.x works just fine, so this is something in minikube config.
Possibly related to the initial file systems, like the initrd (rootfs) or the sysroot (tmpfs) ?

For #7904

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [afbjorklund]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[afbjorklund]

Yup, seems like something went horribly wrong in the init.

It's where we copy the contents of the rootfs to the tmpfs:

#!/bin/sh
mkdir /sysroot
# the value 90% borrowed from tcl via boot2docker
mount -t tmpfs -o size=90% tmpfs /sysroot
# copy from rootfs, to be able to do switch_root(8)
tar -C / --exclude=sysroot -cf - . | tar -C /sysroot/ -xf -

�mount: /sysroot: wrong fs type, bad option, bad superblock on tmpfs, missing codepage or helper program, or other error.

And then, because of that, it dies in the final step:

exec /sbin/switch_root /sysroot /sbin/init "$@"

BusyBox v1.31.1 (2020-05-17 20:20:13 CEST) multi-call binary.

Usage: switch_root [-c CONSOLE_DEV] NEW_ROOT NEW_INIT [ARGS]

Free initramfs and switch to another root fs:
chroot to NEW_ROOT, delete all in /, move NEW_ROOT to /,
execute NEW_INIT. PID must be 1. NEW_ROOT must be a mountpoint.

	-c DEV	Reopen stdio to DEV after switch
[   24.079005] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100

[codecov-io]

Codecov Report

Merging #8187 into master will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8187      +/-   ##
==========================================
- Coverage   35.27%   35.27%   -0.01%     
==========================================
  Files         146      146              
  Lines        9326     9327       +1     
==========================================
  Hits         3290     3290              
- Misses       5637     5638       +1     
  Partials      399      399              

Impacted Files	Coverage Δ
cmd/minikube/cmd/start_flags.go	50.80% <0.00%> (-0.17%)	⬇️

[afbjorklund]

But the actual mount command is (pretty much) the same, from util-linux 2.35.1

So it must be something else that is different, most likely it's the kernel version ?

[afbjorklund]

Duh, sometimes it is the simple stuff. Like checking if the plug is actually in the wall socket.

Or making sure that tmpfs support is actually in the kernel... (maybe a default changed ?)

At least it boots:

But ssh is sad.

[afbjorklund]

ssh-keygen is segfaulting (when generating the host keys). That is probably bad...

The package was upgraded from 7.9p1 to 8.1p1, maybe something happened to it.

[afbjorklund]

Well, ssh starts now. But unfortunately we don't have any network (besides lo)

vboxguest: PCI device not found, probably running on physical hardware

lspci: /sys/bus/pci/devices: No such file or directory

Apparently CONFIG_VIRTIO_PCI went missing along with CONFIG_PCI.

[afbjorklund]

Good, now ssh is back on track as well. Maybe it will start now ?

 minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ cat /etc/os-release 
NAME=Buildroot
VERSION=2020.02.2
ID=buildroot
VERSION_ID=2020.02.2
PRETTY_NAME="Buildroot 2020.02.2"
$ 

[afbjorklund]

Well, it's always the little things.. No DNS, and other parts missing.

command_runner.go:102] ! sudo: pgrep: command not found

But beyond those, it looks happy enough! Will continue with it later.

NAME       STATUS   ROLES    AGE    VERSION
minikube   Ready    master   111s   v1.18.2

[afbjorklund]

The missing /etc/resolv.conf was for the same root cause as ssh.

Systemd didn't like LibreSSL, and I hadn't rebuilt everything...

minikube systemd-resolved[23193]: /usr/lib/systemd/systemd-resolved: error while loading shared libraries: libcrypto.so.45: cannot enable executable stack as shared object requires: Operation not permitted

It requires the libraries to be built with -Wa,--noexecstack

MemoryDenyWriteExecute=yes

---

And the missing access of k8s.gcr.io was for a different reason:

command_runner.go:102] ! bash: curl: command not found

[afbjorklund]

@medyagh : was there a way to share these "snapshot" ISO, or to make the PR build it ?

[afbjorklund]

Probably should try to get upstream to upgrade sysdig, rather than patching the old version.

I don't really know what we are using it for in minikube, but it was enabled in the old config...

[medyagh]

@medyagh : was there a way to share these "snapshot" ISO, or to make the PR build it ?

hm... there should be a way to at leasst have the ISO URL for manual testing