Compare a Buildroot-based VM with an Ubuntu-based VM

[afbjorklund]

As part of the preparations for #9992, we should compare the two OS (i.e. Linux distributions)

The regular minikube.iso:

minikube start --driver=virtualbox --memory=2048 --cpus=2

With the vagrant equivalent:

(see #12185 for the setup, basically minikube start --driver=ssh ...)

Note down the differences when it comes to boot time and start time, and disk and memory usage.

---

$ grep PRETTY /etc/os-release 
PRETTY_NAME="Buildroot 2020.02.12"
$ uname -r
4.19.182
$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1985         624         163         690        1198         519
Swap:             0           0           0
$ df -h / /mnt/sda1
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.8G  674M  1.1G  38% /
/dev/sda1        17G  1.4G   15G   9% /mnt/sda1

vagrant@minikube:~$ grep PRETTY /etc/os-release 
PRETTY_NAME="Ubuntu 20.04.2 LTS"
vagrant@minikube:~$ uname -r
5.4.0-80-generic
vagrant@minikube:~$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1987         192         712           0        1081        1627
Swap:             0           0           0
vagrant@minikube:~$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        39G  1.9G   37G   5% /

[afbjorklund]

Once we remove the container runtimes from the default buildroot installation, the memory usage will be a lot less.

See #9989

At least if we store them on the disk image instead of on ram disk (tmpfs), like we do with the binaries and images:

/var/lib/minikube/binaries/

/var/lib/docker/
/var/lib/containers/
/var/lib/containerd/
/var/lib/buildkit/