Virtualbox based, vagrant provisioned Kubernets Homelab from skratch
looking for a simpler way? use the kind version: https://github.com/wlanboy/vagrantkind
Don't use minikube if you want to understand Kubernetes. This is not for developers searching for an easy way to k8s.
- directory debian-k3s
- Single minimal Debian based VM
- Using k3s
- Installing Calico, Metallb and Istio
- Ideal minimal starting place for Kubernetes
- directory homelab
- Single VM with k8 including network, storage, ingress provider
- Using kubeadm for setup
- Good start for beginners
- directory homelabrancher
- Single VM with k3 including network provider
- Using Arkade, k3sup for setup
- directory homelabstepbystep
- Single VM with all dependencies, java build tools and kubectl
- Good start to setup your k8 step by step
- directory homelabwithnodes
- Multy VM k8 cluster including network, storage, ingress provider
- You can add VMs by alterting the vagrant var NODE_COUNT
- Increase NODE_COUNT and run vagrat up again to add more VMs
- mattermost
- Kubernets files for a mattermost instance
- postgresqlservice
- Kubernets files for a PostgreSQL instance
- This is a tutorial
- This is a playground to get in touch with k3 and k8
- Normally you have to choose your own network provider, see: https://kubernetes.io/docs/concepts/cluster-administration/networking/
- Normally you have to choose your own storage provider, see: https://kubernetes.io/docs/concepts/storage/storage-classes/
- Normally you have to choose your own ingress provider, see: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
- You need an external load balancer provider too, the one from your cloud provider
- Kubernetes is a framework, not a toolset, see: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#what-kubernetes-is-not
- Consider to use a local apt cache server to minimize traffic
- We have to patch Flannel to use the second iface. First Interface in Virtualbox is natted. See: https://github.com/flannel-io/flannel/blob/master/Documentation/configuration.md
wget https://download.virtualbox.org/virtualbox/6.1.18/VirtualBox-6.1.18-142142-OSX.dmg
- Windows: (asuming you are using gitbash: https://gitforwindows.org/)
wget https://download.virtualbox.org/virtualbox/6.1.18/VirtualBox-6.1.18-142142-Win.exe
- OS X: homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
- Windows: chocolatey ( with powershell)
Set-ExecutionPolicy AllSigned
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
- OS X:
brew cask install vagrant
brew install kubernetes-cli
brew install helm
- Windows:
choco install vagrant
choco install kubernetes-cli
choco install kubernetes-helm
Please look at the Vagrant file first and adjust the values!
- at least we can now have ONE documentation
cd ~
git clone https://github.com/wlanboy/virtualbox-kubernets.git
cd homelab
vagrant up
cd ~/virtualbox-kubernets/homelab
vagrant destroy -f
cd ~/virtualbox-kubernets/homelab
vagrant up
mkdir -p ${HOME}/.kube
vagrant ssh -c "cat /home/vagrant/token.txt" > ${HOME}/.kube/token.txt
vagrant ssh -c "cat /home/vagrant/.kube/config" > ${HOME}/.kube/config
kubectl cluster-info
kubectl get nodes -o wide
mkdir -p ${HOME}/.kube
vagrant ssh kubecontrol -c "cat /home/vagrant/token.txt" > ${HOME}/.kube/token.txt
vagrant ssh kubecontrol -c "cat /home/vagrant/.kube/config" > ${HOME}/.kube/config
sed -i 's/192.168.50.100/192.168.56.100/g' ${HOME}/.kube/config
kubectl cluster-info
kubectl get nodes -o wide
cat ${HOME}/.kube/token.txt # <-- this is your token to login
kubectl proxy
- use your browser: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
kubectl apply -f longhorn-ingress.yml
- use your browser: http://192.168.56.100:31294/
kubectl cluster-info
Kubernetes control plane is running at https://192.168.56.100:6443
CoreDNS is running at https://192.168.56.100:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-vhv8h 0/1 Completed 0 3m48s
ingress-nginx ingress-nginx-admission-patch-767kf 0/1 Completed 0 3m48s
ingress-nginx ingress-nginx-controller-75c478c599-6s4d8 1/1 Running 0 3m48s
kube-system coredns-558bd4d5db-k9nmv 1/1 Running 0 3m48s
kube-system coredns-558bd4d5db-qxddh 1/1 Running 0 3m48s
kube-system etcd-kube 1/1 Running 0 4m3s
kube-system kube-apiserver-kube 1/1 Running 0 4m3s
kube-system kube-controller-manager-kube 1/1 Running 0 3m59s
kube-system kube-flannel-ds-fhwxt 1/1 Running 0 3m48s
kube-system kube-proxy-cq9ff 1/1 Running 0 3m48s
kube-system kube-scheduler-kube 1/1 Running 0 3m59s
kubernetes-dashboard dashboard-metrics-scraper-856586f554-xqmls 1/1 Running 0 3m48s
kubernetes-dashboard kubernetes-dashboard-78c79f97b4-4595v 1/1 Running 0 3m48s
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-r4cns 0/1 Completed 0 5m30s
ingress-nginx ingress-nginx-admission-patch-mm42b 0/1 Completed 2 5m30s
ingress-nginx ingress-nginx-controller-75c478c599-6pkr5 1/1 Running 0 5m29s
kube-system coredns-558bd4d5db-ls6nj 1/1 Running 0 5m29s
kube-system coredns-558bd4d5db-nfr6z 1/1 Running 0 5m29s
kube-system etcd-kubecontrol 1/1 Running 0 5m43s
kube-system kube-apiserver-kubecontrol 1/1 Running 0 5m43s
kube-system kube-controller-manager-kubecontrol 1/1 Running 0 5m46s
kube-system kube-flannel-ds-5wqdp 1/1 Running 0 2m56s
kube-system kube-flannel-ds-tfh7d 1/1 Running 0 5m30s
kube-system kube-proxy-5rg2s 1/1 Running 0 5m30s
kube-system kube-proxy-t447h 1/1 Running 0 2m56s
kube-system kube-scheduler-kubecontrol 1/1 Running 0 5m43s
kubernetes-dashboard dashboard-metrics-scraper-856586f554-5rrzp 1/1 Running 0 5m29s
kubernetes-dashboard kubernetes-dashboard-78c79f97b4-x4vwv 1/1 Running 0 5m29s
longhorn-system csi-attacher-5dcdcd5984-f4qlq 1/1 Running 0 4m25s
longhorn-system csi-attacher-5dcdcd5984-npclz 1/1 Running 0 4m25s
longhorn-system csi-attacher-5dcdcd5984-r4k2b 1/1 Running 0 4m25s
longhorn-system csi-provisioner-5c9dfb6446-rqxvx 1/1 Running 0 4m25s
longhorn-system csi-provisioner-5c9dfb6446-vwjfc 1/1 Running 0 4m25s
longhorn-system csi-provisioner-5c9dfb6446-zrds8 1/1 Running 0 4m25s
longhorn-system csi-resizer-6696d857b6-h9dwc 1/1 Running 0 4m25s
longhorn-system csi-resizer-6696d857b6-k4wxw 1/1 Running 0 4m25s
longhorn-system csi-resizer-6696d857b6-zgp8x 1/1 Running 0 4m25s
longhorn-system csi-snapshotter-96bfff7c9-l2frg 1/1 Running 0 4m24s
longhorn-system csi-snapshotter-96bfff7c9-q2jzn 1/1 Running 0 4m24s
longhorn-system csi-snapshotter-96bfff7c9-tpm9c 1/1 Running 0 4m24s
longhorn-system engine-image-ei-611d1496-5p42w 1/1 Running 0 2m26s
longhorn-system engine-image-ei-611d1496-wnxvb 1/1 Running 0 4m37s
longhorn-system instance-manager-e-0dea7bab 1/1 Running 0 4m37s
longhorn-system instance-manager-e-8a8af286 1/1 Running 0 116s
longhorn-system instance-manager-r-197160bf 1/1 Running 0 4m36s
longhorn-system instance-manager-r-bbe3230c 1/1 Running 0 115s
longhorn-system longhorn-csi-plugin-262gd 2/2 Running 0 4m24s
longhorn-system longhorn-csi-plugin-42hxl 2/2 Running 0 2m26s
longhorn-system longhorn-driver-deployer-ccb9974d5-6ghwh 1/1 Running 0 5m29s
longhorn-system longhorn-manager-4q47x 1/1 Running 0 2m26s
longhorn-system longhorn-manager-df55b 1/1 Running 0 5m13s
longhorn-system longhorn-ui-5b864949c4-499vx 1/1 Running 0 5m29s
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.104.5.33 <none> 80:30067/TCP,443:32469/TCP 7m52s
ingress-nginx-controller-admission ClusterIP 10.97.167.38 <none> 443/TCP 7m52s
- see: https://github.com/wlanboy/virtualbox-kubernets/blob/main/deploy-a-service.md
vagrant ssh
sudo su shutdown -h now
- change folder to 'homelabwithnodes'
- use 'vagrant ssh kubecontrol' instead of 'vagrant ssh'
- change NODE_COUNT var in Vagrant file to add additional hosts
- run 'vagrant up' after changing the NODE_COUNT, it will only add new vms
- change folder to 'homelabsteps'
- everything is prepared. Call each command from https://github.com/wlanboy/virtualbox-kubernets/blob/main/homelab/setup-kubernets-controlnode.sh
- change folder to 'homelabrancher'
- this will install a k3s kubernets server
- don't install vms by hand
- vms are containers too
- play around and test things, break things early and often
- automate everything to be able to rebuild anything, anytime
- think in roles not in serverboxes
- policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+, see: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125