... managed by Flux and serviced with RenovateBot 🤖
This repository is my home Kubernetes cluster in a declarative state. Flux watches my cluster folder and makes the changes to my cluster based on the YAML manifests.
Feel free to open a Github issue or join the k8s@home Discord if you have any questions.
This repository is built off the k8s-at-home/template-cluster-k3s repository.
This cluster consists of both VMs provisioned on PVE with and without the Terraform Proxmox provider as well as physical nodes. These run k3s provisioned overtop Ubuntu 20.04 using the Ansible galaxy role ansible-role-k3s. This is a semi hyper-converged cluster, workloads and block storage are sharing the same available resources on my nodes while I have a separate server for (NFS) file storage.
See my server/ansible directory for my playbooks and roles, and server/terraform for infrastructure provisioning.
- metallb: Provides a load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols.
- calico: For internal cluster networking.
- traefik: Provides ingress cluster services.
- rook-ceph: Provides persistent volumes, allowing any application to consume RBD block storage.
- SOPS: Encrypts secrets which is safe to store - even to a public repository.
- cert-manager: Configured to create TLS certs for all ingress services automatically using LetsEncrypt.
The Git repository contains the following directories under cluster and are ordered below by how Flux will apply them.
- base directory is the entrypoint to Flux
- crds directory contains custom resource definitions (CRDs) that need to exist globally in my cluster before anything else exists
- core directory (depends on crds) are important infrastructure applications (grouped by namespace) that should never be pruned by Flux
- apps directory (depends on core) is where my common applications (grouped by namespace) are placed.
cluster
├── apps
├── base
├── core
└── crds
| Tool | Purpose |
|---|---|
| ansible | Preparing Ubuntu for Kubernetes and installing k3s |
| flux | Operator that manages your k8s cluster based on your Git repository |
| go-task | A task runner / simpler Make alternative written in Go |
| dir-env | Sets environment variable based on present working directory |
| sops | Encrypts k8s secrets with GnuPG |
| Node | Hostname | CPU | RAM | Storage | Function | Operating System |
|---|---|---|---|---|---|---|
| Lenovo M72e | k8s-master-lt9 | 2C4T i5 3470T | 8GB | 180GB SSD | Kube Master Node | Ubuntu 20.04.3 LTS |
| Lenovo M72e | k8s-master-lx0 | 2C4T i5 3470T | 8GB | 180GB SSD | Kube Master Node | Ubuntu 20.04.3 LTS |
| Lenovo M72e | k8s-master-lr3 | 2C4T i5 3470T | 8GB | 180GB SSD | Kube Master Node | Ubuntu 20.04.3 LTS |
| Proxmox VM w/o GPU passthrough | k8s-w-pve | 6C i5 10400 | 16GB | 120GB SSD OS, 350GB NVME ceph virtual block storage | Kube Worker Node | Ubuntu 20.04.3 LTS |
| Proxmox VM w/ Intel iGPU passthrough | k8s-w-pvegpu | 4C i7 8700k | 12GB | 120GB SSD OS, 350GB NVME ceph virtual block storage | Kube Worker Node | Ubuntu 20.04.3 LTS |
| HP590 | k8s-w-590 | 4C i3 10100 | 16GB | 500GB SSD OS, 500GB NVME ceph | Kube Worker Node | Ubuntu 20.04.3 LTS |
| PC | Hostname | CPU | RAM | Storage | Function | Operating System |
|---|---|---|---|---|---|---|
| Unraid on Proxmox pve | NAS | 8C i5 10400 | 8GB | 3 x 8TB HDD, 3TB HDD, 2 x 240GB SSD | NFS Server | unRaid |
| Lenovo 310s | pfsense | 4C j4205 | 8GB | 100GB SSD | Router | pfsense |
| Node 304 NAS/Virtualization Host | pve | i5 10400 | 64GB | 500GB SSD, 500GB NVME | Virtualization host | Proxmox 7 |
| Fractal Design NR200 Virtualization Host | pve8700k | i7 8700k | 32GB | 500GB SSD, 1TB NVME | Virtualization host | Proxmox 7 |
A lot of inspiration for my cluster came from the people that have shared their clusters over at kubesearch.dev