k3s cluster

Lightweight kubernetes cluster version v1.23.4+k3s1 with traefik ingress (default).

Cluster details

• 5 x raspberry Pi 3 nodes (1 API node, 4 worker nodes)
• https://dietpi.com/ OS
• system architecture armhf (arm7v64)
• provisioning tools:
  • sup
  • direnv
• HP Proliant microserver (persistence layer):
  • DBs
  • apt-cacher-ng for easier system updates
  • Grafana & Prometheus for node monitoring

Applications

• certs - lightweight certification management based on DNS, slightly modified to support k8s v1.23 API.
• NATS - really performant message queue.
• nfs-provisioner - bind NFS volumes as PVs.
• whoami - static HTTP application for testing setup (SSL, etc.)

Secrets

Secrets exported as shell environment variables, through .envrc files, see the direnv project for details

Application Monitoring (WIP)

• https://sysdig.com/blog/monitor-kubernetes-api-server/

TODO:

• Find a way to monitor containers without in-cluster prometheus instance.