diff --git a/changelog.d/4-docs/ingress-traffic b/changelog.d/4-docs/ingress-traffic new file mode 100644 index 0000000000..b539b90dd8 --- /dev/null +++ b/changelog.d/4-docs/ingress-traffic @@ -0,0 +1 @@ +Document the new ingress-nginx-controller, current recommendations, and adjust older documentation on the topic diff --git a/charts/ingress-nginx-controller/values.yaml b/charts/ingress-nginx-controller/values.yaml index d8fd654f93..fbe933f2a3 100644 --- a/charts/ingress-nginx-controller/values.yaml +++ b/charts/ingress-nginx-controller/values.yaml @@ -24,14 +24,6 @@ ingress-nginx: # upstream services. Comes with tradeoff considerations, see # documentation on "ingress" on docs.wire.com externalTrafficPolicy: Local - nodePorts: - # If you set service.type = NodePort, then the nginx controller instance - # is exposed on ports 31773 (https) and 31772 (http) on the node on - # which it runs. You should add a port-forwarding rule on the node or on - # the loadbalancer that forwards ports 443 and 80 to these respective - # ports. - https: 31773 - http: 31772 config: # NOTE: These are some sane defaults (compliant to TR-02102-2), you may # want to overrride them on your own installation For TR-02102-2 see diff --git a/docs/src/how-to/install/ingress.md b/docs/src/how-to/install/ingress.md index 03a4329052..72800e3e31 100644 --- a/docs/src/how-to/install/ingress.md +++ b/docs/src/how-to/install/ingress.md @@ -38,8 +38,19 @@ To inspect default TLS settings, see [defaults in the latest code](https://githu ## Installing on bare-metal without dynamic load balancer support -In case you cannot create a `kind: service` of `type: LoadBalancer`, then you -can fall back to manually ensure traffic reaches your installation: +```{note} +Most managed K8s clusters have support for LoadBalancers. Kubernetes clusters that are manually set up can also support this, by using a provider/environment-specific [CCM (Cloud Controller Manager)](https://kubernetes.io/docs/concepts/architecture/cloud-controller/) (see hcloud and digitalocean for examples). + +In case you're provisioning on bare metal, without any hardware load balancer support in front, +you might be using MetalLB, which supports BGP or Failover [L2 ARP announcements](https://metallb.universe.tf/configuration/_advanced_l2_configuration/). + +Please read [baremetal deployment documentation](https://kubernetes.github.io/ingress-nginx/deploy/baremetal/) + +The choice of CCM highly depends on the environment you choose to deploy wire-server in. +``` + +In case you cannot change the type of kubernetes cluster you run, and therefore cannot create a `kind: service` of `type: LoadBalancer`, then you +can fall back to NodePorts and manually ensure traffic reaches your installation: ```yaml # helmfile.yaml @@ -61,6 +72,9 @@ ingress-nginx: kind: DaemonSet service: type: NodePort + nodePorts: + https: 31773 + http: 31772 ``` Then, on each of your kubernetes worker nodes, two ports are exposed: ports @@ -71,8 +85,6 @@ forwards ports 443 and 80 to these respective ports. Any traffic hitting the htt Downsides of this approach: The NodePort approach always requires manual configuration of some external load balancer/firewall to round-robin between node IPs and is error-prone. It's also a bit annoying to have to decide on some global ports that may not be used otherwise. -Most managed K8s clusters have support for LoadBalancers, you can also get this for your own clusters in hcloud etc. It's even possible to do it for pure bare metal, without any "load balancer hardware", by using BGP or some leadership election over who's announcing the "load balancer ip" via ARP (https://metallb.universe.tf/configuration/_advanced_l2_configuration/). - ### Using NodePort (not the default) with externalTrafficPolicy=Local (the default) Normally, NodePort will listen to traffic on all nodes, and uses kube-proxy @@ -95,8 +107,8 @@ redirect them to the correct service, whilst maintaining the source ip address. The ingress controller is sort of taking over the role of what kube-proxy was doing before. More information: -- https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typenodeport -- https://kubernetes.github.io/ingress-nginx/deploy/baremetal/ +- [source-ip](https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typenodeport) +- [ingress and baremetal](https://kubernetes.github.io/ingress-nginx/deploy/baremetal/) There are also downsides to setting `externalTrafficPolicy: Local`, please look at the [following blog post](https://www.asykim.com/blog/deep-dive-into-kubernetes-external-traffic-policies), which very clearly explains the upsides and downsides of this setting