Switch Ingress ELB to a network load balancer

[dghubble]

Typhoon clusters include an AWS load balancer for pairing with Ingress Controller deployments to terminate multiple HTTP and HTTPS applications exposed via Ingress.

Until now, Typhoon has used elastic load balancers (ELBs) (now called "classic balancers"). AWS now offers Network Load Balancers (NLBs) which can handle millions of requests per second. This is important given that the balancer multiplexes or terminates multiple end-user applications.

Impact:

• Create an Ingress network load balancer instead of an elastic load balancer
• Workers are added to worker-http and workers-https target groups (similar to GCP target pools)
• Relax the firewall rules to expose the Ingress health port. NLBs do not have their own security group, so they cannot be independently permitted to health check Ingress controllers.
• Require terraform-provider-aws 1.7 or higher (NLBs were introduced in 1.1, but it took until 1.7 for the bugs to be sorted out)

Testing

Used for the maintainer's AWS clusters and to serve a few HTTPS apps via Ingress. This has been used with Nginx Ingress 0.10.2 and 0.11.0.