Feature: Support Network Load Balanced Service

[kohidave]

Customers who want to expose non port 80/443 services (such as SMTP servers, Rabbit MQ servers, etc) can't use the Load Balanced Web Service pattern, as it only presents ports 80/443 to the world.

We should support the Network Load Balanced Service pattern which allows customers to expose multiple, arbitrary ports.

This pattern will be very close to the LB Web Service pattern we support now but with a few differences.

Differences:

• When we parse a Dockerfile for expose ports, we can just proxy them directly to the NLB
• Scaling policies will need to be different (non request based)
• Health Checks will need to be non-HTTP based
• Multiple listener rules based on ports exposed

Similar:

• HTTPS support if a customer has a DNS name on their app
• {svc}.{env}.{app}.domain pattern will be used for DNS delegation
• Port based listener rules
• No HTTP section

Some ideas we may want to explore:

1. Can we make it so a single service could be attached to both an ALB and an NLB? This would reduce the need to create a new pattern - but we'd have to find a way to let customers tell us which ports should be fronted by an ALB and which should be forwarded to NLB.
2. If we choose to create a separate pattern, how do we position it within the init workflow? TCP Service or something like that?

[kohidave]

@dajulia3 reached out on Gitter for this so we should run any ideas we have around this with him so he can see how it'd fit his use case :D

[dajulia3]

Re option 1 (attaching one service to both an ALB and NLB), I'd expect the service health to be reported as the aggregate of health checks across both ALB and NLB. So blue/green deploys, canary deploys, etc would need a "composite" health check that takes into account both ALB and NLB health checks.

I'm a bit torn as to whether option 1 or 2 is better. For those of us migrating large 8+ year old java apps (ahem me) to ECS, or even if I'm building a new Modular Monolith, I can see a strong case to be made for option 1. It's far easier to migrate an app to ECS if I don't have to introduce runtime configuration to run the same app as either listening for HTTP or TCP or split my monolith up into separate services based purely on protocol. That said, it does seem a bit harder to get the health checking/deployments right with option 1 vs splitting out TCP vs HTTP. I also know that @kohidave had an initial gut reaction that option 2 would be better, and I want to really dig into that-- I'm very new to ECS so I really appreciate the veteran opinion... maybe option 1 somehow is setting me up for pain :) .

Would love to help move this one along if at all possible. I really would love to use copilot rather than dropping down to CDK for this migration, but I appreciate that you all have a roadmap and priorities. And I super appreciate all the time that @kohidave took earlier today over gitter to discuss this. LOVE what you all are doing with copilot!

[efekarakus]

Heya!

We're leaning towards "Approach 1" for the design: single service attached to both an ALB and an NLB, with the following high-level proposal:

Load Balanced Web Services

This service type is going to be only for internet-facing load balancers.

• If the manifest contains the http section, then we'll create an internet-facing ALB:

http:
  path: '/'

• Alternatively, users should be able to expose ports with "tcp" or "udp", example: "6060/udp", which will result in the creation of an internet-facing NLB.

  The internet-facing NLB will be created in the Copilot environment and hence shared between other "Load Balanced Web Services". If a user specifies multiple LB Web Services that uses the same protocol and port then Copilot should throw an error.

• Users should be able to enable both a NLB and ALB or just one one of them.

Backend Services

Currently, Backend Services can be reached only via service discovery, but we'll evolve it to also work with internal load-balancers.

• If a user specifies a http section, then we'll create an internal ALB that's in the service stack and not the environment stack.
• If a user specifies ports that uses "tcp" or "udp", then we'll create an internal NLB that's in the service stack and not the environment stack.

[dajulia3]

Hey there @efekarakus that sounds great to me! One question/clarification
when you say " If a user specifies ports that uses "tcp" or "udp", then we'll create an internal ALB that's in the service stack and not the environment stack."
Did you mean to say create an internal NLB?

Or am I misunderstanding something?

[efekarakus]

My bad, I meant to type NLB! I fixed the comment above :)

[kohidave]

In terms of organizing in the manifest, one idea to help customers configure their TCP/UDP listeners would be to allow those sections in the manifest:

http:
 # Typical ALB listener rule
 path: '/'

udp:
  # Creates NLB UDP Listeners for port 69 and 70 - forwards to the main container
  publish: [69, 70]

tcp:
  # Creates four TCP listeners on the NLB. Two (123 and 124) to the main container 
  # and port 20 to the shell container, port 53 to the dns container. 
  publish: 
    - 123
    - 124
    - shell:20
    - dns:53

This would result in two load balancers being created - an ALB that handles HTTP 80/443 traffic, and an NLB which handles the other TCP/UDP traffic. This is just some high level thinking on what the manifest could look like. It obviously doesn't cover everything but be an interesting starting point. We'd still need to express things like target group stickiness, etc.

Alternatively, we could smoosh the publish section into the image/sidecar configurations. That'd minimize the new sections but may limit the expressiveness (group level stickiness).

name: api
type: Load Balanced Web Service

image:
  build: api/Dockerfile
  port: 3000
  # Enable JMX port on my main service
  publish: 9010/tcp 

http:
  path: 'api'
  healthcheck: '/api/health-check'

cpu: 256
memory: 512
count: 1

sidecars:
  nginx:
    port: 20
    # Expose port 20/tcp on the NLB
    publish: 20/tcp
    image: linuxserver/openssh-server

[iamhopaul123]

Hello @dajulia3, would you mind telling us how do you usually use NLBs: do you share one NLB by multiple services or it is usually strictly one NLB one service? Thank you!

[phishy]

What is the current recommended workaround for this? Configure an NLB manually after setting up with Copilot?

[iamhopaul123]

@phishy yeah if you want the NLB to be able to load traffics to multiple services then it is the only way. However, if you just want the NLB to work with one specific service then maybe you could define it as an addon to your service.

[Lou1415926]

Hello friends! We have a proposal for supporting network load balancer in #2918. Please Feel free to take a look - we'd love to hear your thoughts!

[efekarakus]

This feature was released was released for "Load Balanced Web Service" in v1.14 🥳 https://github.com/aws/copilot-cli/releases/tag/v1.14.0
Documentation: https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#nlb

We would love to hear your feedback, for example the first release does not have UDP support. We'd love to hear more about:

• What usecases of NLB (i.e. why) you'd like to have that's not currently supported
• What does your architecture look like (example: is the NLB for an internal service or public?)

Resolving the issue in the mean time.