fix: allow multiple environments per VPC #2394

bvtujo · 2021-05-28T16:34:22Z

This PR modifies the default service discovery namespace for new environments to the form {env}.{app}.local. This change allows customers to place multiple environments in the same VPC.

The PR also creates an upgrade path for older environments. Those upgraded environments will retain the default app.local namespace to avoid a backward breaking change for services deployed there, but new environments will use env.app.local.

In the image above, I have deployed the same service to 5 environments.
test and beta were created at env version 1.4.0 and upgraded to 1.5.0.
delta-shared-vpc was created at version 1.5.0 shares its vpc and subnets with beta.
epsilon-shared-test was created at 1.5.0 and shares its vpc and subnets with test.
gamma is a new environment.

There currently is not an upgrade path for existing environments to switch to new service discovery. This should not be an issue in most cases, however, as the environment variable points to the correct namespace in all cases and new+old environments can coexist in the same VPC.

This change does not disrupt any existing environments. If we eventually add an "environment v2.0.0," we can force migration to the new discovery service by removing the UseLegacyServiceDiscoveryIfBlank parameter and updating our workloads to remove the LegacyServiceDiscovery bool from the template package.

Related #2377

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

This is necessary because services can only have ONE registry, not multiple, as would be suggested by the fact that it's an array. https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateService.html#API_CreateService_RequestParameters

bvtujo · 2021-05-28T16:46:48Z

For easy comparison, an annotated diff between v1.4.0 and 1.5.0

5c5
<   Version: 'v1.4.0'
---
>   Version: 'v1.5.0'
30a31,33
>   UseLegacyServiceDiscoveryIfBlank:
>     Type: String
>     Default: ""
44a48,49
>   UseLegacyServiceDiscovery:
>     !Equals [ !Ref UseLegacyServiceDiscoveryIfBlank, "" ]

New parameter and new condition. If the environment is upgraded, this is left blank. If the environment is new, this parameter's value is set to "doNotCreate". The condition is used to change the value of a stack output.

51a57
>   # This is a legacy construct and only used in environments created before v1.5.0.

Change to comment on old svc discovery namespace.

60a67,77
>   # Creates a service discovery namespace with the form:
>   # {svc}.{env}.{appname}.local
>   ServiceDiscoveryNamespaceWithEnv:
>     Type: AWS::ServiceDiscovery::PrivateDnsNamespace
>     Properties:
>       Name: !Sub ${EnvironmentName}.${AppName}.local
> {{- if .ImportVPC}}
>       Vpc: {{.ImportVPC.ID}}
> {{- else}}
>       Vpc: !Ref VPC
> {{- end}}

New namespace.

304a323
>     Condition: UseLegacyServiceDiscovery

Add a condition to the old service discovery output

307a325,332
>   ServiceDiscoveryNamespaceIDWithEnv:
>     Value: !GetAtt ServiceDiscoveryNamespaceWithEnv.Id
>     Export:
>       Name: !Sub ${AWS::StackName}-ServiceDiscoveryNamespaceIDWithEnv
>   UseLegacyServiceDiscovery:
>     Value: !If [ UseLegacyServiceDiscovery, true, false ]
>     Export:
>       Name: !Sub ${AWS::StackName}-UseLegacyServiceDiscovery

Create a new output with the new namespace, and an output to use to tell svc deploy whether to use the legacy namespace or the new one.

bvtujo · 2021-05-28T17:48:32Z

Just realized this leaves the env template in a state with modes--applying the do-not-merge label while I fix that.

Ed: the environment template is now non-modal. Both namespaces are always created, and it's up to svc deploy to choose which namespace to register a service with based on whether the environment's UseLegacyServiceDiscovery output is "true" or empty.

ed: non-modal behavior breaks the ability of environments to coexist. Kept the check on the output of the stack, but only create the legacy namespace when the input parameter is empty.

Added the conditions back to the export and the legacy service discovery namespace. Turns out the condition was necessary in order for environments to coexist in the same VPC.

huanjani · 2021-05-28T21:35:39Z

Wow, what a comprehensive PR-- you have all the things, even including Japanese docs!
Quick question: do we need to worry about the 1024 limit for namespace names? Is it worth building in a check/truncation?

bvtujo · 2021-06-02T18:06:27Z

@huanjani I don't believe so; we cap service, app, and environment names at 255 characters so we'll never hit that limit!

iamhopaul123

Can't believe this will be fixed so quickly. Thank you!

iamhopaul123 · 2021-06-03T15:59:02Z

internal/pkg/deploy/cloudformation/env.go

+	}
+	stackOutputs := make(map[string]string)
+	for _, output := range descr.Outputs {
+		stackOutputs[*output.OutputKey] = *output.OutputValue


Shall we use aws.StringValue() for consistency?

iamhopaul123 · 2021-06-03T16:17:38Z

templates/environment/versions/cf-v1.5.0.yml

+    Value: !GetAtt ServiceDiscoveryNamespaceWithEnv.Id
+    Export:
+      Name: !Sub ${AWS::StackName}-ServiceDiscoveryNamespaceIDWithEnv
+  UseLegacyServiceDiscovery:


Why do we need this output? Can't we check if UseLegacyServiceDiscoveryIfBlank this param is blank to tell?

iamhopaul123 · 2021-06-03T16:37:17Z

internal/pkg/deploy/env.go

@@ -13,7 +13,7 @@ const (
 	// LegacyEnvTemplateVersion is the version associated with the environment template before we started versioning.
 	LegacyEnvTemplateVersion = "v0.0.0"
 	// LatestEnvTemplateVersion is the latest version number available for environment templates.
-	LatestEnvTemplateVersion = "v1.4.0"
+	LatestEnvTemplateVersion = "v1.5.0"


Just a reminder if #2413 is merged earlier we need to include changes in v1.5.0

iamhopaul123 · 2021-06-03T16:43:36Z

templates/workloads/partials/cf/servicediscovery.yml

+      {{- if .LegacyServiceDiscovery}}
        !Sub '${AppName}-${EnvName}-ServiceDiscoveryNamespaceID'
+      {{- else}}
+        !Sub '${AppName}-${EnvName}-ServiceDiscoveryNamespaceIDWithEnv'
+      {{- end}}


Do we want to include both if they recently upgraded their environment? since in the env template if they were using legacy sd we also add new sd for them. Or we don't create new sd if they are using legacy at all.

iamhopaul123 · 2021-06-03T16:45:37Z

internal/pkg/cli/job_deploy.go

-			AdditionalTags:    tags.Merge(o.targetApp.Tags, o.resourceTags),
+			AddonsTemplateURL:      addonsURL,
+			AdditionalTags:         tags.Merge(o.targetApp.Tags, o.resourceTags),
+			LegacyServiceDiscovery: svcDiscovery,


does copilot job use service discovery 🤔

We inject the endpoint as an environment variable, so we still have to update this here.

efekarakus

Sorry I haven't had the time to read the PR completely! Can you explain to me the sequence of events for this fix? I think it would be nice to have a 1-2 page short doc/issue explaining how you got this resolved to the team as it's on the critical path.

Can we do a flow that's like this?

1. If running `env init`, then create service discovery with new format
2. If running `env upgrade`:
   2.1. Keep old service discovery namespace
   2.2. Add the new service discovery namespace
3. In svc templates only use the new service discovery namespace

Instead of passing CFN parameters, I wonder if we can use Go templates to decide if we want to keep the old svc discovery

efekarakus · 2021-06-07T16:11:05Z

internal/pkg/cli/svc_deploy.go

-			AdditionalTags:    tags.Merge(o.targetApp.Tags, o.resourceTags),
+			AddonsTemplateURL:      addonsURL,
+			AdditionalTags:         tags.Merge(o.targetApp.Tags, o.resourceTags),
+			LegacyServiceDiscovery: svcDiscovery,


I wonder if this fits under RuntimeConfig. The original idea behind this struct was to capture additional configuration passed by the client when calling deploy: 1) the url for addons/ template 2) if they use the deploy --resource-tags flag.

Maybe we can push down the LegacyServiceDiscovery one level below into the stack pkg? so that the clients have to do less work

internal/pkg/cli/svc_package.go

efekarakus · 2021-06-07T16:19:29Z

internal/pkg/deploy/cloudformation/env.go

+	output, ok := stackOutputs[stack.EnvOutputLegacyServiceDiscovery]
+	if !ok || output != "false" {
+		return true, nil
+	}


Do we need the UseLegacyServiceDiscovery output? Can we instead check if we have the new output ServiceDiscoveryNamespaceIDWithEnv in the template?

bvtujo · 2021-06-24T20:32:45Z

Closing and reopening due to merge conflicts.

bvtujo added 10 commits May 27, 2021 21:43

chore: add Parameters method to aws package

524454a

chore: add EnvironmentParameters method to cloudformation

ad9f6f0

chore: add template changes for service and new environment version

3f34ed9

chore: add new features to workloads in stack pkg

f4b9a71

chore: add LegacySvcDisco to job runtime config

f1149c9

chore: fix integration tests

1b516e6

chore: fix integ tests

6bced2e

chore: go mod tidy

c20eb72

chore: rewrite comment in env template

b3db3d5

bvtujo requested a review from a team as a code owner May 28, 2021 16:34

bvtujo requested a review from efekarakus May 28, 2021 16:34

amazon-ecs-cli-v2-pr-manager requested a review from Lou1415926 May 28, 2021 16:34

Merge branch 'mainline' into svc-discovery-env

6cc8758

bvtujo added the do-not-merge Pull requests that mergify shouldn't merge until the requester allows it. label May 28, 2021

chore: update CF stack to be non-modal

6c55fb3

bvtujo removed the do-not-merge Pull requests that mergify shouldn't merge until the requester allows it. label May 28, 2021

bvtujo added 4 commits May 28, 2021 18:55

chore: revert change removing conditions

044fbf8

Added the conditions back to the export and the legacy service discovery namespace. Turns out the condition was necessary in order for environments to coexist in the same VPC.

chore: always output a value

7b6a55f

docs: aupdate service discovery docs

8e6679e

chore: fix unit tests

106f111

bvtujo requested a review from toricls as a code owner May 28, 2021 19:53

bvtujo added 3 commits June 2, 2021 21:09

chore: job package picks up env stack output for svc disco

760024f

Merge branch 'mainline' into svc-discovery-env

88ee6a9

chore: remove bad print statement

78573aa

bvtujo added 2 commits June 2, 2021 21:19

chore: staticcheck

09bcc28

fix: segfault in job package

5f844d0

iamhopaul123 reviewed Jun 3, 2021

View reviewed changes

efekarakus reviewed Jun 7, 2021

View reviewed changes

efekarakus self-assigned this Jun 7, 2021

efekarakus added the do-not-merge Pull requests that mergify shouldn't merge until the requester allows it. label Jun 16, 2021

bvtujo closed this Jun 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: allow multiple environments per VPC #2394

fix: allow multiple environments per VPC #2394

bvtujo commented May 28, 2021 •

edited

Loading

bvtujo commented May 28, 2021 •

edited

Loading

bvtujo commented May 28, 2021 •

edited

Loading

huanjani commented May 28, 2021

bvtujo commented Jun 2, 2021

iamhopaul123 left a comment

iamhopaul123 Jun 3, 2021

iamhopaul123 Jun 3, 2021

iamhopaul123 Jun 3, 2021

iamhopaul123 Jun 3, 2021

iamhopaul123 Jun 3, 2021

bvtujo Jun 7, 2021

efekarakus left a comment

efekarakus Jun 7, 2021

efekarakus Jun 7, 2021

bvtujo commented Jun 24, 2021

fix: allow multiple environments per VPC #2394

fix: allow multiple environments per VPC #2394

Conversation

bvtujo commented May 28, 2021 • edited Loading

bvtujo commented May 28, 2021 • edited Loading

bvtujo commented May 28, 2021 • edited Loading

huanjani commented May 28, 2021

bvtujo commented Jun 2, 2021

iamhopaul123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

efekarakus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bvtujo commented Jun 24, 2021

bvtujo commented May 28, 2021 •

edited

Loading

bvtujo commented May 28, 2021 •

edited

Loading

bvtujo commented May 28, 2021 •

edited

Loading