feat: run local --watch flag #5413

CaptainCarpensir · 2023-10-26T21:32:37Z

Implements the --watch flag for run local which watches your copilot workspace for file changes and restarts the docker containers when you make changes.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

github-actions · 2023-10-26T21:43:04Z

🍕 Here are the new binary sizes!

Name	New size (kiB)	size (kiB)	Delta (%)
macOS (amd)	52332	51976	+0.68
macOS (arm)	53176	52812	+0.69
linux (amd)	46028	45716	+0.68
linux (arm)	45316	45060	+0.57
windows (amd)	43480	43168	+0.72

Lou1415926 · 2023-10-26T21:53:46Z

internal/pkg/docker/orchestrator/orchestrator.go

+
+		// ensure that containers are fully stopped after o.stopTask finishes blocking
+		// TODO(Aiden): Implement a container ID system or use `docker ps` to ensure containers are stopped
+		time.Sleep(1 * time.Second)


ah, what error do you see otherwise - something like "xx container is already running"? Is the "xx" consistent?

We'd see the following error:

[front-end] docker: Error response from daemon: Conflict. The container name "/basic-test-front-end-front-end" is already in use by container "1e49e6b3407f538ac362fd29607c9e38a1ed0309aae2a455a8ee63eea6e38b25". You have to remove (or rename) that container to be able to reuse that name. [front-end] See 'docker run --help'.

Danny said we could fix this by running docker ps to check when it's stopped but I think that tagging containers with a random id would be simpler and easier to implement. I can see that maybe users expect their containers to be named the same as it would be on ECS though.

ah I see, docker stop returns after the container is stopped, but this doesn't rm the container. To do this, docker rm is probably required, which is automatically run (because we've started the container by docker run --rm iirc) but not waited for.

Both docker ps -a and random container name (maybe by appending random string) sound like good solutions that are worth further considerations!

with a random id would be simpler and easier to implement.

probably yes - with the downside of polluting someone's docker engine with a bunch of tags.

…Carpensir/copilot-cli into cli/run-local-watch-flag

codecov-commenter · 2023-10-26T23:24:50Z

Codecov Report

Attention: 58 lines in your changes are missing coverage. Please review.

Comparison is base (e5aebd9) 69.99% compared to head (6349e0c) 69.97%.

❗ Current head 6349e0c differs from pull request most recent head dd9dd0b. Consider uploading reports for the commit dd9dd0b to get more accurate results

Additional details and impacted files

@@             Coverage Diff              @@
##           mainline    #5413      +/-   ##
============================================
- Coverage     69.99%   69.97%   -0.03%     
============================================
  Files           299      301       +2     
  Lines         45641    45789     +148     
  Branches        295      295              
============================================
+ Hits          31946    32039      +93     
- Misses        12148    12189      +41     
- Partials       1547     1561      +14

Files	Coverage Δ
internal/pkg/cli/flag.go	`90.47% <ø> (ø)`
internal/pkg/docker/orchestrator/orchestrator.go	`93.79% <0.00%> (-0.35%)`	⬇️
internal/pkg/cli/file/hidden.go	`0.00% <0.00%> (ø)`
internal/pkg/cli/file/watch.go	`68.42% <68.42%> (ø)`
internal/pkg/cli/run_local.go	`66.42% <67.54%> (-0.96%)`	⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

iamhopaul123

Have you looked into https://github.com/rjeczalik/notify 🤔 I wonder what the benefit would be for implementing our own wrapper to get a recursive watcher, given the fact that we'll introduce a 3rd party package anyway.

internal/pkg/cli/file/watch.go

internal/pkg/cli/file/hidden.go

internal/pkg/cli/file/watch.go

CaptainCarpensir · 2023-10-27T21:49:25Z

Have you looked into https://github.com/rjeczalik/notify 🤔 I wonder what the benefit would be for implementing our own wrapper to get a recursive watcher, given the fact that we'll introduce a 3rd party package anyway.

Will look into it, there's no benefit to writing our own wrapper as far as I know, I just didn't find any existing packages that did this well.

internal/pkg/cli/file/watch.go

internal/pkg/cli/run_local.go

internal/pkg/cli/file/watch.go

internal/pkg/cli/file/hidden.go

Lou1415926 · 2023-10-30T19:46:25Z

internal/pkg/cli/file/watch.go

+			if info.IsDir() {
+				switch event.Op {
+				case fsnotify.Create:
+					err := rw.Add(event.Name)


It seems like event.Name is relative to the "input" (https://pkg.go.dev/github.com/fsnotify/fsnotify#Event). Does rw.Add expects an absolute path?

Nope, rw.Add calls the underlying Add function in the fsnotify.Watcher. This function can be both absolute or relative. I think this can only become an issue if the relative position being called from changes. This should error out though, and because --watch programmatically calls this, it's just reusing the directory provided by ws.Path(), so we'll never run into this issue here anyways.

internal/pkg/cli/file/watch.go

CaptainCarpensir · 2023-10-30T22:24:19Z

Converted to draft to write local integration tests for watch.go

internal/pkg/cli/file/watch.go

internal/pkg/cli/run_local.go

internal/pkg/cli/file/watch.go

internal/pkg/cli/run_local.go

KollaAdithya

LGTM!

I assume TODO's are going to be finished after the release 🤔

internal/pkg/cli/file/watch.go

Co-authored-by: Adithya Kolla <[email protected]>

CaptainCarpensir · 2023-11-07T00:10:03Z

I assume TODO's are going to be finished after the release 🤔

Dockerignore file blacklist definitely won't be finished before this release. I don't think properly waiting for docker rm to finish will be able to either but if I have time I think I could implement it. The main blocker for that one is just deciding which of the two solutions is a better approach. For now I'm going to assume that both will be implemented for v1.33.

iamhopaul123

iamhopaul123 · 2023-11-07T17:54:41Z

internal/pkg/cli/file/watch_integration_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func TestRecursiveWatcher(t *testing.T) {


Awesome tests!

iamhopaul123 · 2023-11-07T17:56:27Z

internal/pkg/cli/file/filetest/watchertest.go

+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+package filetest


Do we use this?

Yeah it's used in run_local_test.go but maybe the package name should be changed.

internal/pkg/cli/run_local.go

internal/pkg/cli/file/hidden.go

internal/pkg/cli/file/watch.go

Lou1415926 · 2023-11-07T18:20:26Z

internal/pkg/cli/run_local.go

@@ -223,6 +237,10 @@ func newRunLocalOpts(vars runLocalVars) (*runLocalOpts, error) {
 		}
 		return containerURIs, nil
 	}
+	o.debounceTime = 5 * time.Second


why does this need to be a field of runLocalOpts 👀

For tests to set the debounce time to 0

internal/pkg/cli/run_local.go

Aiden Carpenter added 3 commits October 26, 2023 14:25

write file package to track filesystem changes

9b15223

run local --watch flag

3bb3f2f

reverse accidental line spacing

e265dd5

CaptainCarpensir requested a review from a team as a code owner October 26, 2023 21:32

CaptainCarpensir requested review from Lou1415926 and removed request for a team October 26, 2023 21:32

amazon-ecs-cli-v2-pr-manager requested a review from dannyrandall October 26, 2023 21:32

Lou1415926 reviewed Oct 26, 2023

View reviewed changes

Aiden Carpenter and others added 5 commits October 26, 2023 14:54

handle returned errors

423fafd

fix merge conflict

61bcafc

Merge branch 'mainline' into cli/run-local-watch-flag

14eb108

reorder test calls

8a7d8a2

Merge branch 'cli/run-local-watch-flag' of https://github.com/Captain…

9701f70

…Carpensir/copilot-cli into cli/run-local-watch-flag

Aiden Carpenter added 2 commits October 27, 2023 13:50

simplify debounce

21d6605

removed unused parameter

8ea2c38

iamhopaul123 reviewed Oct 27, 2023

View reviewed changes

internal/pkg/cli/file/watch.go Outdated Show resolved Hide resolved

internal/pkg/cli/file/hidden.go Show resolved Hide resolved

internal/pkg/cli/file/watch.go Outdated Show resolved Hide resolved

dannyrandall reviewed Oct 30, 2023

View reviewed changes

Lou1415926 reviewed Oct 30, 2023

View reviewed changes

address feedback

142bc12

CaptainCarpensir marked this pull request as draft October 30, 2023 22:23

Aiden Carpenter added 3 commits November 3, 2023 09:42

dont update for hidden files and folders

80aa093

watcher integration test

3ccfae2

localintegration build tag

0364052

CaptainCarpensir marked this pull request as ready for review November 3, 2023 17:26

Merge branch 'mainline' into cli/run-local-watch-flag

64dffc8

Aiden Carpenter added 6 commits November 3, 2023 11:02

fix merge errors

52ca8e7

remove race conditions

497df05

fix merge change causing tests to time out

b09f63a

buffer watcher for tests

e0c82a8

make integration test non-blocking on failure

28c34b7

two events

edcc10f

KollaAdithya reviewed Nov 6, 2023

View reviewed changes

internal/pkg/cli/file/watch.go Outdated Show resolved Hide resolved

internal/pkg/cli/run_local.go Show resolved Hide resolved

internal/pkg/cli/file/watch.go Outdated Show resolved Hide resolved

internal/pkg/cli/run_local.go Outdated Show resolved Hide resolved

Aiden Carpenter added 2 commits November 6, 2023 09:22

address adi feedback

563bdc3

make watcher close not handle outstanding events/errors

c361a7d

KollaAdithya approved these changes Nov 6, 2023

View reviewed changes

internal/pkg/cli/file/watch.go Outdated Show resolved Hide resolved

Update internal/pkg/cli/file/watch.go

6439229

Co-authored-by: Adithya Kolla <[email protected]>

Merge branch 'mainline' into cli/run-local-watch-flag

e4c2f41

iamhopaul123 approved these changes Nov 7, 2023

View reviewed changes

iamhopaul123 reviewed Nov 7, 2023

View reviewed changes

fix merge changes

193b84e

CaptainCarpensir added do-not-merge Pull requests that mergify shouldn't merge until the requester allows it. and removed do-not-merge Pull requests that mergify shouldn't merge until the requester allows it. labels Nov 7, 2023

Lou1415926 reviewed Nov 7, 2023

View reviewed changes

Aiden Carpenter and others added 2 commits November 7, 2023 10:56

address wanxian feedback, fix tests

6349e0c

Merge branch 'mainline' into cli/run-local-watch-flag

dd9dd0b

mergify bot merged commit 90d3af3 into aws:mainline Nov 7, 2023
12 checks passed

CaptainCarpensir deleted the cli/run-local-watch-flag branch November 13, 2023 17:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: run local --watch flag #5413

feat: run local --watch flag #5413

CaptainCarpensir commented Oct 26, 2023

github-actions bot commented Oct 26, 2023 •

edited

Loading

Lou1415926 Oct 26, 2023

CaptainCarpensir Oct 26, 2023

CaptainCarpensir Oct 26, 2023

Lou1415926 Oct 26, 2023

dannyrandall Oct 27, 2023

codecov-commenter commented Oct 26, 2023 •

edited

Loading

iamhopaul123 left a comment

CaptainCarpensir commented Oct 27, 2023 •

edited

Loading

Lou1415926 Oct 30, 2023

CaptainCarpensir Oct 30, 2023

CaptainCarpensir commented Oct 30, 2023

KollaAdithya left a comment

CaptainCarpensir commented Nov 7, 2023

iamhopaul123 left a comment

iamhopaul123 Nov 7, 2023

iamhopaul123 Nov 7, 2023

CaptainCarpensir Nov 7, 2023

Lou1415926 Nov 7, 2023

CaptainCarpensir Nov 7, 2023

feat: run local --watch flag #5413

feat: run local --watch flag #5413

Conversation

CaptainCarpensir commented Oct 26, 2023

github-actions bot commented Oct 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Oct 26, 2023 • edited Loading

Codecov Report

iamhopaul123 left a comment

Choose a reason for hiding this comment

CaptainCarpensir commented Oct 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CaptainCarpensir commented Oct 30, 2023

KollaAdithya left a comment

Choose a reason for hiding this comment

CaptainCarpensir commented Nov 7, 2023

iamhopaul123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 26, 2023 •

edited

Loading

codecov-commenter commented Oct 26, 2023 •

edited

Loading

CaptainCarpensir commented Oct 27, 2023 •

edited

Loading