Reduce/Adjust/Manage the logging from svc deploy

[coopsmoss]

Hi there,

I'm wondering if anyone has any ideas on how better to handle the logging from svc deploy in a ci job.
The output overwrites itself on a local machine but when you're dealing with log files it looks like bellow. With all of that being printed out ten times a second. I think it's actually really slowing down the job too managing some sort of log piping around.

- Updating the infrastructure for stack api-test-v1                           [update in progress]  [537.3s]
  - An ECS service to run and maintain your tasks in the environment cluster  [update in progress]  [513.8s]
    Deployments                                                                                     
               Revision  Rollout        Desired  Running  Failed  Pending                           
      PRIMARY  10        [in progress]  1        0        2       0                                 
      ACTIVE   9         [completed]    1        1        0       0                                 
  - An ECS task definition to group your containers and run them on ECS       [update complete]     [6.3s]
- Updating the infrastructure for stack api-test-v1                           [update in progress]  [537.4s]
  - An ECS service to run and maintain your tasks in the environment cluster  [update in progress]  [513.9s]
    Deployments                                                                                     
               Revision  Rollout        Desired  Running  Failed  Pending                           
      PRIMARY  10        [in progress]  1        0        2       0                                 
      ACTIVE   9         [completed]    1        1        0       0                                 
  - An ECS task definition to group your containers and run them on ECS       [update complete]     [6.3s]
- Updating the infrastructure for stack api-test-v1                           [update in progress]  [537.5s]
  - An ECS service to run and maintain your tasks in the environment cluster  [update in progress]  [514.0s]
    Deployments                                                                                     
               Revision  Rollout        Desired  Running  Failed  Pending                           
      PRIMARY  10        [in progress]  1        0        2       0                                 
      ACTIVE   9         [completed]    1        1        0       0                                 
  - An ECS task definition to group your containers and run them on ECS       [update complete]     [6.3s]
- Updating the infrastructure for stack api-test-v1                           [update in progress]  [537.6s]
  - An ECS service to run and maintain your tasks in the environment clus
  ....etc.

And I exceed the log file length (4194304 bytes) and can't tell whats doing on.
I've considered outputting the logs to a file and then printing the head and tail afterwards but then you can see whats going on.

Hoping someone has a tip/solution, thanks.

[coopsmoss]

Currently this is what I'm doing:

  script:
    - echo "Building & Deploying Container...."
     # pipe to a logfile and then print the head and tail
     # the cli outputs a status 10 times a second and will fill up the log file too soon
    - copilot svc deploy --app api --name v1 --env $COPILOT_ENV &> deploylog.txt
  after_script: # Print out here because this will run even if script fails
    - echo "1000 line head:"
    - head -n -1000 deploylog.txt # First thousand lines
    - echo ........................ END OF HEAD .............................
    - echo "100 line tail:"
    - tail -n -100 deploylog.txt # Last 100 lines
    - echo "Deploy Stage End"

Basically just log it in a file then print out the head and tail

[efekarakus]

Hi @coopsmoss ! Thanks for reaching out, yeah we don't have a nice way to disable the spam of logs at the moment 😭

We should be detecting if the file description is a TTY and then perhaps just print only on state transitions. Unfortunately, we don't have a fix for it atm.

[coopsmoss]

Appreciate the reply, this will do for now.

[efekarakus]

Hi @coopsmoss #2440 should hopefully make this more manageable! It increases the render interval from 0.1s to 30s if the environment variable CI is set to true

[coopsmoss]

Fantastic. You guys are the best!

[efekarakus]

The fix is out now in v1.8.0! https://github.com/aws/copilot-cli/releases/tag/v1.8.0