Skip to content

Commit 59b62e0

Browse files
committed
container stats: ignore context.Canceled errors
we often receive context.Canceled errors when a container exits during docker stats collection. there is a benign race condition where if we process the error first before processing that the context is "Done", then we will log this canceled error as a warning message here: https://github.com/aws/amazon-ecs-agent/blob/5be7aa08bed215a557f48c16d8201ad3db59a9be/agent/stats/container.go#L118-L122 this change ignores these context.Canceled errors so that we don't log them. This will eliminate log messages that look like this when a container exits: level=warn time=2020-12-25T07:51:33Z msg="Error encountered processing metrics stream from docker, this may affect cloudwatch metric accuracy: DockerGoClient: Unable to decode stats for container REDACTED: context canceled" module=container.go
1 parent 8784e86 commit 59b62e0

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

agent/stats/container.go

+7-2
Original file line numberDiff line numberDiff line change
@@ -118,8 +118,13 @@ func (container *StatsContainer) processStatsStream() error {
118118
case <-container.ctx.Done():
119119
return nil
120120
case err := <-errC:
121-
seelog.Warnf("Error encountered processing metrics stream from docker, this may affect cloudwatch metric accuracy: %s", err)
122-
returnError = true
121+
select {
122+
case <-container.ctx.Done():
123+
// ignore error when container.ctx.Done()
124+
default:
125+
seelog.Warnf("Error encountered processing metrics stream from docker, this may affect cloudwatch metric accuracy: %s", err)
126+
returnError = true
127+
}
123128
case rawStat, ok := <-dockerStats:
124129
if !ok {
125130
if returnError {

0 commit comments

Comments
 (0)