Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: TestServerGracefulClose failures with connection reset by peer #57084

Closed
gopherbot opened this issue Dec 5, 2022 · 17 comments
Closed

net/http: TestServerGracefulClose failures with connection reset by peer #57084

gopherbot opened this issue Dec 5, 2022 · 17 comments
Assignees
@bcmills
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Milestone
Go1.22

Comments

@gopherbot
Copy link
Contributor

gopherbot commented Dec 5, 2022 •
edited by bcmills
Loading 

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`

Issue created automatically to collect these failures.

Example (log):

--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.24s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:33546->127.0.0.1:33545: read: connection reset by peer

— watchflakes
@gopherbot gopherbot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 5, 2022
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose"
2022-12-02 20:16 freebsd-riscv64-unmatched go@5efa4dab net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.24s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:33546->127.0.0.1:33545: read: connection reset by peer

— watchflakes

@bcmills bcmills mentioned this issue Dec 6, 2022
Closed
@bcmills bcmills changed the title net/http: TestServerGracefulClose failures net/http: TestServerGracefulClose failures with connection reset by peer Dec 6, 2022
@bcmills
Copy link
Contributor

bcmills commented Dec 6, 2022

(attn @neild)

@bcmills
Copy link
Contributor

bcmills commented Dec 6, 2022

See also #36178.

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2022-12-05 16:41 freebsd-arm64-dmgk go@ad55b878 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (4.89s)
        serve_test.go:3153: ReadLine: read tcp 127.0.0.1:29763->127.0.0.1:29762: read: connection reset by peer
2022-12-19 21:44 darwin-amd64-12_0 go@7683c71a net/http.TestServerGracefulClose (log) 
2022/12/19 20:58:18 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc00053a008 127.0.0.1:50540 in state active
2022/12/19 20:58:18 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc00053a118 127.0.0.1:50559 in state active
--- FAIL: TestServerGracefulClose (3.00s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50835->127.0.0.1:50834: read: connection reset by peer
2022-12-21 19:08 darwin-amd64-nocgo go@856ec210 net/http.TestServerGracefulClose (log) 
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000096040 127.0.0.1:50524 in state active
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000203880 127.0.0.1:50527 in state active
2022/12/21 19:27:32 httptest.Server blocked in Close after 5 seconds, waiting for connections:
  *net.TCPConn 0xc000174070 127.0.0.1:50540 in state active
--- FAIL: TestServerGracefulClose (2.41s)
    serve_test.go:3144: ReadLine: read tcp 127.0.0.1:50784->127.0.0.1:50783: read: connection reset by peer
2022-12-22 19:21 darwin-amd64-12_0 go@91bc4cd1 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (2.75s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50514->127.0.0.1:50513: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-01-04 22:23 darwin-amd64-11_0 go@e04be8b2 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (4.77s)
    serve_test.go:3151: ReadLine: read tcp 127.0.0.1:50617->127.0.0.1:50616: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-02-09 15:21 dragonfly-amd64-622 go@3161081c net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.11s)
        serve_test.go:3149: ReadLine: read tcp 127.0.0.1:50143->127.0.0.1:58048: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-03-28 17:34 netbsd-arm64-bsiegert go@c02e1bfb net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.80s)
        serve_test.go:3099: ReadLine: read tcp 127.0.0.1:58793->127.0.0.1:58794: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-03 22:13 netbsd-arm64-bsiegert go@f742ddc3 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.05s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:63207->127.0.0.1:63208: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-10 21:43 dragonfly-amd64-622 go@8e8f4be3 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.01s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:56413->127.0.0.1:51576: read: connection reset by peer

— watchflakes

@bcmills bcmills added this to the Backlog milestone May 17, 2023
@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-05-16 21:06 freebsd-arm64-dmgk go@298ff309 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.94s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:57637->127.0.0.1:57636: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-06-06 15:33 dragonfly-amd64-622 go@46249e56 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (1.02s)
        serve_test.go:3105: ReadLine: read tcp 127.0.0.1:50051->127.0.0.1:64824: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-07-18 05:23 netbsd-arm-bsiegert go@28ca813a net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (2.30s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:54521->127.0.0.1:54522: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-08-17 16:33 freebsd-arm64-dmgk go@51fd0cb3 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (5.16s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:65279->127.0.0.1:65278: read: connection reset by peer

— watchflakes

@gopherbot
Copy link
Contributor Author

Found new dashboard test flakes for:

#!watchflakes
post <- pkg == "net/http" && test == "TestServerGracefulClose" && `connection reset by peer`
2023-09-06 14:19 freebsd-arm64-dmgk go@d25a9355 net/http.TestServerGracefulClose (log) 
--- FAIL: TestServerGracefulClose (0.00s)
    --- FAIL: TestServerGracefulClose/h1 (3.14s)
        serve_test.go:3108: ReadLine: read tcp 127.0.0.1:13595->127.0.0.1:13594: read: connection reset by peer

— watchflakes

@bcmills
Copy link
Contributor

bcmills commented Sep 11, 2023

This seems to be closely related to #51104 / #3595.

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/527196 mentions this issue: net/http: scale rstAvoidanceDelay to reduce test flakiness

@bcmills bcmills self-assigned this Sep 11, 2023
@bcmills bcmills added the Testing An issue that has been verified to require only test changes, not just a test failure. label Sep 11, 2023
@bcmills bcmills modified the milestones: Backlog, Go1.22 Sep 11, 2023
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 11, 2023
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 11, 2023
@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/528398 mentions this issue: net/http: avoid leaking goroutines when TestServerGracefulClose retries

gopherbot pushed a commit that referenced this issue Sep 15, 2023
@gopherbot
net/http: avoid leaking goroutines when TestServerGracefulClose retries
561a507 
If the call to ReadString returns an error, the closure in
testServerGracefulClose will return an error and retry the test with a
longer timeout. If that happens, we need to wait for the conn.Write
goroutine to complete so that we don't leak connections across tests.

Updates #57084.
Fixes #62643.

Change-Id: Ia86c1bbd0a5e5d0aeccf4dfeb994c19d1fb10b00
Reviewed-on: https://go-review.googlesource.com/c/go/+/528398
Auto-Submit: Bryan Mills <[email protected]>
Reviewed-by: Than McIntosh <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Run-TryBot: Bryan Mills <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
@golang golang locked and limited conversation to collaborators Sep 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bcmills bcmills

Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure.
Projects
Test Flakes
Archived in project
Milestone
Go1.22
Development

No branches or pull requests
2 participants
@bcmills @gopherbot