Fix unstable vz driver network issue #1543
Conversation
|
Lets hold the merge. Unfortunately this doesn't fix it :( I tried a long running instance in M1 and the freeze still happened |
Signed-off-by: Balaji Vijayakumar <[email protected]>
|
@AkihiroSuda In this we are calling setReadDeadline before every read. This will make sure if the UDP connection is not closed. Before trying to read. Reading on closed UDP connection will end up getting stuck in that read We can merge these current set of changes if it looks good mainly to get more info when error happens again |
|
I am testing in my M1 from yesterday, with both long running VM and as well as load testing network with multiple torrent download. Unfortunately freeze didn't till now. |
|
Thanks
s/Unfortunately/Fortunately/ ? 🙂 |
I was expecting it to freeze so unfortunate :D |
| // Check if the connection has been closed | ||
| if err := conn.SetReadDeadline(time.Time{}); err != nil { | ||
| if opErr, ok := err.(*net.OpError); ok && opErr.Err.Error() == "use of closed network connection" { | ||
| return 0, errors.New("UDPFileConn connection closed") |
There was a problem hiding this comment.
Can we wrap the original err?
| return 0, errors.New("UDPFileConn connection closed") | |
| return 0, fmt.Errorf"the UDPFileConn connection closed: %w", err) |
There was a problem hiding this comment.
We could but since we are already checking if error is equal to use of closed network connection
I thought we won't be getting anything new out of error object
This should hopefully fix the network freeze when using vz driver
#1200 (comment)
I didn't add fix issue id's mainly to try and see if this really fixes or not.
What is happening, why this change ?
When we use file.FD(), whenever the file gets closed or garbage collected (finaliser closing file) the FD give before becomes invalid.
In our case, we use file.FD() for both client and server connections. Since these are DGRAM connection, if we try to read on a closed FD we won't get any error instead it get stuck indefinitely. This could be the reason for the freeze of vz network
Before this change, i verified that that file getting garbage collected in ~2mins. So logically FD would become invalid at some point.