You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I found this issue while testing how our app handles a network interruption and noticed this behavior with using node sockets in Linux. I have verified this issue happens with both 0.10.41 and 4.2.3.
In our test we need to wait for an ETIMEDOUT which can take 15 minutes with the default setting. To make the test bearable run sysctl -w net.ipv4.tcp_retries2=8 in the test container to make this around 100s instead.
Dockerfile
FROM node:0.10.41
RUN apt-get update && apt-get install -y iptables net-tools
RUN npm install -g ecco
CMD ["bash"]
Socket write code
#!/usr/bin/env node
"use strict"varnet=require('net');varMESSAGE_RATE=1000;varRANDOM_DATA=700;varhost='172.17.0.4';varport='9999';vardebug=require('debug')('SocketTest');varsocket=net.createConnection(port,host);socket.addr=host+':'+port;socket.host=host;socket.port=port;socket.on('connect',function(){debug(this.address())varlastError=this.error;this.error=null;if(lastError){debug('reconnect');}else{startLoop()}});socket.on('error',function(err){this.error=err;debug('error',err);});socket.on('close',function(had_error){debug('close');retry(this);});socket.on('end',function(){debug('Socket End')retry(this);});socket.buffer=newBuffer([]);socket.on('data',function(data){debug('received data',data)});socket.on('drain',function(){debug('socket drained');});socket.setKeepAlive(true,20000);functionretry(s){if(s.retrying||s.closing)return;s.retrying=true;s.retryTimer=setTimeout(function(){s.retrying=false;s.connect(s.port,s.host);},1000);}varmessageNumber=0varlastTimer=nullfunctionstartLoop(){if(!socket.error){varcurrentNumber=messageNumberdebug('Sending message #'+messageNumber)varret=socket.write(newArray(RANDOM_DATA).join(Math.random().toString(36))+' Received message # '+messageNumber++,sendCb)debug('Socket write %s',ret ? 'flushed to kernel': 'queued in memory');debug('Socket %d bytes written',socket.bytesWritten);debug('Socket %s bytes read',socket.bytesRead);debug('Socket bufferSize %d',socket.bufferSize);}else{debug('socket error not writing.')}lastTimer=setTimeout(startLoop,MESSAGE_RATE);functionsendCb(){debug('Callback called for #%d',currentNumber);currentNumber=null}}
Make note of the IP address for that container using awk 'NR==1 {print $1}' /etc/hosts
Update the IP in the test code (in my case it's 172.17.0.4) and run the test app
Observe the data streaming in the echo container's terminal
In container test: don't remove empty.txt on win32 #2 add the firewall rule to block the echo server's IP iptables -A INPUT -s "172.17.0.4" -j DROP;iptables -A OUTPUT -s "172.17.0.4" -j DROP;
Notice the data stops streaming in the echo terminal. This is expected behavior.
The socket appears to write out data but no data appears to the echo terminal. This is unexpected.
Apply the block rule again and do netstat again. We expect the Send-Q of the new connection to accumulate but it does not. This is also unexpected.
(don't forget to remove the rule after testing)
Observations
This issue doesn't happen on a Mac. I don't have an windows enviorment so I have not tested there.
This issue also does not happen when you send small amounts of data to socket.write. if you change the RANDOM_DATA to a lower number like 70 it will recover.
In 0.10.41 I noticed process._errno contains an ECANCELED error and doesn't look like socket code errors out the socket when it receives this error string from the handle.
The text was updated successfully, but these errors were encountered:
I don't suppose any Docker enthusiasts in @nodejs/docker might be able to see if this is still happening in Node.js 8 and/or determine if there's some non-Node.js cause to the issue as described?
It has been over two years, no one has been able to investigate and no further information is available. While I don't doubt this is a valid report, it seems unlikely that anyone will take it up now. Also it's impossible to know whether this issue persists into more current Node.js versions (since 4.x is EOL very soon) or newer Docker versions.
That said, if you feel I've made an error in closing this, feel free to re-open.
I found this issue while testing how our app handles a network interruption and noticed this behavior with using node sockets in Linux. I have verified this issue happens with both
0.10.41
and4.2.3
.Reproduce Setup
ecco
iptables
andnetstat
to run the test.--net=host --privileged
ETIMEDOUT
which can take 15 minutes with the default setting. To make the test bearable runsysctl -w net.ipv4.tcp_retries2=8
in the test container to make this around 100s instead.Dockerfile
Socket write code
Reproduce Steps
ecco -l -v -p 9999 --address 0.0.0.0
awk 'NR==1 {print $1}' /etc/hosts
172.17.0.4
) and run the test appiptables -A INPUT -s "172.17.0.4" -j DROP;iptables -A OUTPUT -s "172.17.0.4" -j DROP;
netstat -napc
in container test: don't remove empty.txt on win32 #2 and observe the socket'sSend-Q
increasing over time.ETIMEDOUT
occurs and then remove the rule.iptables -D INPUT -s "172.17.0.4" -j DROP;iptables -D OUTPUT -s "172.17.0.4" -j DROP;
iptables -nvL
client connected
after container test: don't remove empty.txt on win32 #2 reconnects.Send-Q
of the new connection to accumulate but it does not. This is also unexpected.(don't forget to remove the rule after testing)
Observations
RANDOM_DATA
to a lower number like 70 it will recover.0.10.41
I noticedprocess._errno
contains anECANCELED
error and doesn't look like socket code errors out the socket when it receives this error string from the handle.The text was updated successfully, but these errors were encountered: