Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: write after end #296

Closed
chattes opened this issue Sep 21, 2016 · 5 comments
Closed

Error: write after end #296

chattes opened this issue Sep 21, 2016 · 5 comments

Comments

@chattes
Copy link

chattes commented Sep 21, 2016

Hi,
we are using RethinkDBdash as our rethinkDB driver along with Nodejs. Its working great. But i started a bit of stress testing on my dev machine, and when I send 500+ concurrent requsts per second, the server is crashing with the following error:

events.js:160
throw er; // Unhandled 'error' event
^

Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:651:40)
    at Connection._sendProof (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:281:19)
    at Connection._computeSaltedPassword (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:247:10)
    at Socket.<anonymous> (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:184:18)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:543:20)

We are using Node 6.5.0 and "rethinkdbdash": "^2.3.23"
Request you to kindly help with the same.

Thanks

@neumino
Copy link
Owner

neumino commented Sep 21, 2016

My guess is that you're hitting your max open files limit and can't properly open connections after that. Looking at the stacktrace, it's probably a bug introduced with the new handshake, 2.2.x should work for now.

It should be relatively straightforward to fix.

@chattes
Copy link
Author

chattes commented Sep 22, 2016

Okay let me try out with 2.2.x. I will keep you posted. although i increased my max pool to 10000 after your suggestion and it disappeared for a while. But it has come back again today.

Thanks

@chattes
Copy link
Author

chattes commented Sep 22, 2016

Hi quick question this error 'Fail to create a new connection for the connection pool. Error:{"message":"Failed to connect to localhost:28015 in less than 20s.","isOperational":true}'
Will it try and create connections once sockets are available again?

@chrisabrams
Copy link

@neumino any updates on this? I'm getting the exact same error. I only started getting this once the number of connections per second increased. The connection is made, and then after a few seconds it is lost, and no new connections can seem to be made.

Fail to create a new connection for the connection pool. Error:{"msg":"Failed to connect to cluster.local:28015 in less than 5s","message":"Failed to connect to cluster.local:28015 in less than 5s.","isOperational":true}

Here is my driver file:

driver.js

const driverConfig = {
  authKey: process.env.RETHINK_DB_AUTH_KEY || '',
  buffer:  process.env.NODE_ENV === 'production' ? 50 : 3,
  db:      process.env.RETHINK_DB_NAME,
  host:    process.env.RETHINK_DB_HOST || 'localhost',
  min:     process.env.NODE_ENV === 'production' ? 50 : 3,
  pool:    true,
  port:    process.env.RETHINK_DB_PORT || 28015,
  timeout: process.env.RETHINK_DB_DRIVER_TIMEOUT || 5
}
const r = require('rethinkdbdash')(driverConfig)
module.exports = r

@chrisabrams
Copy link

I changed my driver config to:

const driverConfig = {
  authKey: process.env.RETHINK_DB_AUTH_KEY || '',
  buffer:  process.env.RETHINK_DB_DRIVER_BUFFER || 50,
  db:      process.env.RETHINK_DB_NAME,
  host:    process.env.RETHINK_DB_HOST || 'localhost',
  pool:    true,
  port:    process.env.RETHINK_DB_PORT || 28015,
  timeout: process.env.RETHINK_DB_DRIVER_TIMEOUT || 20,
  timeoutGb: process.env.RETHINK_DB_DRIVER_TIMEOUTGB || 60 * 60 * 1000
}

And the issue has gone away for the moment.

@neumino neumino closed this as completed in ec62697 Nov 6, 2016
thejsj added a commit to thejsj/rethinkdbdash that referenced this issue Dec 17, 2016
* 'master' of github.com:neumino/rethinkdbdash: (304 commits)
  Fix leaked variable from test
  Bump to 2.3.27
  Fix neumino#313
  Bump to 2.3.26
  fix: return null where new Promises are created but not returned
  Fix leak in tests
  Fix leak in tests
  Bump to 2.3.25
  Make identification errors operational such that they can be properly caught - Fix neumino#295
  Bump to 2.3.24
  Close the stream if an error is emitted since we can't recover frm it
  Remove the use of Buffer with new for node 7 - Fix neumino#304
  Wrap sending the proof in a try/catch in case the connection is closed - Fix neumino#296
  Bump to 2.3.23
  Prevent bluebird warning - Fix neumino#287
  Bump to 2.3.22
  Prevent bluebird warning - Fix neumino#285
  function r: remove unused parameter `options`
  readme: minor typo fix
  Bump to 2.3.21
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants