Default NodeJS max heap limit to 250MB. Override by setting NODE_OPTIONS

[mrdavidlaing]

Relates to #5170 - this limits NodeJS to using only 250MB of heap before the garbage collector kicks in - useful in low memory environments.

To override, set the NODE_OPTIONS environment variable, eg, to allow Kibana to use 512MB of heap, set

NODE_OPTIONS="--max-old-space-size=512" bin/kibana

[ghost]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

[ghost]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

[mrdavidlaing]

I've just signed the CLA - not sure how to kick off the check aout

[spalger]

@mrdavidlaing I'm unable to find your CLA signature. Would you mind submitting another?

[spalger]

Can you add this to bin/kibana.bat too please?

[mrdavidlaing]

@spalger - Here is the signed CLA from my company.
Corporate Contributor License Agreement - signed (1).pdf

[spalger]

Ah, business signatures are a little more complex. Thanks for that!

[spalger]

Okay, thinking about this a second time it feels wrong to set the value to anything by default. I'm thinking we should just inject NODE_OPTIONS so that we can use node's (probably intelligent) default but let others pick a value that's better for them.

[rashidkpc]

+1, I think we should keep node's default, and document the fact it exists and how to lower it.

[nabheet]

+1

[nabheet]

Can someone please resolve and merge this pull request? I have been quite anxiously waiting for this fix. It is quite annoying that I have to start my kibana docker container everyday after the OOM Killer kills it.

How can I help?

[simianhacker]

Remove this line, we decided not to set a default.

[w33ble]

Feature implemented in #5598

[mrdavidlaing]

@w33ble - Thanks for following up and merging a similar fix. Sorry for the radio silence.

[Bargs]

This is great, I was just looking for a clean way to run node in debug mode when using our npm scripts and grunt tasks that execute bin/kibana. This fix works great for that use case as well.

[splitice]

If you have to optimize resources then node runs out of memory (FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory) if you limit memory to 250mb.

[ageis]

Did you guys really just ship a broken systemd service in your release?

May 09 17:32:12 logs systemd[1]: Started Elasticsearch.
root@logs:/usr/share/logstash/bin# systemctl status kibana
● kibana.service - Kibana
   Loaded: error (Reason: Bad message)
   Active: inactive (dead) since Tue 2017-05-09 12:11:45 PDT; 5h 23min ago
 Main PID: 3692 (code=killed, signal=TERM)

May 09 12:11:38 logs systemd[1]: [/etc/systemd/system/kibana.service:17] Missing '='.
May 09 12:11:45 logs systemd[1]: Stopping Kibana...
May 09 12:11:45 logs systemd[1]: Stopped Kibana.
May 09 12:11:45 logs systemd[1]: [/etc/systemd/system/kibana.service:17] Missing '='.

[Unit]
Description=Kibana

[Service]
Type=simple
User=kibana
Group=kibana
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/kibana
EnvironmentFile=-/etc/sysconfig/kibana
ExecStart=/usr/share/kibana/bin/kibana "-c /etc/kibana/kibana.yml"
Restart=always
WorkingDirectory=/
Environment=NODE_ENV=production
> # Workaround to prevent OOM errors with kibana/node. Should be fixed in v4.4.
> # Details: https://github.com/elastic/kibana/issues/5170
Environment=NODE_OPTIONS="--max-old-space-size=200"

[Install]
WantedBy=multi-user.target

[ageis]

Also, not only is that an invalid comment, the option doesn't work anyhow....

May 09 17:54:58 logs kibana[26665]: throw err;
May 09 17:54:58 logs kibana[26665]: ^
May 09 17:54:58 logs kibana[26665]: Error: Cannot find module '/"--max-old-space-size=200"'
May 09 17:54:58 logs kibana[26665]: at Function.Module._resolveFilename (module.js:469:15)
May 09 17:54:58 logs kibana[26665]: at Function.Module._load (module.js:417:25)
May 09 17:54:58 logs kibana[26665]: at Module.runMain (module.js:604:10)
May 09 17:54:58 logs kibana[26665]: at run (bootstrap_node.js:393:7)
May 09 17:54:58 logs kibana[26665]: at startup (bootstrap_node.js:150:9)
May 09 17:54:58 logs kibana[26665]: at bootstrap_node.js:508:3

[w33ble]

@ageis could you open a new issue for the problems you're having with the service? It'll help us track a fix.

I'm guessing that's a problem in 5.4.0, but please also note the version you're having problems with in the new issue.

[tylersmalley]

@ageis, our kibana.service file does not include those Environment settings. Are you downloading from our repository?

[ageis]

@tylersmalley Yes. I had a previous installation of Kibana 4 (when it was in /opt instead of /usr/share so there's also a file laying around at /etc/systemd/system/kibana.service.dpkg-old:

[Unit]
Description=Kibana 4

[Service]
Type=simple
User=kibana
Environment=CONFIG_PATH=/opt/kibana/config/kibana.yml
Environment=NODE_ENV=production
# Workaround to prevent OOM errors with kibana/node. Should be fixed in v4.4.
# Details: https://github.com/elastic/kibana/issues/5170
Environment=NODE_OPTIONS="--max-old-space-size=200"
ExecStart=/opt/kibana/node/bin/node /opt/kibana/src/cli

[Install]
WantedBy=multi-user.target

I removed everything related to that installation a long time ago though and had been running cleanly on version 5 for quite a while.