High CPU usage #35

flynfish · 2013-01-09T19:04:14Z

When running grunt watch, the process sits around 70% cpu usage on my mac (10.8.2). Is this a known issue?

shama · 2013-01-09T19:10:37Z

It's not. How many files are you watching and could you post your Gruntfile.js please? Does it happen with just watching or as tasks are ran? Also are you using [email protected] and [email protected]?

flynfish · 2013-01-09T19:26:09Z

Versions:
"grunt": "~0.3.17",
"grunt-contrib-watch": "~0.1.4",
"grunt-contrib": "~0.3.0"

As for the amount of files, since I am using bootstrap there are more 40 less files being watched. There really isn't much of a need for me to watch those though so perhaps that is one of the issues? Looks like it happens just when watching.

module.exports = function(grunt) {

  grunt.initConfig({

    // The lint task will run the build configuration and the application
    // JavaScript through JSHint and report any errors.  You can change the
    // options for this task, by reading this:
    // https://github.com/cowboy/grunt/blob/master/docs/task_lint.md
    lint: {
      files: [
        "public/**/*.js"
      ]
    },

    // The jst task compiles all application templates into JavaScript
    // functions with the underscore.js template function from 1.2.4.  You can
    // change the namespace and the template options, by reading this:
    // https://github.com/tbranyen/build-tasks/tree/master/jst
    //
    // The concat task depends on this file to exist, so if you decide to
    // remove this, ensure concat is updated accordingly.
    jst: {
      "public/js/templates.js": [
        "views/templates/**/*.html"
      ]
    },

    // The concatenate task is used here to merge the almond require/define
    // shim and the templates into the application code.  It's named
    // dist/debug/require.js, because we want to only load one script file in
    // index.html.
    concat: {
      "public/js/debug.js": [
        "public/js/vendor/*.js",
        "public/js/vendor/bootstrap/*.js",
        "public/js/templates.js",
        "public/js/app/*.js",
        "public/js/app/models/*.js",
        "public/js/app/views/*.js",
        "public/js/app/collections/*.js",
        "public/js/app/views/nests/*.js"
      ]
    },

    // This task uses the MinCSS Node.js project to take all your CSS files in
    // order and concatenate them into a single CSS file named index.css.  It
    // also minifies all the CSS as well.  This is named index.css, because we
    // only want to load one stylesheet in index.html.
    mincss: {
      "public/dist/release.css": [
        "public/stylesheets/*.css"
      ]
    },

    // Takes the built require.js file and minifies it for filesize benefits.
    min: {
      "public/dist/release.js": [
        "public/js/debug.js"
      ]
    },

    // less to CSS
    less: {
      compile: {
        files: {
          "public/stylesheets/style.css": "public/stylesheets/style.less",
          "public/stylesheets/nest.css": "public/stylesheets/nest.less",
          "public/stylesheets/style.css": "public/stylesheets/style.less",
          "public/stylesheets/vendor/bootstrap/bootstrap.css": "public/stylesheets/vendor/bootstrap/less/bootstrap.less",
          "public/stylesheets/vendor/fontawesome/font-awesome.css": "public/stylesheets/vendor/fontawesome/less/font-awesome.less"
        }
      }
    },

    watch: {
      scripts: {
        files: '**/*.less',
        tasks: ['less']
      },
      templates: {
        files: 'views/templates/**/*.html',
        tasks: ['jst']
      }
    }

  });

  // The default task will remove all contents inside the dist/ folder, lint
  // all your code, precompile all the underscore templates into
  // dist/debug/templates.js, compile all the application code into
  // dist/debug/require.js, and then concatenate the require/define shim
  // almond.js and dist/debug/templates.js into the require.js file.
  grunt.registerTask("default", "concat min less mincss jst");
  grunt.loadNpmTasks('grunt-contrib');
};

shama · 2013-01-09T19:34:05Z

Thanks for the info. If you get a chance, could you try the latest grunt and grunt-contrib-watch? Let me know if it still runs up the CPU. We've made a lot of changes since v0.1.4 in the watch. I'll try your setup shortly to see if I can duplicate. Thanks!

flynfish · 2013-01-09T20:41:46Z

I updated to the lastest grunt and grunt-contrib versions and now the process never gets above 1% cpu, so I would say you guys did some awesome work on that update!

When running watch now and updating a less file, I get this error:

running "less:compile" (less) task
Warning: Object #<Object> has no method 'expandFiles' Use --force to continue.

Aborted due to warnings.

Any ideas on what I need to change to get it to work? (I already update the registerTask default alias to use an array instead of string"

shama · 2013-01-09T20:52:41Z

Awesome, glad to hear!

To get less working, you'll need to install the Grunt v0.4 version with: npm install git://github.com/gruntjs/grunt-contrib-less.git. In fact all of your plugins will need updating if moving to Grunt v0.4 (https://github.com/gruntjs/grunt/wiki/Upgrading-from-0.3-to-0.4). FWIW, I use Grunt v0.4 exclusively. Just know it is still under development so not all grunt plugins support it yet and things can change.

Another thing to try if staying on Grunt v0.3 for the time being. Try uninstalling grunt-contrib-watch and using the previously built-in watch task in Grunt v0.3. grunt-contrib-watch is intended as a replacement for the watch task in Grunt v0.4. We just had Grunt v0.3 support on the v0.1.x branch to get some early testers. :)

flynfish · 2013-01-09T22:37:02Z

I tried upgrading to grunt .4 and upgrading the grunt plugins but I ran into some issues I don't have time to figure out. I removed the bootstrap less files from being watched and that dropped the cpu to under 5%.

shama · 2013-01-28T17:14:27Z

Hopefully the other issues you got figured out :) Otherwise please reopen if one of those is with the watch task, thanks!

alexbeletsky · 2013-03-23T19:02:08Z

I'm having the same issue on my Windows 8, cpu level near 70%, watching just a few files.

Jesterovskiy · 2013-05-22T16:47:30Z

Have issue with high load too.

Versions:
"grunt": "~0.4.1",
"grunt-contrib-watch": "~0.4.3",
"grunt-contrib-sass": "~0.3.0"

System: Ubuntu 12.04 LTS

Gruntfile:

...
   watch: {
      sass: {
        files: ['app/stylesheets/{,**/}*.scss'],
        tasks: ['sass:dev']
      },
      html: {
        files: ['app/index.html'],
        tasks: ['html:dev']
      }
    },

  html: {
      dev: {
        options: {
          src: 'app/index.html',
          dest: 'temp/index.html',
          vars: config('development')
        }
      }
    },

    sass: {
      dev: {
        options: {
          style: 'expanded'
        },
        files: {
          'temp/stylesheets/application.css': 'app/stylesheets/application.scss'
        }
      }
    },
...

shama · 2013-05-22T16:58:30Z

Try running with --verbose to see which files it is watching. Also which version of node?

Jesterovskiy · 2013-05-22T17:35:35Z

Task is watching 45 scss files and version of node is v0.10.4.

shama · 2013-05-22T18:02:36Z

I'm curious if the problem is within this lib or just watching in general (as that greatly differs on environments). Are you game to try some examples for me?

Could you first make sure any previous node processes are not running with killall -9 node and then run this example to see if that causes the CPU spike?

var fs = require('fs');
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
].forEach(function(file) {
  fs.watch(file, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});

Jesterovskiy · 2013-05-23T12:51:07Z

Not fully understand, how to run this example. Can you explain, please =)

shama · 2013-05-23T16:05:56Z

Oh that example will watch files using the core node.js api because I want to see if the problem is from this watch task or with any file watching.

Throw that example into a file named test.js. Add a bunch more actual file paths and then do this in your terminal: killall -9 node && node test.js to kill any other running node process and then run the example. Thanks!

Jesterovskiy · 2013-05-23T17:00:21Z

Thank you for explain) I can't find other process of node on machine and have this error on start:

vagrant@precise32:/vagrant$ killall -9 node                
node: no process found
vagrant@precise32:/vagrant$ node test.js

fs.js:1051
    throw errnoException(process._errno, 'watch');
          ^
Error: watch ENOENT
    at errnoException (fs.js:1019:11)
    at FSWatcher.start (fs.js:1051:11)
    at Object.fs.watch (fs.js:1076:11)
    at /vagrant/test.js:31:6
    at Array.forEach (native)
    at Object.<anonymous> (/vagrant/test.js:30:3)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

shama · 2013-05-23T17:04:40Z

Thanks for giving it go. That error means one of the files you've tried doesn't exist. Could you try on only files that exists?

Jesterovskiy · 2013-05-24T08:17:16Z

Thanks, you're right) It's my bad knowledge of english =(
This is results:
configuration

watch: {
      sass: {
        files: ['app/stylesheets/application.scss',
                'app/stylesheets/app/{,**/}*.scss',
                'app/stylesheets/lib/{,**/}*.scss'
               ],
        tasks: ['sass:dev']
      },
sass: {
      dev: {
        options: {
          style: 'expanded'
        },
        files: {
          'temp/stylesheets/application.css': 'app/stylesheets/application.scss'
        }
      },

screenshot of htop when run watch:sass

configuration

var fs = require('fs');
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
  'app/stylesheets/app/_authentication.scss',
  'app/stylesheets/app/_dynamic-content.scss',
  'app/stylesheets/app/_main.scss',
  'app/stylesheets/app/_news-feed.scss',
  'app/stylesheets/app/crime-reporting/_incident-show.scss',
  'app/stylesheets/app/crime-reporting/_incidents.scss',
  'app/stylesheets/app/crime-reporting/_print-content.scss',
  'app/stylesheets/app/crime-reporting/_subjects.scss',
  'app/stylesheets/app/crime-reporting/_video-edit.scss',
  'app/stylesheets/app/features/_comments.scss',
  'app/stylesheets/app/features/_entities.scss',
  'app/stylesheets/app/features/_groups.scss',
  'app/stylesheets/app/features/_messages.scss',
  'app/stylesheets/app/features/_premises.scss',
  'app/stylesheets/app/features/_statistics.scss',
  'app/stylesheets/app/features/_users.scss',
  'app/stylesheets/app/headers/_action-bar.scss',
  'app/stylesheets/app/headers/_header.scss',
  'app/stylesheets/app/plugins/_custom-redactor.scss',
  'app/stylesheets/app/plugins/_on-off-switch.scss',
  'app/stylesheets/app/sidebars/_left.scss',
  'app/stylesheets/app/sidebars/_right.scss',
  'app/stylesheets/app/ui/_flashes.scss',
  'app/stylesheets/app/ui/_forms.scss',
  'app/stylesheets/app/ui/_tables.scss',
  'app/stylesheets/lib/_lt-ie7.scss',
  'app/stylesheets/lib/_lt-ie8.scss',
  'app/stylesheets/lib/_lt-ie9.scss',
  'app/stylesheets/lib/_mixins.scss',
  'app/stylesheets/lib/bootstrap/_custom-icons.scss',
  'app/stylesheets/lib/bootstrap/_ie7buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_ie8buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_mixins-custom.scss',
  'app/stylesheets/lib/bootstrap/_modal.scss',
  'app/stylesheets/lib/bootstrap/_overrides.scss',
  'app/stylesheets/lib/bootstrap/_popover.scss'
].forEach(function(file) {
  fs.watch(file, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});

screenshot of htop when run node test.js

shama · 2013-05-24T16:26:30Z

Hmm bummer. I'll reopen although I'm not sure how to fix as I can't reproduce and I am out of ideas. I really appreciate you trying those examples for me @Jesterovskiy! If someone has this issue and discovers a fix please let us know! Thanks.

alexbain · 2013-05-24T22:01:38Z

Just chiming in to say that we're experiencing the same issue watching around 100 files. We ran the same test above and didn't see any serious CPU usage, so no idea what's causing it.

Jesterovskiy · 2013-05-27T08:05:53Z

Some test with https://github.com/shama/gaze shows same situation: 5 process and quiescent load 20-23% of cpu. I will dig deeper =)

Jesterovskiy · 2013-05-27T08:43:05Z

I found the problem! =) In Gaze by default interval sets https://github.com/shama/gaze/blob/master/lib/gaze.js#L36 but in fs lib we see different default value https://github.com/joyent/node/blob/master/lib/fs.js#L1128 I changed this value in my test file and the result - no more high cpu load =)
configuration

var fs = require('fs'),
    opts = { persistent: true, interval: 5007 };
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
  'app/stylesheets/app/_authentication.scss',
  'app/stylesheets/app/_dynamic-content.scss',
  'app/stylesheets/app/_main.scss',
  'app/stylesheets/app/_news-feed.scss',
  'app/stylesheets/app/crime-reporting/_incident-show.scss',
  'app/stylesheets/app/crime-reporting/_incidents.scss',
  'app/stylesheets/app/crime-reporting/_print-content.scss',
  'app/stylesheets/app/crime-reporting/_subjects.scss',
  'app/stylesheets/app/crime-reporting/_video-edit.scss',
  'app/stylesheets/app/features/_comments.scss',
  'app/stylesheets/app/features/_entities.scss',
  'app/stylesheets/app/features/_groups.scss',
  'app/stylesheets/app/features/_messages.scss',
  'app/stylesheets/app/features/_premises.scss',
  'app/stylesheets/app/features/_statistics.scss',
  'app/stylesheets/app/features/_users.scss',
  'app/stylesheets/app/headers/_action-bar.scss',
  'app/stylesheets/app/headers/_header.scss',
  'app/stylesheets/app/plugins/_custom-redactor.scss',
  'app/stylesheets/app/plugins/_on-off-switch.scss',
  'app/stylesheets/app/sidebars/_left.scss',
  'app/stylesheets/app/sidebars/_right.scss',
  'app/stylesheets/app/ui/_flashes.scss',
  'app/stylesheets/app/ui/_forms.scss',
  'app/stylesheets/app/ui/_tables.scss',
  'app/stylesheets/lib/_lt-ie7.scss',
  'app/stylesheets/lib/_lt-ie8.scss',
  'app/stylesheets/lib/_lt-ie9.scss',
  'app/stylesheets/lib/_mixins.scss',
  'app/stylesheets/lib/bootstrap/_custom-icons.scss',
  'app/stylesheets/lib/bootstrap/_ie7buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_ie8buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_mixins-custom.scss',
  'app/stylesheets/lib/bootstrap/_modal.scss',
  'app/stylesheets/lib/bootstrap/_overrides.scss',
  'app/stylesheets/lib/bootstrap/_popover.scss'
].forEach(function(file) {
  fs.watchFile(file, opts, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});

shama · 2013-05-27T17:47:38Z

@Jesterovskiy Awesome find! Thanks for looking into this! That makes a lot of sense. I'll have to increase the default interval for polling in gaze. In the meantime that setting is available through the options: options: { interval: 5007 }. Thanks! :D

jonvuri · 2014-04-21T23:24:09Z

I see that this has been closed but I still see grunt using about 9% (of a 2.4 i5) on a watch of a few dozen files on OS X. Doesn't seem like a lot but if I leave the watch running all day it keeps power management from kicking in and costs me battery life. I'll play around with that polling interval option, I just wanted to mention it here so that you know the defaults aren't ideal.

shama · 2014-04-22T03:10:48Z

FWIW, the next version of gaze uses native OS events for everything and will dramatically reduce the CPU usage. In the meantime, just make sure it's not matching unintended files grunt watch -v, set a higher interval.

showrav017 · 2014-12-10T11:08:52Z

Hi,

I know its a very old issue but i face this problem again on My windows 8.1 PC.

Its regarding the Sails.js mvc. I have tried all the above suggestions but these were not working for me. I have to commented out the codes under Create watcher per target line from the file tasks/watch.js of this module.

I just stop the watchers to pushing items on it under a infinite type loop. I just did it so don't know what happened further.

Hope it will help others temporarily.

Thanks

joeyespo · 2014-12-14T05:26:46Z

Also seeing a CPU usage on a Windows 7 machine. And high memory usage (> 100MB).

tigrr · 2014-12-20T15:15:37Z

Had the same problem.
It was caused by Grunt watching "node_modules" directory.
https://github.com/gruntjs/grunt-contrib-watch#why-is-the-watch-devouring-all-my-memorycpu

timomayer · 2015-02-06T14:40:25Z

i have the same issue!

pwdonald · 2015-02-16T16:15:13Z

I am also experiencing this issue. It starts out running at around 10-20%, then after about five minutes it shoots up to 99% CPU and stays there until I stop the watch task. I was unaware this was happening on my desktop but when I run it on my laptop it gets very warm and the fan spins constantly, which is what alerted me to the issue.

Edit: Just noticed the comment about watching the node_modules folder, that was my problem as well. Once I specified the watch directories to exclude that it fixed the issue.

timomayer · 2015-02-16T16:42:57Z

@pwdonald exactly same for me. but i have node_modules excluded already...

jifeon · 2015-02-25T13:47:58Z

Have the same problem on Yosemite on mac mini, 50% CPU for couple hundreds of scripts. I use the same configuration on fedora and grunt uses less than 1%

jifeon · 2015-02-25T15:42:16Z

I've found out that updating gaze to 0.6.4 solves my issue. PR about it: #412

jmorvan · 2015-06-16T15:29:35Z

@tikosar spot on! I was watching node_modules! tku

doivosevic · 2015-09-16T22:52:19Z

I've had the issue of watching node_modules/ as well. I've added

   '!node_modules/**/*.*'

like:

    watch: {
        options: {
            livereload: true
        },
        express: {
            files: ['**/*.js', '!node_modules/**/*.*'],

Should you maybe disable watching node_modules/ by default since I don't see why anyone would do that anyways and they could always enable it manually?

yang · 2015-09-25T06:20:17Z

I'm seeing the same disparity in CPU usage between Linux (lower/more manageable) and Mac (high). I'm using 0.6.1.

WayneKeenan · 2016-03-08T16:24:24Z

obviously ymmv, but just thought I mention another potential biggie that caused slowdown on my Mac which was solved by ignoring the bower_components folder, e.g. !app/bower_components/**/*.*

before : ~3100 files using ~%60 CPU usage
after: ~10 files using ~5% CPU usage

versions:

grunt-contrib-watch v0.6.1
grunt-cli v0.1.13
grunt v0.4.5
node v5.5.0

I would have thought watching 3100 files would not be such a problem giving the Mac's FSEvent API is supposed to be fast and light weight.

But, I've just noticed that the grunt-contrib-watch package.json doesn't explicitly mention the node module https://www.npmjs.com/package/fsevents as a dependancy and uses gaze which at first glance also doest use node's fsevent. (this was just a cursory check, but given the behaviour it would seem it doesn't)

I guess a potential fix (for very large projects) is to use fsevent on Mac (either directly in grunt-watch or by patching node's gaze)

rbosneag · 2016-03-10T10:49:07Z

My special dumb case:

Took me 3h (of looking in watch source and gaze and resetting Gruntfile.js to a minimum and update packages one by one and changing versions, etc) to realize that my task was configured as follows:

grunt.registerTask('watch', ['watch']);

Guess what? Recursion! CPU went in 100% instantly, no info in terminal, verbose mode showed watch task running continuously.

That was stupid but made me laugh in the end.

I don't think my solution will help someone but I posted it just to cover all scenarios...

saltcod · 2017-06-30T17:39:40Z

Someone had composer'd a /vendor/ dir in my theme. Ignoring that with '!/vendor/' did the trick for me. Thanks!

shama closed this as completed Jan 28, 2013

shama reopened this May 24, 2013

shama closed this as completed May 27, 2013

shama mentioned this issue Jul 5, 2013

node cpu when watch #145

Closed

rwjblue mentioned this issue Aug 20, 2013

Experiencing extreme Finder hang when rebuild happens stefanpenner/ember-app-kit#75

Closed

mderk mentioned this issue Jan 21, 2014

Brunch watch mode uses too much CPU brunch/brunch#783

Closed

tomusdrw mentioned this issue May 12, 2014

Huge CPU usage tomusdrw/grunt-sync#10

Closed

kriswill mentioned this issue Dec 6, 2014

Docker shared volume causes Grunt to consume increasing CPU over time sane/sane#20

Closed

Stratus3D mentioned this issue Jul 23, 2015

High CPU usage #429

Open

High CPU usage #35

High CPU usage #35

Comments

flynfish commented Jan 9, 2013

shama commented Jan 9, 2013

flynfish commented Jan 9, 2013

shama commented Jan 9, 2013

flynfish commented Jan 9, 2013

shama commented Jan 9, 2013

flynfish commented Jan 9, 2013

shama commented Jan 28, 2013

alexbeletsky commented Mar 23, 2013

Jesterovskiy commented May 22, 2013

shama commented May 22, 2013

Jesterovskiy commented May 22, 2013

shama commented May 22, 2013

Jesterovskiy commented May 23, 2013

shama commented May 23, 2013

Jesterovskiy commented May 23, 2013

shama commented May 23, 2013

Jesterovskiy commented May 24, 2013

shama commented May 24, 2013

alexbain commented May 24, 2013

Jesterovskiy commented May 27, 2013

Jesterovskiy commented May 27, 2013

shama commented May 27, 2013

jonvuri commented Apr 21, 2014

shama commented Apr 22, 2014

showrav017 commented Dec 10, 2014

joeyespo commented Dec 14, 2014

tigrr commented Dec 20, 2014

timomayer commented Feb 6, 2015

pwdonald commented Feb 16, 2015

timomayer commented Feb 16, 2015

jifeon commented Feb 25, 2015

jifeon commented Feb 25, 2015

jmorvan commented Jun 16, 2015

doivosevic commented Sep 16, 2015

yang commented Sep 25, 2015

WayneKeenan commented Mar 8, 2016

rbosneag commented Mar 10, 2016

saltcod commented Jun 30, 2017