Javascript Heap out of memory when Karma serving files

[CTaylor1118]

Expected behaviour

Single run tests runs consistently and watch mode can recompile consistently

Actual behaviour

Node crashes because of the heap running out of memory when just serving a few dozen files.

Environment Details

OS: Mac OS Sierra 10.12.4
Node: 6.9.5
"karma": "^1.6.0",
"karma-chai": "^0.1.0",
"karma-chrome-launcher": "^2.0.0",
"karma-cli": "^1.0.1",
"karma-mocha": "^1.3.0",
"karma-mocha-reporter": "^2.2.3",
"karma-phantomjs-launcher": "^1.0.4",
"karma-sinon-chai": "git://github.com/kmees/karma-sinon-chai#bca7e7a2d6e90d4c7c6a6884550c2b3228596d23",
"karma-sourcemap-loader": "^0.3.7",
"karma-webpack": "^2.0.3",
"mocha": "^3.2.0",
"webpack": "^1.12.14",
"webpack-dev-server": "^1.14.1"

• Karma version (output of karma --version): 1.6.0
• Relevant part of your karma.config.js file
  karma.config.js

// modify the config for testing
var webpackProdConfig = require('../js/webpack.config.js');

var webpackTestConfig = _.mergeWith({}, webpackProdConfig, {
  // use cheaper source maps
  devtool: '#cheap-module-inline-source-map',
  resolve: {
    alias: {
      // resolve the test directory
      test: __dirname,
      // need to use local jquery
      jquery: jslibpath('jquery-1.9.1/jquery.js'),
    },
    // add the test directory path to the root for resolving
    root: [__dirname]
  }
},
// Use function to make sure arrays are merged by unique values
function (objValue, srcValue) {
  if (_.isArray(objValue)) {
    return _.union(objValue, srcValue);
  }
});
// delete the external jquery
delete webpackTestConfig.externals.jquery;
// delete the entry, karma takes care of this
delete webpackTestConfig.entry;
// delete the output, karma takes care of this
delete webpackTestConfig.output;

module.exports = function(config) {
  config.set({

  // base path that will be used to resolve all patterns (eg. files, exclude)
  basePath: '../..',


  plugins: [
    'karma-mocha',
    'karma-webpack',
    'karma-chrome-launcher',
    'karma-phantomjs-launcher',
    'karma-sinon-chai',
    'karma-mocha-reporter',
    'karma-sourcemap-loader'
  ],

  // frameworks to use
  // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
  frameworks: ['mocha', 'sinon-chai'],


  // list of files / patterns to load in the browser
  files: [
    'htdocs/test/**/*test.js'
  ],

  // list of files to exclude
  exclude: [
  ],


  // preprocess matching files before serving them to the browser
  // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
  preprocessors: {
    'htdocs/test/**/*.js': ['webpack']
  },

  webpack: webpackTestConfig,

  // This is a better webpack config than the one below because it outputs
  // errors.
  webpackMiddleware: {
    progress: true,
    stats: true,
    debug: true,
    noInfo: false,
    silent: false
  },
  
  /*
  webpackMiddleware: {
    progress: false,
    stats: false,
    debug: false,
    noInfo: true,
    silent: true
  },
  */

  client: {
    mocha: {
      // change Karma's debug.html to the mocha web reporter
      reporter: ['html'], 
      // Use 'HTML' when running in chrome
      //reporter: 'HTML', 
      timeout: 30000
    }
  },

  // reporter options
  mochaReporter: {
    showDiff: true
  },

  // test results reporter to use
  // possible values: 'dots', 'progress'
  // available reporters: https://npmjs.org/browse/keyword/karma-reporter
  reporters: ['mocha'],


  // enable / disable colors in the output (reporters and logs)
  colors: true,


  // level of logging
  // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
  logLevel: config.LOG_INFO || config.LOG_DEBUG,


  // enable / disable watching file and executing tests whenever any file changes
  autoWatch: false,


  // start these browsers
  // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
  browsers: ['PhantomJS'],

  // Continuous Integration mode
  // if true, Karma captures browsers, runs the tests and exits
  singleRun: true,
  
  /*
  browsers: ['Chrome'],
  singleRun: false,
  */

  // Concurrency level
  // how many browser should be started simultaneous
  concurrency: Infinity,
  
  // 10 seconds (the default) isn't long enough. Give phantomjs 60 seconds to
  // start up.
  browserNoActivityTimeout: 0,
  });
};

Steps to reproduce the behaviour

This does not happen consistently in single run mode but virtually every time when watching. The problem presents itself after the compilation is complete and karma is serving the files. Happens in bother Phantomjs and Chrome. Judging from the stack trace provided by node, it seems like a buffer on a stream may be getting too large. The heap getting to about 1.259 GB for just a few dozen files. I have seen the same issue with using webpack 2+; I downgraded to see if that would resolve the issue and it did not. The problematic code seems to be in the karma middleware.

[wesleycho]

I have seen this as well, but when using the watcher and Chrome. It was not consistently reproducible for me though (Node 6.9.4).

Could this be related to karma-webpack? My team uses karma-webpack, but we don't use the webpackMiddleware option.

[CTaylor1118]

The webpack middleware settings just turn on some reporting. I am almost positive this has to do with karma as running webpack-dev-server with similar settings does not result in the same crash in node. It seems to me the karma has a memory leak somewhere. When analyzing the memory usage for karma, I did notice that out of about 1 .4 GB of heap space used, almost 1 GB was being used to store strings. It seems like most of the strings are not actually being referenced by a file Karma is serving, and I am sure that some of them are needed, the larger ones just seem to extraneous copies of other strings. I am not the greatest as debugging memory issues but It seems like only one of these should exist at a time since the are the same file. The other issue I am seeing is the string size being stored is much larger than the actual size of the file. In the screenshot attached, the file is emitted by webpack. Together with its source map, it is 9.79 MB but karma is using about 50 MB to store the string representation of the file, which seems to indicate that it is somehow adding extra data to the strings.

[CTaylor1118]

I tracked down the core issue to a webpack loader (jsx-loader) some how causing a memory leak. However I think there are optimization that can be made to karma to reducing its memory usage, as in the above post, having to different file object constructed for the same file seems like a waste.

[maksimr]

@CTaylor1118 thanks for your investigation 👍

About files from the picture, This is strange that files don't have a path/originalPath. Maybe this files was created by webpack-plugin

[CTaylor1118]

@maksimr Sorry about that, they actually have a complete filepath to a real, I just commented it out for work purposes.

[dnsco]

Has anyone figured out any ways to mitigate this issue?

[varya]

I'm having the same issue (without using jsx-loader). Any plans to fix?

[bneumann]

I tried narrowing down this on my PC. It only occurs with node x86. 100% reproducable with version 6.13.1.
Unfortunatley when trying to take a heap snapshot my PC got stuck at 19% and my RAM litterally exploded so there is some heavy memory leak in karma-webpack. We are using the ts loader and have a lot of test files.
Anyways setting the old_ram_size doesn't help (even when putting in ludicrous amounts like 8GB).

I will try to take a snapshot but I think my machine is too weak for this leak (woooord ;) )

[briandipalma]

Who is meant to clean up following a test run (with singleRun option). The Plugin itself or Karma? I mean if Karma doesn't clean up the Socket.io connection properly the Injector leaks as it's part of the Karma Server class and that means the entire Webpack compilation leaks but to my mind that's not Webpack's fault.

[bneumann]

How could a socket.io connection accumulate so much memory unless you constantly open new connections which shouldn't be possible?

[briandipalma]

I have this PR #2981 but I think this issue is different as you all seem to be using one instance of Karma while I spawn many.

[vprokashev]

Hi guys.
This is strange, but it was exactly such an error that began to appear when I added the line to webpack
config.devtool = 'cheap-inline-module-source-map';

[johnjbarton]

I'm unsure if this is related, but we recently began preprocessing our karma sources to move inline sourcemaps into .map files before bundling and serving. This dropped the number of bytes shipped to the browser and allowed tests that failed on memory to pass.

[blacelle]

We fixed this issue by following what's suggested at: https://github.com/webpack-contrib/karma-webpack#alternative-usage

A config like:

  // list of files / patterns to load in the browser
  files: [
    'htdocs/test/**/*test.js'
  ],

leads to webpack bundling the whole app for EACH .test.js file. The alternative-usage will bundle the app only once -> in our cases, instaed of having dozens of 20MB files (i.e. strings maintained in memory), we ended with a single 50MB file.