Update single run callback

[rinatio]

Hello guys,

I faced an issue described here and here using example from readme:

[17:34:54] Error: 1
    at formatError (/usr/local/lib/node_modules/gulp/bin/gulp.js:169:10)
    at Gulp.<anonymous> (/usr/local/lib/node_modules/gulp/bin/gulp.js:195:15)
    at Gulp.emit (events.js:95:17)

Is it possible to update example like this:

gulp.task('test', function(done) {
    karma.start({
        configFile: __dirname + '/karma.conf.js',
        singleRun: true
    }, function() {
        done();
    });
});

This way gulp exists properly

[pkozlowski-opensource]

I don't think this is correct. Doing what you suggest would mean that the error code from Karma (ex. failing test) wouldn't be propagated to Gulp and consequently - not propagated to the OS. So by doing this you would have gulp process exit with code 0 for failing unit tests. This is not good since it would prevent us from signalling test failures to the OS and this would make integrating with CI servers pretty difficult.

I'm going to close this issue as I believe that the proper solution is to improve how Gulp reports tasks failures: it should print a task-exit error instead of showing a stack-trace with internal Gulp invocations.

[rinatio]

Makes sense. Thank you

[aciccarello]

@pkozlowski-opensource
I think gulp is expecting an error object. The stack trace is supposed to stop at reportError but is throwing an error when trying to get the value to print.

Perhaps the callback function passed to the Karma server should be wrapped in something that will pass such an error to gulp. Something such as:

gulp.task('test', function(done) {
    karma.start({
        configFile: __dirname + '/karma.conf.js',
        singleRun: true
    }, function() {
        error = error ? new Error('Karma returned with the error code: ' + error) : undefined;
        done(error);
    });
});

It doesn't seem that Karma passes anything more than an error code though. An error that states 3 Tests failed instead of 1 would be helpful.

For reference, here is the formatError() implementation

// Format orchestrator errors
function formatError(e) {
  if (!e.err) {
    return e.message;
  }

  // PluginError
  if (typeof e.err.showStack === 'boolean') {
    return e.err.toString();
  }

  // Normal error
  if (e.err.stack) {
    return e.err.stack;
  }

  // Unknown (string, number, etc.)
  return new Error(String(e.err)).stack;
}