adding json-spec reporter

[ben-bradley]

The intended use of this reporter is to make it possible for a hybrid of manual/automated tests to be executed. In some cases, we want to be able to run tests and observe the results in real time via the spec reporter, but we also want to be able to capture the results in json so that they can be archived and reported on in our CI system.

The json-spec reporter allows for this by having the spec output sent to stderr and the json output sent to stdout. Exit codes are left unaffected. This makes it possible to run tests like this:

$ mocha -R json-spec >> ~/testresults/mytest.json
/* stderr =>
 * {{ spec reporter output here }}
 */
$ cat ~/testresults/mytest.json
/* stdout =>
 * {{ json output here }}
 */

[ben-bradley]

I always name my first commit "initial drop", but when I read the title of this merge as "dropping json-spec reporter" I thought it might sound like I was trying to remove something so I renamed it.

[boneskull]

This looks like a workaround for the lack of multiple reporters.

[boneskull]

@ben-bradley Will #1360 theoretically solve your issue?

I'm wanting to decline this; seems like it's opening the door for more mashup-like frankenreporters.

[ben-bradley]

@boneskull #1360 wouldn't work since it's still outputting the spec to stdout and any kind of output redirection would include that and invalidate the JSON.

The thing that is great about the spec reporter is that it provides immediate feedback so that if you're running a long test, you can see how it's going right away, but you can't preserve the results in an automated fashion. This means that you have to run tests twice if you want to do any kind of validation prior to publishing the results.

The thing that's great about the json reporter is that it integrates very nicely into a multitude of automated CI systems, but you can't observe the progress while the tests run. This means that if there is a small typo in a config file, you won't know about it until you've looked at the resulting json produced at the very end of the test.

By sending the spec output to stderr and the resulting json output to stdout, we're getting the best of both worlds at the cost of slightly misusing the concept of stderr.

[boneskull]

Instead of random mashups of reporter x with reporter y, we need support for multiple reporters.

#1360 doesn't work, but in theory, it could, if output of every reporter is configurable: stdout, stderr, file, some other stream.

Going to decline this in lieu of an improved #1360 or some other PR for multiple reporters.