Run Julia tests with resource limits, with worker process, and collate the results

[timholy]

This makes it much more feasible to run "all" of Julia's tests. The key advance is the ability to run a specified maximum number of statements in the interpreter, and abort early when that number is reached. Choosing a number of statements, rather than execution time, should make it easier to compare results on machines with different performance. On my laptop, the default settings run all of Julia's tests in < 5 minutes. This is not a sign that the interpreter is really efficient (it's not), just that it aborts a lot of slow tests.

Sample output is now visible in #13.

[KristofferC]

Would bolding the text in the table for those files that somehow fail make sense? Might make it easier to find them in the table and get an overview of the "density" of failures.

[codecov-io]

Codecov Report

Merging #24 into master will not change coverage.
The diff coverage is 0%.

@@          Coverage Diff          @@
##           master    #24   +/-   ##
=====================================
  Coverage       0%     0%           
=====================================
  Files           6      6           
  Lines        1115   1117    +2     
=====================================
- Misses       1115   1117    +2

Impacted Files	Coverage Δ
src/interpret.jl	0% <0%> (ø)	⬆️
src/JuliaInterpreter.jl	0% <0%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f099f83...6036751. Read the comment docs.

[timholy]

In the long run, I agree, but even if we discount the "aborted" column, a sizable majority of tests have some problem that Base itself does not. So most lines would be bolded. (The only tests that may not need attention are those with zeros in the "Fails" and "Errors" columns.)

[timholy]

OK to merge? Given that I've advertised #13 I'd like to make this runnable by others.

[timholy]

I now have the suspicion that anonymous functions are the overwhelming reason for the tests with Xs in #13. This is tricky territory; check out the results of Meta.lower(Main, :(map(x->x^2, [1,2,3]))). Because you're defining new structs there are things that have to be done at top-level and as a consequence you can get into world-age issues depending on how you handle this. While this PR tries to solve that problem, I suspect it's not really correct yet. I'm not immediately sure about the best fix.

[KristofferC]

Feel free to merge this and future PRs at will :)

[timholy]

I was just wondering how strongly you felt about the bold text suggestion.

[KristofferC]

weakly [======|                                ] strongly