Reduced output during testing#1110
Conversation
The Makefile has been modified to reduce the amount of output during testing. Output is generally omitted on a successful test. When a test fails, we only display a small portion of the total output. We also now run all tests, even if they fail, in order to give a complete profile of the test failures. Regression testing rules have been integrated into the general rules. Finally, Travis config has been modified to further reduce output, and to run all of tests (make -k).
Test logging is now much shorter, so folding is less important.
CodeCov reporting log is now saved to results/ rather than piped to stdout, further reducing test logging output.
Codecov Report
@@ Coverage Diff @@
## dev/gfdl #1110 +/- ##
=========================================
Coverage 46.06% 46.07%
=========================================
Files 214 214
Lines 69361 69352 -9
=========================================
- Hits 31954 31952 -2
+ Misses 37407 37400 -7
Continue to review full report at Codecov.
|
We were saving codecov output to results, but this was breaking the `test.summary` test, which only returns true if `results` is empty. This change should resolve this issue.
|
This seems good enough to merge from my end, but there is no hurry and plenty of room for feedback. |
|
Sample output from a run with answer changes: https://travis-ci.org/github/marshallward/MOM6/jobs/687177632 |
Hallberg-NOAA
left a comment
Hallberg-NOAA
left a comment
There was a problem hiding this comment.
I am not an expert in the syntax of Makefiles, so I will leave it to others to approve these changes, but the sample output showing the new log for a deliberately errant commit immediately shows that that this commit leads to a vast improvement over what was coming out before. Very well done, @marshallward !
4 participants
The Makefile has been modified to reduce the amount of output during testing. Output is generally omitted on a successful test. When a test fails, we only display a small portion of the total output.
We also now run all tests, even if they fail, in order to give a complete profile of the test failures.
Regression testing rules have been integrated into the general rules.
Finally, Travis config has been modified to further reduce output, and to run all of tests (make -k).