Fix #5451: Produce a score of 0 for fatal errors

[jacobtylerwalls]

Type of Changes

	Type
✓	🐛 Bug fix

Description

Before
Modules without statements (such as typo'd paths) don't generate scores. When a linter is run over both valid and invalid paths, this can lead to a perfect score. Scores meeting the --fail-under threshold exit with a 0 code (passing), which is counterintuitive for the case just described.

Now
As long as --fail-under is 10.0, this PR fails runs with perfect scores if they had any non-zero exit codes.

Of course, the other approach would be to start emitting non-perfect scores for modules without statements, but this seemed less invasive.

Update: new approach is to simply produce a score of 0 for fatal errors.

Closes #5451

[DanielNoord]

I'm not sure if emitting non-perfect scores for all statement-less modules is correct, but I think we should emit a score for this error.
A fatal message is emitted and the user has taken extra care to setup pylint how they want it to behave (ie., change from the default 10 value). Fixing this in _report_evaluation makes more sense to me.

[Pierre-Sassoulas]

the other approach would be to start emitting non-perfect scores for modules without statements, but this seemed less invasive.

I think this makes more sense. We won't have to add a special case and the corresponding tests this way. The score should be 10 if there isn't any error or 0 if there is at least an error/warning/convention/info.

[jacobtylerwalls]

Sure, happy to go in the other direction.

the user has taken extra care to setup pylint how they want it to behave (ie., change from the default 10 value).

Just to be clear, the case was when the user has not changed from the default 10. We were emitting 10 anyway.

[coveralls]

Pull Request Test Coverage Report for Build 1579479507

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 73 unchanged lines in 5 files lost coverage.
• Overall coverage increased (+0.002%) to 93.656%

---

Files with Coverage Reduction	New Missed Lines	%
pylint/interfaces.py	1	96.77%
pylint/testutils/checker_test_case.py	3	93.48%
pylint/checkers/utils.py	9	95.46%
pylint/checkers/variables.py	28	95.63%
pylint/checkers/typecheck.py	32	94.98%

Totals
Change from base Build 1573223459:	0.002%
Covered Lines:	14217
Relevant Lines:	15180

---

💛 - Coveralls

[Pierre-Sassoulas]

I was wondering where the actual code change was, then I realized that the changed default value is valid python code. That's pretty clever !

[DanielNoord]

Small changes, mostly because I forgot to do stuff in #5318 😅

Note to self: copy pylintrc to examples/pylintrc in a follow-up PR.

[DanielNoord]

Thanks for the quick work @jacobtylerwalls!

I'll try and think of the pylintrc move tomorrow.