-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add coverage statistics #5907
Open
mildm8nnered
wants to merge
62
commits into
realm:main
Choose a base branch
from
mildm8nnered:mildm8nnered-add-coverage-statistics
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add coverage statistics #5907
mildm8nnered
wants to merge
62
commits into
realm:main
from
mildm8nnered:mildm8nnered-add-coverage-statistics
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Generated by 🚫 Danger |
2 tasks
mildm8nnered
force-pushed
the
mildm8nnered-add-coverage-statistics
branch
4 times, most recently
from
December 27, 2024 14:55
761d4f4
to
3274fa8
Compare
mildm8nnered
force-pushed
the
mildm8nnered-add-coverage-statistics
branch
from
December 28, 2024 13:05
17a31e1
to
61d61b5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WIP - Addresses #5906
Adds coverage reporting to SwiftLint, via a
--report-coverage
command line option, and areport_coverage
configuration file setting."Enabled rules coverage" is helpful for monitoring whether contributors are simply disabling SwiftLint for large sections of code or entire files (although blanket_disable_command, if not disabled, should help with that already).
"All rules coverage" is more useful - low values here indicate that a lot of rules are disabled (or suppressed), and that you could potentially be getting a lot more value out of SwiftLint.
Related changes - we now cache calls to
file.regions()
, as we'll call this at least twice if coverage is enabled.From https://github.com/mildm8nnered/SwiftLint/blob/mildm8nnered-add-coverage-statistics/Source/SwiftLintFramework/Coverage.swift:
Coverage is defined as the sum of the number of rules applied to each line of input, divided by the product of the
number of input lines and the total number of rules.
If all rules are applied to every input line, then coverage would be
1
(or 100%). If half the rules are appliedto all input lines, or if all the rules are applied to half of the input lines, then coverage would be
0.5
, andif no rules are enabled, or there is no input, then coverage would be zero.
No distinction is made between actual lines of Swift source, versus blank lines or comments, as SwiftLint may
apply rules to those as well. Coverage is only calculated over input files, so if you exclude files in your
configuration, they will be ignored. Empty input files, or files containing a single empty line will be ignored, as
SwiftLint ignores these files automatically.
"All rules" can be defined either as all enabled rules, or all available rules, enabled or not, resulting
in two different coverage metrics:
swiftlint:disable
commands.
Typically, enabled rules coverage will be close to
1
, asswiftlint:disable
is used sparingly. All rulescoverage will generally be much lower, as some rules are contradictory, and many rules are optin. With no opt-in
rules enabled, all rules coverage will be about
0.4
, rising to0.8
or more if many opt-in rules are enabled.When calculating all rules coverage
swiftlint:disable
commands are still accounted for, but only for enabledrules.
Coverage figures will be different for linting and analyzing as these use different sets of rules.
The number of enabled rules is determined on a per-file basis, so child and local configurations will be accounted
for.
Custom rules, if enabled and defined, will be counted as first class rules for both enabled and all rules coverage.
If not enabled,
custom_rules
will be counted as a single rule, even if a configuration exists for it.When calculating enabled rules coverage, the custom rules in the configuration for each input file (e.g. including
child configurations) will be taken into account. When calculating all rules coverage, only the main configurations
custom rules settings will be used.