Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set php and js coverage threshold to 0.1% #5670

Merged
merged 1 commit into from
Dec 13, 2020

Conversation

pierlon
Copy link
Contributor

@pierlon pierlon commented Dec 8, 2020

Summary

Bumps the coverage threshold for the php and js projects from 0% to 0.1%

Checklist

  • My pull request is addressing an open issue (please create one otherwise).
  • My code is tested and passes existing tests.
  • My code follows the Engineering Guidelines (updates are often made to the guidelines, check it out periodically).

@pierlon pierlon added the WS:Core Work stream for Plugin core label Dec 8, 2020
@pierlon pierlon self-assigned this Dec 8, 2020
@pierlon
Copy link
Contributor Author

pierlon commented Dec 8, 2020

Would love to get @schlessera's feedback on this to see if we would be shooting ourselves in the foot with this change.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2020

Plugin builds for dffc5c4 are ready 🛎️!

@schlessera
Copy link
Collaborator

@pierlon I might be missing something here: It looks from the diff like you're overriding the default threshold: 0.1% with language-specific settings of that very same threshold... ?

@pierlon
Copy link
Contributor Author

pierlon commented Dec 12, 2020

@schlessera the Codecov docs gives the impression that the default threshold is applied to the language-specific settings, but in practice that has turned out to be false. We've been having PR checks fail even when the threshold is below the expected 0.1%.

To verify this I forked the project and tested my hypothesis. As you can see in pierlon#3, at pierlon@424ddda the codecov php test failed when it decreased by 0.01%. When I set the threshold to 0.1% for php in pierlon@384e224, the coverage test passed.

@schlessera
Copy link
Collaborator

Alright. I do think that looks like a bug in CodeCov, it does not make full sense the way I read the configs.

Copy link
Collaborator

@schlessera schlessera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try and see how this goes. Right now, I think our code coverage is a bit flaky overall because it use pcov, which seems to have trouble flagging everything it should...

@westonruter westonruter merged commit dbd7d60 into develop Dec 13, 2020
@westonruter westonruter deleted the set-coverage-threshold-for-projects branch December 13, 2020 17:42
@westonruter westonruter added this to the v2.1 milestone Dec 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
WS:Core Work stream for Plugin core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants