Add warning status for checks

[rousseldenis]

Referring to this issue: isaacs/github#1465

Should be a great feature!

Thanks

UPDATE: This is the original issue description:

When creating a status on a commit, there are only four choices: error, failure, pending, or success.

https://developer.github.com/v3/repos/statuses/

It would be useful to have a level called "warning". If a commit has statuses that are all success or warning, then an orange exclamation mark should be used to annotate the commit, instead of a red X. If there is at least one failure, then you should get the red X.

This would clean up the implementation of "recommended but not required" checks for PRs. For example, in unicode-org/icu, we have a recommended policy that a PR contains only one commit. We currently have this set up as a non-required status that fails, which causes the whole PR to get the red X, even when all of the other checks are green.

[xmo-odoo]

@rousseldenis you should copy over the entire description, possibly reworded or augmented with feedback from the comments, because it's uncomfortable to have to go over the linked issue every time in order to see / remember what the subject and justification actually is.

And since the isaacs repo has been archived it's not even possible to react (let alone comment), so the link really has no value other than archival.

[rousseldenis]

@xmo-odoo I've updated the description

[mattyb]

One use case for this would be visual diffs, where a detected difference requires review but doesn't necessarily mean there's an error.

[kuteninja]

I would use this also on our terraform repository, on cases where terraform plan works correctly and at least one element is going to be destroyed by it when applied. Just as an extra precaution.

[mturley]

This would also be useful for unit test coverage checks, where we don't want to require a coverage percentage to merge but we still want to be alerted to low coverage.

[rjeffman]

It is also useful when testing for future versions to be supported (e.g.: Python 3.12 as of today). CI should not break if a beta version fails, but should warn that some attention is needed.

[rousseldenis]

Any update on this ?

[Quba1]

Is there any new information about this feature?

[iliya-malecki]

any updates on this?

[mattcobb]

The general case for this would be any check that fails, but isn't set to block merging. You could drop the required to merge wording. Anyone visual can't see passed the red to read that anyway. I've worked with devs that been looking at PRs for years with red Xs and didn't realize the check was merge optional.

[daltonv]

This would help enable a setup where only CI blocks code that generates warnings from merge. Something like talked about here https://embeddedartistry.com/blog/2017/05/22/werror-is-not-your-friend/

[phyzical]

Our usecase is around implementing new checks for example say we implement a CVE checker, but don't want to make it blocking out the gate. we currently have to mark it as green which results in people forgetting to actually action them to a green state to start enforcing the checks.

I feel if it was always "warning" (orange) people would atleast notice it every now and then providing a nudge

[Julienraptor01]

This is smth that would really be nice
i would say it's actually even needed, because otherwise some people don't see those