Failed check status 1 - No coverage information found on head

[Adal3n3]

CI failure scenarios: No coverage info for HEAD commit

PR check status message: Your project check has failed due to HEAD missing a report. Please upload the report before proceeding to push a pull request.

Use-case: Project coverage

User configurable: yes

[aj-codecov]

Verbiage revision: "This commit (sha link) is missing coverage information causing a failure of your Codecov project check. Please upload a coverage report for this commit."

[Adal3n3]

umm... the revision doesn't call out the HEAD missing a report

[giovanni-guidini]

Interestingly it seems that we already do this: https://github.com/codecov/worker/blob/d4496ed0e950f772c5ad0e33e770bf251223d87e/services/notification/notifiers/mixins/message/sections.py#L148

Apparently we have a message in the PR comment when there's no coverage information for the base or the head. It links to https://docs.codecov.com/docs/error-reference#section-missing-pull_request-commit

It's not explicit if it's the BASE or HEAD that's missing, but you have the branch and the commit info.

[Adal3n3]

@giovanni-guidini That's great, and can we improve the existing error messaging? A good error message should be specific and give user a good direction so...

Should it be link to https://docs.codecov.com/docs/error-reference#missing-head-commit

Can we be specific about it's missing a head commit?

[giovanni-guidini]

Yes, we can change the error messaging for sure.

I'm not quite convinced this message will ever be sent for the head commit though.
I'll try to have the message sent to some PR, but the reality might be that we just don't notify at all if the HEAD report is missing.
Stand by for further updates.

UPDATE
As I suspected I was not able to make the 'missing info for head commit' message appear in the PR comment. Maybe we should instead focus on editing this message - current head differs from pull most recent head - and make it more actionable.

As mentioned in the detailed investigation, the UI messages could also be more actionable.

See more details here

[Adal3n3]

Current message:

❗️ Current HEAD 0a9b952 differs from pull request most recent HEAD aadfd57. Consider uploading reports for the commit aadfd57 to get more accurate results.

Proposed message:

❗️ Current HEAD 0a9b952 differs from the most recent HEAD aadfd57 of the pull request. Please upload reports for the commit aadfd57 before proceeding to push a pull request.

Please review the proposed message @giovanni-guidini @aj-codecov
Q: report or reports?

[giovanni-guidini]

I don't particularly like "before proceeding to push a pull request". Technically you push commits. They can be to a pull request. And pushing a new commit would change the HEAD, and we might have a new report for it anyway.

I'd keep the ending of the current message. "Please upload reports for the commit aadfd57 to get more accurate results."

I'd also maybe break it into 2 separate lines to make it more similar to a callout.

❗️ Current HEAD 0a9b952 differs from pull request most recent HEAD aadfd57.

Please upload reports for the commit aadfd57 to get more accurate results.

[Adal3n3]

@giovanni-guidini Thanks for elaborating the nuances. That makes sense.