Clarify when no coverage files appear

[codecovdesign]

*marking as bug to discuss whether bug for fr in sync

Problem to solve

The Codecov report appears in the PR comment even when there are no coverable files – this causes some user confusion as reported recently. (scenario when yaml or JSON file appears)

Solution

IF no coverable files are changed, the report can appear as:

[thomasrockhu-codecov]

I think this is a feature request @codecovdesign

[eliatcodecov]

My hunch is this is really straightforward, but is still a feature request. @adrian-codecov can you spend an hour on this and if it's easy knock it out. If not, we can kick it back into the queue.

[adrian-codecov]

My thoughts is to leverage the diff_totals https://github.com/codecov/worker/blob/6cb0a30e90fcbfa64931d28ae51152306161bade/services/notification/notifiers/mixins/message/writers.py#L31-L32 and seeing if it has any file -> did an experiment with a PR just with a readme change, with a coverable file, and a readme + a coverable file, and only the readme change came with 0 files in it. This should be a reasonable assumption that those files didn't lead to coverage.

Adding that logic unexpectedly broke many tests, I think because those aren't correctly mocking the diff_totals/apply_diff functionality. Notifications tend to be spaghetti code/difficult to deal with, so this could maybe like ~1-2 days to figure out those tests. Probably for the better but a little more time expensive than anticipated.

Please let me know how I should go about this and I'll get it going 👌

[codecovdesign]

sync with @eliatcodecov

• based on discussion and @adrian-codecov write up; agreed to move forward so long as it's 1-2 days
• related UX could be not showing comment at all, related to potential changes from {Codecov upload issue debugging} - Ensure Codecov consistently waits for ALL reports to successfully process before reporting on coverage on UI and PR comment #2702 we could opt to NOT show comment and update the copy in the status check – therefore it aligns with the upcoming UX (if that becomes feasible)

[eliatcodecov]

From bug sync: this ended up being non-trivial, so it may need to be a larger feature request @codecovdesign to follow up with @adrian-codecov to better capture the complexity on this ticket. Once captured we can try to slot this in as more of a feature request to be tackled as a larger scope of work.