Ensure API returns file list consistent with PR comment

[trent-codecov]

The PR comment is displaying files with 0% change. We want the UI (via the API) to display these files as well.

Tasks

Beta Give feedback

1. API returns 0% changed files
2. Look for areas of further discrepancy between PR comment and API return

[scott-codecov]

@aj-codecov Do we have an example of this not working somewhere in the wild? I just tested this and I see files with 0% change in both the PR comment as well as the dashboard (main/bar.py is a file w/ diff changes but no coverage changes).

PR comment:

Dashboard:

[scott-codecov]

Oh maybe this is an example? codecov/worker#61

The PR comment is showing test files b/c they're part of the diff. Is that the behavior we'd like to replicate in the dashboard? That honestly seems a little weird to me. I'd maybe suggest removing the test files from the PR comment instead. WDYT?

[aj-codecov]

@scott-codecov Yes, that's an example, basically those are files that are changed that can impact coverage within a PR, almost more of an informational thing than anything. @codecovdesign curious if you have thoughts here - I see the potential value in including this in the UI, but we'd need to probably put them in their own category, or label them somehow to call out that they're test files and thus different from everything else in the UI.

[scott-codecov]

Thanks @aj-codecov - I did a little more exploration here and better understand what's going on now. In worker we're actually including test coverage of the test files (which I think is where the "mistake" is). I think it makes sense to add something like this to our codecov.yml:

ignore:
  - "**tests**/test_*.py"

Regardless, there is still a discrepancy between the PR comment and the UI in that the PR comment will show files that are part of the diff AND have some coverage info even if there is no coverage change in the diff for that file. Question is - do we want to show those files in the UI as well?

Am I understanding that correctly?

[aj-codecov]

Scott and I chatted through this further in Slack, but the end result of that convo was that we want to include all files changed in a PR that could impact coverage in the "files changed" section of the pulls view.