Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub test annotations can fail validation #832

Open
72636c opened this issue Apr 7, 2022 · 1 comment
Open

GitHub test annotations can fail validation #832

72636c opened this issue Apr 7, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@72636c
Copy link
Member

72636c commented Apr 7, 2022

I observed the following on a couple flaky database tests:

Summary of all failing tests
 FAIL  src/app.int.test.ts


  ● Test suite failed to run

    Connection terminated

      at Connection.<anonymous> (node_modules/pg/lib/client.js:132:36)
      at Socket.<anonymous> (node_modules/pg/lib/connection.js:57:12)


Test Suites: 1 failed, 23 passed, 24 total
Tests:       182 passed, 182 total
Snapshots:   61 passed, 61 total
Time:        56.839 s
Ran all test suites.
Failed to annotate results.
HttpError: Validation Failed: {"resource":"CheckRun","code":"invalid","field":"annotations"}

Unfortunately the error isn't very descriptive and I can't see anything obvious that would trip up GitHub's validation. I tried to fake a similar error locally and it seemed to produce a reasonable list of annotations:

[
  {
    annotation_level: 'failure',
    path: 'src/app.int.test.ts',
    start_line: 1,
    end_line: 1,
    message: 'Connection terminated',
    title: 'Jest'
  }
]
@samchungy
Copy link
Contributor

Should we log the parameters we call the checkRun with if we fail? I suspect the only way will be a bit of trial and error against the checkrun API to see what it doesn't like.

72636c added a commit that referenced this issue Apr 14, 2022
This should help to troubleshoot validation failures.

Suggested by @samchungy in #832.

Co-authored-by: Sam Chung <[email protected]>
72636c added a commit that referenced this issue Apr 14, 2022
This should help to troubleshoot validation failures.

Suggested by @samchungy in #832.

Co-authored-by: Sam Chung <[email protected]>
@72636c 72636c added the bug Something isn't working label Jun 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants