Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add Actions annotation output #34590

Closed
wants to merge 2 commits into from

Conversation

mmarchini
Copy link
Contributor

@mmarchini mmarchini commented Aug 1, 2020

It's possible to annotate failures in Actions by printing ::error file={},line={},col={}::{message}. This methos is preferrable over using a problem matcher because problem matchers only allow single-line messages, whereas ::error allows multi-line messages.

When the test being annotated is edited in a Pull Request, GitHub will show it inline where the error happened:

image

If the failing test was not edited, it will still be possible to see the error with the following steps:

image

image

image

The link on the annotation in this case will point to the last commit in the PR instead of pointing to the file (which is a bit unfortunate, but that's a GitHub Actions Runner limitation apparently). The filename on the annotation is still correct though, so this is still an improvement over the current situation where we need to look at the logs (which can be confusing for newcomers).

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Aug 1, 2020

not lite ci: redacted to fix ncu

@nodejs-github-bot nodejs-github-bot added meta Issues and PRs related to the general management of the project. test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory. labels Aug 1, 2020
@mmarchini mmarchini marked this pull request as ready for review August 1, 2020 07:13
tools/test.py Outdated Show resolved Hide resolved
It's possible to annotate failures in Actions by printing
"::error file={},line={},col={}::{message}". This methos is preferrable
over using a problem matcher because problem matchers only allow
single-line messages, whereas ::error allows multi-line messages.
mmarchini added a commit that referenced this pull request Aug 7, 2020
It's possible to annotate failures in Actions by printing
"::error file={},line={},col={}::{message}". This methos is preferrable
over using a problem matcher because problem matchers only allow
single-line messages, whereas ::error allows multi-line messages.

PR-URL: #34590
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
@mmarchini
Copy link
Contributor Author

Landed in 46bef7b

@addaleax
Copy link
Member

addaleax commented Aug 8, 2020

@mmarchini This would need a manual backport to v14.x, unfortunately

targos pushed a commit that referenced this pull request May 16, 2021
It's possible to annotate failures in Actions by printing
"::error file={},line={},col={}::{message}". This methos is preferrable
over using a problem matcher because problem matchers only allow
single-line messages, whereas ::error allows multi-line messages.

PR-URL: #34590
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
targos pushed a commit that referenced this pull request Jun 11, 2021
It's possible to annotate failures in Actions by printing
"::error file={},line={},col={}::{message}". This methos is preferrable
over using a problem matcher because problem matchers only allow
single-line messages, whereas ::error allows multi-line messages.

PR-URL: #34590
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta Issues and PRs related to the general management of the project. test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants