Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: Comment to look up Flaky test tracker on e2e test failure #6292

Merged
merged 17 commits into from
Jun 12, 2024
Merged

CI: Comment to look up Flaky test tracker on e2e test failure #6292

merged 17 commits into from
Jun 12, 2024

Conversation

potatowagon
Copy link
Contributor

@potatowagon potatowagon commented Jun 11, 2024 •
edited
Loading

Description

Flaky test tracker: #6289

i chose to use discussions because we can reply to a comment. each comment will record the flaky test name, and each reply will contain the link to the failed workflow run.

on e2e test failure (change this to run: exit 1), a comment is left on the PR to look up the flaky test tracker

Test plan

#6292 (comment)

Screenshot 2024-06-11 at 3 41 27 PM

@vercel Vercel
Copy link

vercel bot commented Jun 11, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
lexical ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 11, 2024 8:06am
lexical-playground ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 11, 2024 8:06am

@potatowagon potatowagon marked this pull request as draft June 11, 2024 03:49
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 11, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

size-limit report 📦

Path Size
lexical - cjs 28.31 KB (0%)
lexical - esm 28.13 KB (0%)
@lexical/rich-text - cjs 36.77 KB (0%)
@lexical/rich-text - esm 28.09 KB (0%)
@lexical/plain-text - cjs 35.36 KB (0%)
@lexical/plain-text - esm 25.33 KB (0%)
@lexical/react - cjs 38.51 KB (-0.12% 🔽)
@lexical/react - esm 29.14 KB (0%)

@github-actions GitHub Actions
Copy link

Hello world ! 👋

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited by potatowagon
Loading

E2E tests failed on this run. Please check if the failure is due to a flaky test. Visit the Flaky Test Tracker for known flaky tests, and record the failed test run there if its a flaky failure.

@potatowagon potatowagon changed the title test CI: Comment to look up Flaky test tracker on e2e test failure Jun 11, 2024
@potatowagon potatowagon marked this pull request as ready for review June 11, 2024 08:02
@github-actions GitHub Actions
Copy link

E2E tests failed on this run. Please check if the failure is due to a flaky test. Visit the Flaky Test Tracker for known flaky tests, and record the failed test run there if its a flaky failure.

1 similar comment
@github-actions GitHub Actions
Copy link

E2E tests failed on this run. Please check if the failure is due to a flaky test. Visit the Flaky Test Tracker for known flaky tests, and record the failed test run there if its a flaky failure.

@Sahejkm
Copy link
Contributor

Sahejkm commented Jun 11, 2024

LGTM, Does the bot work on every failure (it commented 2wice after approval for 2 failures) ? does it make sense to comment once after all e2e run is finished (the size-limit bot right now edits existing comment rather creating a new comment, could refer that as well)

@potatowagon
Copy link
Contributor Author

LGTM, Does the bot work on every failure (it commented 2wice after approval for 2 failures) ? does it make sense to comment once after all e2e run is finished (the size-limit bot right now edits existing comment rather creating a new comment, could refer that as well)

yup the idea is to just comment once, i couldnt test untill after approval so ill fix it

@potatowagon
Copy link
Contributor Author

will raise a seperate PR to address duplicate comment

@potatowagon potatowagon added this pull request to the merge queue Jun 12, 2024
Merged via the queue into main with commit 25ba423 Jun 12, 2024
33 of 39 checks passed
This was referenced Jul 23, 2024
Open
Open
@potatowagon potatowagon deleted the flaky-test-notice branch July 26, 2024 05:51
@potatowagon potatowagon restored the flaky-test-notice branch July 26, 2024 05:51
@potatowagon potatowagon deleted the flaky-test-notice branch September 11, 2024 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Sahejkm Sahejkm Sahejkm approved these changes

@zurfyx zurfyx Awaiting requested review from zurfyx zurfyx is a code owner

@fantactuka fantactuka Awaiting requested review from fantactuka fantactuka is a code owner

@acywatson acywatson Awaiting requested review from acywatson acywatson is a code owner

@Fetz Fetz Awaiting requested review from Fetz Fetz is a code owner

@ivailop7 ivailop7 Awaiting requested review from ivailop7 ivailop7 is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@potatowagon @Sahejkm @facebook-github-bot