Skip to content

Updated test analytics doc for more details on how to read the data #15494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Updated test analytics doc for more details on how to read the data #15494

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7d0eac4
Updated test analytics doc for more details on how to read the data
Nov 12, 2025
f446f1c
adding image, fixing typo
Nov 12, 2025
9bf1c2b
Update docs/product/test-analytics/index.mdx
sfanahata Nov 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/product/test-analytics/img/all-branches-dash.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 15 additions & 2 deletions docs/product/test-analytics/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,28 @@ Now that you've configured your CI to upload your test results, you can inspect

After your workflow runs, view failed tests in the [failed tests dashboard](https://sentry.io/prevent/tests/?filterBy=failedTests), or in your GitHub pull request comments.

![Test Analytics comment](./img/TA-comment.png)
![Test Analytics comment =800x](./img/TA-comment.png)

You can see a complete overview of your test analytics in the [test analytics dashboard](https://sentry.io/prevent/tests/). **Note**: The `All Branches`dashboard refreshes every 24 hours.
### Reading Test Analytics Data
You can see a complete overview of your test analytics in the [test analytics dashboard](https://sentry.io/prevent/tests/).
**Note**: The `All Branches`dashboard refreshes every 24 hours.

![ALl branches dashboard](./img/all-branches-dash.png)

- `Failed Tests`: Shows all failed tests across all branches. Because of this, the _total run time_ may be higher than the filtered time range selected.
- `Flaky Tests`: These are tests that go from fail to pass or pass to fail compared to the default branch. This analysis will only run when:
- The commit is merged
- The commit is on the default branch (commit.branch == repo.branch)
- Flake detection is enabled on your yaml (default is true)
- Either: The repo is public (regardless of plan type); Or: The repo is private, and is above free or team plan types
Copy link

@thomasrockhu-codecov thomasrockhu-codecov Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adrianviquez can you fact check this?

- `Cumulative Failures`: Shows the number of individual test failures, only on the default branch.

## Permissions and Repository Tokens
When configuring Test Analytics, you'll be asked to choose your upload permissions. For GitHub Actions, you can use [OpenID Connect](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/about-security-hardening-with-openid-connect), or generate a repository token. Using the CLI, generate a repository token.

You can find a list of repository tokens on the Sentry Prevent [tokens page](https://sentry.io/prevent/tokens/).


## Read Next

<PageGrid />
Loading