Skip to content

New test analytics doc for docs.sentry #14296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/product/test-analytics/img/TA-comment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/product/test-analytics/img/TA-dash.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
56 changes: 56 additions & 0 deletions docs/product/test-analytics/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: Test Analytics
sidebar_order: 135
description: Learn about how Sentry Prevent's Test Analytics tools can help you improve your code.
---

Sentry Prevent Test Analytics provides actionable insights into your CI test runs, helping you identify flaky tests, track failures, and optimize your test suite for faster, more reliable deployments.

## Why Use Test Analytics?

- **Spot Flaky Tests:** Quickly identify and address tests that fail intermittently, reducing noise and wasted debugging time.
- **Track Failures and Trends:** Monitor cumulative failures, slowest tests, and overall test run efficiency to improve CI reliability.
- **Actionable Insights:** See which lines of code failed within tests, confirm flaky tests, and prioritize fixes.
- **Integrated with Sentry:** Access test analytics alongside your error and performance data for a unified developer experience.

## Getting Started

<Alert>
Sentry Prevent Test Analytics is currently in <strong>beta</strong>. Beta features are still a work in progress and may have bugs.
</Alert>

![Test Analytics Dashboard](./img/TA-dash.png)

To use Sentry Prevent Test Analytics, you’ll need to install the [Sentry app](https://github.com/apps/sentry-io) on your GitHub organization or specific repositories. Once installed, choose your JUnit XML Report language and [set your permissions](#permissions-and-repository-tokens) using a repository secret.

**There are two ways to upload your test results:**

### 1. Use GitHub Actions to Run Your CI
Add the `getsentry/prevent-action` script to your CI YAML file.

### 2. Use Sentry Prevent’s CLI to Upload Testing Reports
Install the Sentry Prevent CLI using `pip install sentry-prevent-cli` (for Python), or use a binary. Then, upload your test results using the CLI:

```bash
sentry-prevent-cli db upload --report-type test_results --file <report_name>.junit.xml
```

### Run Your Test Suite
Now that you've configured your CI to upload your test results, you can inspect the workflow logs to see if the call to Sentry succeeded. You need to have some failed tests to view the failed tests report.

## Viewing Test Analytics

After your workflow runs, view failed tests in the [failed tests dashboard](https://sentry.io/prevent/failed-tests/), or in your GitHub pull request comments.

![Test Analytics comment](./img/TA-comment.png)

You can see a complete overview of your test analytics in the [test analytics dashboard](sentry.io/prevent/).

## Permissions and Repository Tokens
When configuring Test Analytics, you'll be asked to choose your upload permissions. For GitHub Actions, you can use [OpenID Connect](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/about-security-hardening-with-openid-connect), or generate a repository token. Using the CLI, generate a repository token.

You can find a list of repository tokens on the Sentry Prevent [tokens page](https://sentry.io/prevent/tokens/).

## Read Next

<PageGrid />
Loading