Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docs: pipeline exploration #36961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Docs: pipeline exploration #36961
Changes from all commits
8f31eb639914a4File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing
R&D: Testing via GitHub Actions (CI)
This document captures R&D findings on how GitHub Actions can be used for automated testing in a CI pipeline. It focuses on which test types are suitable for early automation and how test failures are identified and analyzed from a tester’s perspective.
1. What kinds of tests can realistically run in GitHub Actions?
The following types of tests are well suited for execution in GitHub Actions because they are fast, reliable, and provide clear pass or fail results.
Unit Tests
Unit tests are ideal for GitHub Actions because they execute quickly and validate individual functions or components. They provide immediate feedback when core logic is broken and are highly reliable in CI environments.
Lint Checks
Lint checks enforce coding standards and detect common issues such as unused variables or formatting problems. These checks are fast and prevent poor-quality code from entering the main branch.
Build Verification
Build verification ensures that the application compiles or builds successfully. Running builds in CI helps catch dependency issues or configuration errors early before code is merged.
API Smoke Tests
API smoke tests can be executed in GitHub Actions to validate critical endpoints such as health checks or authentication APIs. These tests should remain minimal to keep CI execution fast and stable.
Static Code Analysis
Static code analysis scans the codebase for security vulnerabilities, code smells, and maintainability issues. These tools run without requiring a runtime environment, making them suitable for CI pipelines.
What is worth automating early vs what is not
Early CI automation should focus on tests that are fast, deterministic, and environment independent. Unit tests, lint checks, and build verification provide the highest value early because they fail quickly and are easy to maintain.
API smoke tests are useful early but should remain limited to critical endpoints only to avoid instability.
UI automation and performance testing are not suitable for early CI automation because they are slow, flaky, and require stable environments and test data. These tests are better executed in later stages or dedicated testing pipelines.
2. How test failures are reported in GitHub Actions
Where logs appear
Test failure logs are available in the Actions tab of the GitHub repository. Each workflow run displays jobs and steps with expandable logs, allowing testers to review detailed output and error messages.
How to identify which step failed
When a failure occurs, the failed job or step is marked with a red indicator. The step name clearly shows which action failed, such as test execution, linting, or build steps. Error messages and exit codes are visible in the step logs.
Test failure vs workflow failure
A test failure occurs when tests execute but fail due to assertion or validation errors, indicating an application defect.
A workflow failure occurs when the CI pipeline itself fails due to configuration issues, dependency installation problems, or environment errors, and does not represent an application bug.