feat(hip-tests): add test filter standardization by amilanov-amd · Pull Request #12266 · ROCm/rocm-systems

amilanov-amd · 2026-09-25T13:22:40Z

Motivation

Add test filter standardization to hip-tests

Technical Details

configs/test_categories.yaml defines four categories as sets of levels:

quick {0}
standard {0, 1}
comprehensive {0, 1, 2}
full {0, 1, 2 , 3, 4}

parse_config.py emits each as a Catch2 tag

Because ctest -L labels never reaches the binary, categories are resolves via a map emitted into hip_test_parameters.hh — so HIP_TEST_LEVEL=quick selects and parameterizes, reusing the existing priority/filter/listener logic unchanged.

Exclusions reuse the existing yaml config disabled list, which emits exclusion tags in the form exclude_<>. Supported exclusions are:

{platform}_{os_name} amd_linux, nvidia_windows, ...
{arch} any platform and OS
{os_name}{arch} linux, windows_
asan ASAN builds only

create_test_definition also now accepts {os_name}_{arch} in the disabled list

Issue Tracking

JIRA ID: AIRUNTIME-2908

Test Plan

Ensure build passes. All tests pass

Test Result

Local build passed, local tests pass

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

therock-pr-bot · 2026-09-25T13:23:09Z

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

therock-pr-bot · 2026-09-25T13:23:11Z

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Copilot

Copilot review overview

🟢 Approval recommended

Only a minor documentation schedule alignment nit remains; no blocking issues were identified.

Review effort: Lite
Findings: None

What changed in this PR

Adds standardized quick through full test categories to hip-tests, with YAML-driven Catch2 tags, level resolution, exclusions, tests, and documentation.

Changes:

Generates category tags and C++ level mappings.
Supports category names in HIP_TEST_LEVEL and Catch2 filters.
Adds exclusion handling, tests, documentation, and build integration.

File	Reviewed change
`projects/hip-tests/catch/README.md`	Documents categories and exclusions.
`projects/hip-tests/catch/include/hip_test_level.hh`	Documents category-aware level selection.
`projects/hip-tests/catch/hipTestMain/tests/hip_test_level_test.cc`	Adds category resolution tests.
`projects/hip-tests/catch/hipTestMain/tests/CMakeLists.txt`	Adds generated-header dependencies and include paths.
`projects/hip-tests/catch/hipTestMain/hip_test_level.cc`	Resolves category names into test levels.
`projects/hip-tests/catch/config/parse_config.py`	Generates category tags and mappings.
`projects/hip-tests/catch/config/configs/test_categories.yaml`	Defines categories and execution metadata.
`projects/hip-tests/catch/config/CMakeLists.txt`	Integrates category configuration generation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Copilot review overview

🟢 Approval recommended

The implementation, build integration, documentation, and tests consistently support the proposed category model.

Review effort: Balanced
Findings: None

mangupta

🤖 AI-assisted pre-review (for @mangupta)

mangupta · 2026-10-06T08:58:34Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (db743eacac44) — no new findings at or above the posting threshold.

mangupta · 2026-10-06T13:35:24Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (5ac00be5176f) — no new findings at or above the posting threshold.

mangupta · 2026-10-21T01:06:10Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (190ee174c222) — no new findings at or above the posting threshold.

## Motivation Add test filter standardization to hip-tests ## Technical Details configs/test_categories.yaml defines four categories as sets of levels: - quick {0} - standard {0, 1} - comprehensive {0, 1, 2} - full {0, 1, 2 , 3, 4} parse_config.py emits each as a Catch2 tag Because ctest -L labels never reaches the binary, categories are resolves via a map emitted into hip_test_parameters.hh — so HIP_TEST_LEVEL=quick selects and parameterizes, reusing the existing priority/filter/listener logic unchanged. Exclusions reuse the existing yaml config disabled list, which emits exclusion tags in the form exclude_<>. Supported exclusions are: - {platform}_{os_name} amd_linux, nvidia_windows, ... - {arch} any platform and OS - {os_name}_{arch} linux_<arch>, windows_<arch> - asan ASAN builds only create_test_definition also now accepts {os_name}_{arch} in the disabled list ## Issue Tracking JIRA ID: AIRUNTIME-2908 ## Test Plan Ensure build passes. All tests pass ## Test Result Local build passed, local tests pass ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

systems-assistant · 2026-10-29T04:43:19Z

TheRock Submodule Bump Activity

Newest first

2026-10-29 — Included in TheRock via ROCm/TheRock#9567.

amilanov-amd added this pull request to stack #11938 September 25, 2026 13:22

amilanov-amd requested review from a team as code owners September 25, 2026 13:22

Copilot AI lite review requested due to automatic review settings September 25, 2026 13:22

github-actions Bot added the project: hip-tests label Sep 25, 2026

therock-pr-bot Bot added the Not ready to Review label Sep 25, 2026

Copilot started reviewing on behalf of amilanov-amd September 25, 2026 13:23

View session

Copilot AI reviewed Sep 25, 2026

View reviewed changes

systems-assistant Bot added the organization: ROCm label Sep 25, 2026

mangupta requested a balanced review from Copilot September 28, 2026 13:01

Copilot started reviewing on behalf of mangupta September 28, 2026 13:02

View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from feaea65 to 62be4ed Compare September 28, 2026 15:26

mangupta reviewed Sep 29, 2026

View reviewed changes

Comment thread projects/hip-tests/catch/include/hip_test_level.hh Resolved

Comment thread projects/hip-tests/catch/hipTestMain/hip_test_level.cc Resolved

Comment thread projects/hip-tests/catch/hipTestMain/tests/CMakeLists.txt Resolved

mangupta added the lrt-bot: ready label Sep 30, 2026

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from 62be4ed to db743ea Compare September 30, 2026 14:06

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from db743ea to 5ac00be Compare October 6, 2026 13:24

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from 5ac00be to 4629a19 Compare October 7, 2026 12:14

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from 4629a19 to dd667b7 Compare October 8, 2026 11:42

Base automatically changed from users/amilanov/add-level-default-all-remove-cmd-options to develop October 8, 2026 16:49

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from dd667b7 to a86bc61 Compare October 8, 2026 16:49

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from a86bc61 to b63bd18 Compare October 9, 2026 12:58

amilanov-amd changed the title ~~[hip-tests] Add test filter standardization~~ feat(hip-tests): add test filter standardization Oct 9, 2026

mangupta added lrt-bot: review and removed lrt-bot: ready labels Oct 16, 2026

[hip-tests] add test filter standardization

190ee17

amilanov-amd force-pushed the users/amilanov/add-test-filter-standardization branch from b63bd18 to 190ee17 Compare October 20, 2026 14:51

mangupta added lrt-bot: ready and removed lrt-bot: review labels Oct 22, 2026

mangupta approved these changes Oct 28, 2026

View reviewed changes

amilanov-amd merged commit 727b8b5 into develop Oct 28, 2026
317 of 328 checks passed

amilanov-amd deleted the users/amilanov/add-test-filter-standardization branch October 28, 2026 14:06

mangupta removed the lrt-bot: ready label Oct 29, 2026

Navigation Menu

Repository navigation

feat(hip-tests): add test filter standardization - #12266

feat(hip-tests): add test filter standardization#12266
amilanov-amd merged 1 commit into
developfrom
users/amilanov/add-test-filter-standardization

amilanov-amd commented Sep 25, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Sep 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

mangupta left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mangupta commented Oct 6, 2026

Uh oh!

mangupta commented Oct 6, 2026

Uh oh!

mangupta commented Oct 21, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

amilanov-amd commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Issue Tracking

Test Plan

Test Result

Submission Checklist

Uh oh!

therock-pr-bot Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Policy Checks Passed

Uh oh!

therock-pr-bot Bot commented Sep 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

mangupta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mangupta commented Oct 6, 2026

Uh oh!

mangupta commented Oct 6, 2026

Uh oh!

mangupta commented Oct 21, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 29, 2026

TheRock Submodule Bump Activity

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

amilanov-amd commented Sep 25, 2026 •

edited

Loading

therock-pr-bot Bot commented Sep 25, 2026 •

edited

Loading