Skip to content

Workaround for running full instead of standard #5766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amd-hsivasun merged 2 commits into from
Mar 24, 2026
Merged

Workaround for running full instead of standard #5766

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d09522
Workaround for running full instead of standard
aalabsi-amd Mar 24, 2026
10cc23a
remove duplication
aalabsi-amd Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 34 additions & 7 deletions projects/miopen/test/gtest/test_categories.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,13 +242,7 @@ test_categories:

full:
description: "Complete test suite - (target: hours+)"
test_patterns:
# Everything - all test suite prefixes
- "Smoke/*"
- "Full/*"
- "Perf/*"
- "Unit/*"
- "*"
test_patterns: *positive_patterns

exclude:

@JonathanLichtnerAMD JonathanLichtnerAMD Mar 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this duplicate exclude as well?


Expand All @@ -258,6 +252,39 @@ test_categories:
- "gtest/*.cpp"

exclude:
# Excluding slower tests for faster PR validation
- "*DeepBench*"
- "*MIOpenTestConv*"
- "Full/GPU_Reduce_FP64*"
- "Full/GPU_BNOCLFWDTrainSerialRun3D_BFP16*"
- "Full/GPU_Lrn_FP32*"
- "Full/GPU_Lrn_FP16*"
- "Full/GPU_BNOCLInferSerialRun3D_BFP16*"
- "Smoke/GPU_BNOCLFWDTrainLarge2D_BFP16*"
- "Smoke/GPU_BNOCLInferLarge2D_BFP16*"
- "Full/GPU_BNOCLBWDSerialRun3D_BFP16*"
- "Smoke/GPU_BNOCLBWDLarge2D_BFP16*"
- "Full/GPU_UnitTestActivationDescriptor_FP32*"
- "Full/GPU_UnitTestActivationDescriptor_FP16*"
- "Smoke/GPU_BNOCLBWDLargeFusedActivation2D_BFP16*"
- "Smoke/GPU_BNOCLBWDLargeFusedActivation2D_FP16*"
- "Full/GPU_ConvGrpBiasActivInfer_BFP16*"
- "Full/GPU_ConvGrpBiasActivInfer_FP32*"
- "Full/GPU_ConvGrpBiasActivInfer_FP16*"
- "Full/GPU_ConvGrpActivInfer_BFP16*"
- "Full/GPU_ConvGrpActivInfer_FP32*"
- "Full/GPU_ConvGrpActivInfer_FP16*"
- "Full/GPU_ConvGrpBiasActivInfer3D_BFP16*"
- "Full/GPU_ConvGrpBiasActivInfer3D_FP32*"
- "Full/GPU_ConvGrpBiasActivInfer3D_FP16*"
- "Full/GPU_ConvGrpActivInfer3D_BFP16*"
- "Full/GPU_ConvGrpActivInfer3D_FP32*"
- "Full/GPU_ConvGrpActivInfer3D_FP16*"
- "Smoke/GPU_UnitTestConvSolverHipImplicitGemmV4R1Fwd_BFP16.ConvHipImplicitGemmV4R1Fwd/0"

@randyspauldingamd randyspauldingamd Mar 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one's missing a wildcard. I believe the intention is to disable all BFP16-Fwd tests. This instantiation shares a test case generator with several others, so it should be Smoke/GPU_UnitTestConvSolverHipImplicitGemmV4R1Fwd_BFP16.ConvHipImplicitGemmV4R1Fwd* to immunize us in the off chance someone adds more shapes.

Ref:

rocm-libraries/projects/miopen/test/gtest/unit_conv_solver_ConvHipImplicitGemmV4R1Fwd.cpp

Lines 112 to 124 in 5788c16

// Smoke tests
INSTANTIATE_TEST_SUITE_P(Smoke,
GPU_UnitTestConvSolverHipImplicitGemmV4R1Fwd_FP16,
testing::Combine(testing::Values(GetTestParamsFP16()),
testing::Values(miopenConvolutionAlgoImplicitGEMM),
testing::ValuesIn(GetConvTestCases(miopenHalf))));
INSTANTIATE_TEST_SUITE_P(Smoke,
GPU_UnitTestConvSolverHipImplicitGemmV4R1Fwd_BFP16,
testing::Combine(testing::Values(GetTestParamsBFP16()),
testing::Values(miopenConvolutionAlgoImplicitGEMM),
testing::ValuesIn(GetConvTestCases(miopenBFloat16))));

- "Full/GPU_MIOpenDriverConv2dTransTest*"
- "Full/GPU_MIOpenDriverRegressionBigTensorTest_FP32"
- "*GPU_TestMhaFind20_FP32*" #rocm-1016 fixed in develop. Re-enable once bumped
- "*GPU_TestMhaFind20_FP16*" #rocm-1016 fixed in develop. Re-enable once bumped
# Only exclude explicitly disabled tests
- "*DISABLED*"
Comment on lines 286 to 287

@randyspauldingamd randyspauldingamd Mar 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DISABLED filter should be removed--we shouldn't be automatically filtering DISABLED tests. Gtest provides the --gtest_also_run_disabled_tests flag for debugging purposes, but --gtest_filter takes precedence. This can cause false positives since it is easy to overlook this element inside the very long filter.

Example: unary_tensor_ops.cpp instantiates 10 suites, each having 55 tests, and two of the 10 are DISABLED:

Normal behavior:

miopen/build# bin/test_unary_tensor_ops
...
[==========] 440 tests from 10 test suites ran. (18395 ms total)
[  PASSED  ] 440 tests.

  YOU HAVE 110 DISABLED TESTS

2*55=110, so as expected.
Manually re-enabled:

miopen/build# bin/test_unary_tensor_ops  --gtest_also_run_disabled_tests
...
110 FAILED TESTS

Note: they are DISABLED for good reason :)
Filtered:

...miopen/build# bin/test_unary_tensor_ops --gtest_filter=-<many other filters>:*DISABLED*:<many other filters>
...
[==========] 440 tests from 10 test suites ran. (2558 ms total)
[  PASSED  ] 440 tests.

No mention of disabled tests.
Filtered and manually re-enabled:

miopen/build# bin/test_unary_tensor_ops --gtest_filter=-<many other filters>:*DISABLED*:<many other filters> --gtest_also_run_disabled_tests
...
[==========] 440 tests from 10 test suites ran. (1631 ms total)
[  PASSED  ] 440 tests.

Still no mention of the DISABLED tests, so the user may be tricked into thinking they ran and passed.


Expand Down
Loading