Repository navigation
test(hipdnn): trim ffm-quick: skip tests that cannot run on FFM, mirror quick for miopen - #14657
Merged
Merged
Conversation
…ases from ffm-quick The GPU SDPA reference recomputes QK^T per output element. On the FFM simulator the smallest quick SdpaFwd bundle took 329 s and the 2x32x2048x128 bundle had not finished after 40 minutes. referenceShapeIsAffordable() gates only the CPU reference, assuming the GPU reference is fast; that does not hold on FFM. Real-GPU lanes still run these cases and the CpuRef lane still validates the same bundles.
…filter suite StallGateExcludesHostSubmissionDelay asserts that a stalled start event leaves a 20 ms host delay out of the measured span. The FFM simulator does not honour the stall gate (112 ms stalled vs < 5 ms expected), so the test cannot pass there. test_categories.yaml labels whole CTest entries by name and hipdnn_backend_tests is one entry, so it could not drop one gtest. add_hipdnn_test() takes a GTEST_CATEGORIES flag that also generates <target>_<category>_suite entries from the new test_categories_gtest.yaml, in the build tree and in the installed CTestTestfile.cmake. hipdnn_backend_tests opts in for ffm-quick. The plain entry must not keep the ffm-quick label, or it still runs the excluded test. parse_ctest_categories.py gains a per-category exclude_tests key: matching tests receive none of that category's labels. ctest -L is a regex search, so hipdnn-quick-check (-L quick) also selected every ffm-quick test and would run hipdnn_backend_tests twice. The check targets now anchor the label, with VERBATIM so the anchors survive cmd.exe, where ^ is the escape character.
ffm-quick named six sweep-case ids, one per op. Those ids are renamed whenever a sweep is regenerated, and the FFM run only exercised three of them; the other three skipped. Select quick_* and Smoke/* instead, so FFM gets the same bundle coverage as the quick tier without a list to maintain. Drop the doc passages that described the exact-id list.
adickin-amd
changed the title
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
…test categories add_hipdnn_test(... GTEST_CATEGORIES) ended configure with FATAL_ERROR when shared/ctest was absent. The hipDNN Superbuild CI sparse checkout, and the standalone projects/hipdnn checkout that Building.md documents, have no shared/, and the rest of Tests.cmake already treats that as 'skip CTest categories'. Skip the gtest-filter suites the same way. A missing test_categories_gtest.yaml next to a present shared/ctest is a warning, because it drops the target out of the categories that file defines.
adickin-amd
commented
Oct 29, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #14657 +/- ##
===========================================
- Coverage 77.51% 77.40% -0.11%
===========================================
Files 2903 2902 -1
Lines 490383 488594 -1789
Branches 74218 73851 -367
===========================================
- Hits 380081 378158 -1923
- Misses 85389 85591 +202
+ Partials 24913 24845 -68
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
…a_ffm_quick_investigation
adickin-amd
marked this pull request as ready for review
November 3, 2026 15:20
mousdahl-amd
approved these changes
Nov 3, 2026
mousdahl-amd
left a comment
mousdahl-amd
left a comment
Contributor
There was a problem hiding this comment.
LGTM, after deep review analysis
Contributor
TheRock Submodule Bump ActivityNewest first
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep tests that cannot finish or pass on the FFM simulator out of the
ffm-quicktier. The quick-tier SDPA golden-data GPU-reference cases take minutes each on FFM (hours for the largest bundles), andStallGateExcludesHostSubmissionDelayasserts a timing property FFM does not provide. Real-GPU lanes are unchanged. Separately, the miopen-providerffm-quicktier now selects the same tests as its quick tier instead of a hand-kept list of sweep-case ids.JIRA ID : ROCM-33200
Risk Assessment
Risk level 2. Test selection and CTest registration only; no product code is touched. The one shared-code change is an additive, optional key in
shared/ctest/parse_ctest_categories.pythat no existing yaml uses. The check targets in hipDNN now match their label exactly.ASIC Coverage
No ASIC-specific behavior changes. PR CI is sufficient: every other tier and lane selects the same tests as before. Only
ffm-quickselects fewer tests. The FFM lane itself was not run for this change.Testing Summary
parse_test_categories.pyemits--gtest_filter=…:quick_SdpaFwd_*_GpuRef.*for theffm-quicksuite of the integration-tests binaries.hipdnn_backend_testsbuild):ctest -L '^ffm-quick$'selectshipdnn_backend_tests_ffm-quick_suiteand not the plainhipdnn_backend_tests. The suite ran 3678 tests and passed; the plain binary lists 3679, including the stall-gate test.CTestTestfile.cmake: the generated install file has the same split: plain entry without theffm-quicklabels, suite entry with the gtest filter and labelsffm-quick;ffm-full, timeout 7200.-L "^quick$"and-L "^ffm-quick$".test_parse_ctest_categories.py(3 tests) and the existingtest_parse_test_categories.py(42 tests) pass. Disabling the skip logic makes the build-tree test fail.Testing Checklist
python -m unittest discover -s shared/ctest/tests- Status: Passedhipdnn_backend_testsbuild, label selection and suite run -ctest -N -L '^ffm-quick$',ctest -R hipdnn_backend_tests_ffm-quick_suite- Status: Passedffm-quick- Status: Not runTechnical Changes
dnn-providers/integration-tests/test_categories.yaml:ffm-quickexcludesquick_SdpaFwd_*_GpuRef.*. The GPU SDPA reference recomputes QK^T per output element. On FFM the smallest quick bundle (1x8x256x128, 36 ms on MI300X) took 329 s, and the 2x32x2048x128 bundle had not finished after 40 minutes.referenceShapeIsAffordable()gates only the CPU reference, assuming the GPU reference is fast. Real-GPU lanes still run these cases and the CpuRef lane still validates the same bundles.projects/hipdnn/test_categories_gtest.yaml(new) andadd_hipdnn_test(... GTEST_CATEGORIES):test_categories.yamllabels whole CTest entries by name, so it cannot drop one gtest from a binary. A target that opts in also gets<target>_<category>_suiteentries with a--gtest_filter, generated in the build tree and in the installedCTestTestfile.cmake.hipdnn_backend_testsopts in forffm-quickand excludesTestGpuProfilingControlDescriptor.StallGateExcludesHostSubmissionDelay(stalled timing reported 100-112 ms against an expected < 5 ms on FFM).exclude_testsinshared/ctest/parse_ctest_categories.py: per-category key; matching tests receive none of that category's labels.hipdnn_backend_testsis listed underffm-quickso the plain entry does not also run the excluded test.ctest -Lis a regex search, sohipdnn-quick-check(-L quick) also selected everyffm-quicktest and would have runhipdnn_backend_teststwice. The label is now anchored, withVERBATIMso the anchors survivecmd.exe, where^is the escape character.dnn-providers/miopen-provider/test_categories_integration.yaml:ffm-quickselectsquick_*,Smoke/*andQuick/*, the same as the quick tier. It used to name six sweep-case ids, which are renamed whenever a sweep is regenerated, and the FFM run only exercised three of them. This enlarges the miopenffm-quickselection; its FFM run time has not been measured.projects/hipdnn/docs/TESTING.mddescribes the gtest-filter categories; thednn-providers/integration-tests/docspassages about the exact-id list are removed.