[MIOpen] Adding the rest of the tests to TheRock from MIOpen's gtest folder#1248
Merged
Conversation
…://github.com/ROCm/TheRock into users/Vsevolod1983/AddCPUAndConvAndOtherTests
Vsevolod1983
changed the title
Vsevolod1983
changed the title
|
nice work @Vsevolod1983 - how long do these tests now take to complete on the rock? |
JonathanLichtnerAMD
approved these changes
Sep 3, 2025
Contributor
Author
|
Member
|
Seeing MIOpen related build failures on Windows (gfx1151: https://therock-artifacts.s3.amazonaws.com/17458318068-windows/logs/gfx1151/MIOpen_build.log) but also other build / test failures (partly infra related). I re-triggered the failing jobs but in case the MIOpen issue persists, we might need to revert (@ScottTodd). |
Member
|
Seeing more failures building on gfx1151: https://github.com/ROCm/TheRock/actions/runs/17471481134/job/49620818900?pr=1349#step:11:36751. Do those look related to these changes? Should we revert? |
Member
90 minutes is too long for presubmit. Aim for 30. cc @amd-chrissosa @geomin12 |
Member
|
Proposing a revert: #1398 We're going to need to set up some separate scripts and modify our workflows to support different sets of tests on presubmit/postsubmit/nightly. I'm also noticing that we don't run MIOpen tests at all on Windows, even though we build them: TheRock/build_tools/github_actions/fetch_test_configurations.py Lines 98 to 105 in f9e44a8 |
Member
mi355 test time also increased from 1h7m to 1h54m |
ScottTodd
added a commit
that referenced
this pull request
Sep 4, 2025
…s gtest folder (#1398) This reverts commit 35444a3. See discussion at #1248 (comment). We suspect this is causing flaky build failures on Windows gfx1151 like https://github.com/ROCm/TheRock/actions/runs/17471481134/job/49620818900?pr=1349#step:11:36751. ``` [MIOpen] [894/920] Building CXX object test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj [MIOpen] FAILED: test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj [MIOpen] ccache B:\build\core\clr\dist\lib\llvm\bin\clang++.exe -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_NO_LIB -DBOOST_FILESYSTEM_NO_LIB -DBOOST_SYSTEM_NO_LIB -DHIP_COMPILER_FLAGS=" -x hip -D__HIP_PLATFORM_AMD__=1 -DUSE_PROF_API=1 C:/2E1C510A-F3EC-4287-AB5A-59025DAF1B15/build/core/clr/dist/lib/llvm/lib/clang/20/lib/windows/clang_rt.builtins-x86_64.lib --hip-link C:/2E1C510A-F3EC-4287-AB5A-59025DAF1B15/build/core/clr/dist/lib/llvm/lib/clang/20/lib/windows/clang_rt.builtins-x86_64.lib -fno-offload-uniform-block " -DMIOPEN_BETA_API=1 -DMIOPEN_BUILD_TESTING -DMIOPEN_TEST_DRIVER_MODE=1 -DNOMINMAX -DUSE_PROF_API=1 -D__HIP_PLATFORM_AMD__=1 -IB:/build/math-libs/BLAS/hipBLAS/stage/include -IB:/build/math-libs/BLAS/hipBLAS-common/stage/include -IB:/build/math-libs/rocRAND/stage/include -IC:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/.. -IC:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/../../src/kernels -IC:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/src/include -IB:/build/ml-libs/MIOpen/build/include -IC:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/include -isystem B:/build/base/half/stage/include -isystem B:/build/third-party/frugally-deep/dist/include -isystem B:/build/third-party/FunctionalPlus/dist/include -isystem B:/build/third-party/eigen/dist/include/eigen3 -isystem B:/build/third-party/nlohmann-json/dist/include -isystem B:/build/math-libs/BLAS/rocBLAS/dist/include -isystem B:/build/core/clr/dist/include -isystem B:/build/third-party/googletest/dist/include -isystem B:/build/compiler/amd-comgr/dist/include -isystem B:/build/third-party/boost/cmake_project/dist/include/boost-1_87 -isystem B:/build/third-party/sysdeps/windows/sqlite3/build/dist/lib/rocm_sysdeps/include -isystem B:/build/third-party/sysdeps/windows/bzip2/build/dist/lib/rocm_sysdeps/include -DWIN32 -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_WARNINGS -DNOMINMAX -fms-extensions -fms-compatibility -D_ENABLE_EXTENDED_ALIGNED_STORAGE -Wno-documentation-unknown-command -Wno-documentation-pedantic -Wno-unused-command-line-argument -Wno-explicit-specialization-storage-class -Wno-ignored-attributes -Wno-unknown-attributes -Wno-duplicate-decl-specifier --hip-path=B:/build/core/clr/dist --hip-device-lib-path=B:/build/core/clr/dist/lib/llvm/amdgcn/bitcode -O3 -DNDEBUG -std=c++20 -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -U__HCC__ -Wall -Wextra -Wcomment -Wendif-labels -Wformat -Winit-self -Wreturn-type -Wsequence-point -Wswitch -Wtrigraphs -Wundef -Wuninitialized -Wunreachable-code -Wunused -Wno-ignored-qualifiers -Wno-sign-compare -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-conversion -Wno-double-promotion -Wno-exit-time-destructors -Wno-extra-semi -Wno-extra-semi-stmt -Wno-float-conversion -Wno-gnu-anonymous-struct -Wno-gnu-zero-variadic-macro-arguments -Wno-missing-prototypes -Wno-nested-anon-types -Wno-option-ignored -Wno-padded -Wno-return-std-move-in-c++11 -Wno-shorten-64-to-32 -Wno-sign-conversion -Wno-unknown-warning-option -Wno-unused-command-line-argument -Wno-weak-vtables -Wno-covered-switch-default -Wno-unused-result -Wno-unsafe-buffer-usage -Wno-deprecated-declarations -Wno-shadow-uncaptured-local -Wno-global-constructors -Wno-reserved-identifier -Wno-zero-as-null-pointer-constant -Wno-ignored-attributes -Wno-deprecated -Wno-incompatible-pointer-types -Wno-old-style-cast -Wno-unknown-attributes -Wno-microsoft-cpp-macro -Wno-microsoft-enum-value -Wno-language-extension-token -Wno-c++11-narrowing -Wno-float-equal -Wno-redundant-parens -Wno-format-nonliteral -Wno-unused-template -Wno-comma -Wno-suggest-destructor-override -Wno-switch-enum -Wno-shift-sign-overflow -Wno-suggest-override -Wno-inconsistent-missing-destructor-override -Wno-cast-function-type -Wno-nonportable-system-include-path -Wno-documentation -Wno-deprecated-builtins -Wno-enum-constexpr-conversion -Wno-unused-value -Wno-unused-parameter -Wno-missing-noreturn -Wno-tautological-constant-out-of-range-compare -Wno-c++20-extensions -Wno-unique-object-duplication -Wno-switch-default -Wno-nontrivial-memcall -fms-extensions -fms-compatibility -Wno-undef -U__LP64__ -x hip --offload-arch=gfx1151 -MD -MT test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj -MF test\gtest\CMakeFiles\miopen_gtest.dir\smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj.d -o test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj -c C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp [MIOpen] PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [MIOpen] Stack dump: [MIOpen] 0. Program arguments: C:\\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\\build\\core\\clr\\dist\\lib\\llvm\\bin\\clang++.exe -cc1 -triple x86_64-pc-windows-msvc19.44.35215 -aux-triple amdgcn-amd-amdhsa -emit-obj -mincremental-linker-compatible -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -relaxed-aliasing -fmath-errno -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -fdebug-compilation-dir=B:\\build\\ml-libs\\MIOpen\\build -fcoverage-compilation-dir=B:\\build\\ml-libs\\MIOpen\\build -resource-dir C:\\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\\build\\core\\clr\\dist\\lib\\llvm\\lib\\clang\\20 -dependency-file test\\gtest\\CMakeFiles\\miopen_gtest.dir\\smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj.d -MT test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj -sys-header-deps -internal-isystem C:\\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\\build\\core\\clr\\dist\\lib\\llvm\\lib\\clang\\20\\include\\cuda_wrappers -idirafter B:/build/core/clr/dist\\include -include __clang_hip_runtime_wrapper.h -isystem B:/build/base/half/stage/include -isystem B:/build/third-party/frugally-deep/dist/include -isystem B:/build/third-party/FunctionalPlus/dist/include -isystem B:/build/third-party/eigen/dist/include/eigen3 -isystem B:/build/third-party/nlohmann-json/dist/include -isystem B:/build/math-libs/BLAS/rocBLAS/dist/include -isystem B:/build/core/clr/dist/include -isystem B:/build/third-party/googletest/dist/include -isystem B:/build/compiler/amd-comgr/dist/include -isystem B:/build/third-party/boost/cmake_project/dist/include/boost-1_87 -isystem B:/build/third-party/sysdeps/windows/sqlite3/build/dist/lib/rocm_sysdeps/include -isystem B:/build/third-party/sysdeps/windows/bzip2/build/dist/lib/rocm_sysdeps/include -D BOOST_ALL_NO_LIB=1 -D BOOST_ATOMIC_NO_LIB -D BOOST_FILESYSTEM_NO_LIB -D BOOST_SYSTEM_NO_LIB -D "HIP_COMPILER_FLAGS= -x hip -D__HIP_PLATFORM_AMD__=1 -DUSE_PROF_API=1 C:/2E1C510A-F3EC-4287-AB5A-59025DAF1B15/build/core/clr/dist/lib/llvm/lib/clang/20/lib/windows/clang_rt.builtins-x86_64.lib --hip-link C:/2E1C510A-F3EC-4287-AB5A-59025DAF1B15/build/core/clr/dist/lib/llvm/lib/clang/20/lib/windows/clang_rt.builtins-x86_64.lib -fno-offload-uniform-block " -D MIOPEN_BETA_API=1 -D MIOPEN_BUILD_TESTING -D MIOPEN_TEST_DRIVER_MODE=1 -D NOMINMAX -D USE_PROF_API=1 -D __HIP_PLATFORM_AMD__=1 -I B:/build/math-libs/BLAS/hipBLAS/stage/include -I B:/build/math-libs/BLAS/hipBLAS-common/stage/include -I B:/build/math-libs/rocRAND/stage/include -I C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/.. -I C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/../../src/kernels -I C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/src/include -I B:/build/ml-libs/MIOpen/build/include -I C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/include -D WIN32 -D WIN32_LEAN_AND_MEAN -D _CRT_SECURE_NO_WARNINGS -D NOMINMAX -D _ENABLE_EXTENDED_ALIGNED_STORAGE -D NDEBUG -D _DLL -D _MT -U __HCC__ -U __LP64__ -internal-isystem C:\\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\\build\\core\\clr\\dist\\lib\\llvm\\lib\\clang\\20\\include -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.26100.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.26100.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -internal-isystem C:\\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\\build\\core\\clr\\dist\\lib\\llvm\\lib\\clang\\20\\include -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.26100.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\ATLMFC\\include" -internal-isystem "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\VS\\include" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.26100.0\\ucrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\um" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\shared" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\winrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.26100.0\\\\cppwinrt" -internal-isystem "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um" -O3 -Wno-documentation-unknown-command -Wno-documentation-pedantic -Wno-unused-command-line-argument -Wno-explicit-specialization-storage-class -Wno-ignored-attributes -Wno-unknown-attributes -Wno-duplicate-decl-specifier -Wall -Wextra -Wcomment -Wendif-labels -Wformat -Winit-self -Wreturn-type -Wsequence-point -Wswitch -Wtrigraphs -Wundef -Wuninitialized -Wunreachable-code -Wunused -Wno-ignored-qualifiers -Wno-sign-compare -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-conversion -Wno-double-promotion -Wno-exit-time-destructors -Wno-extra-semi -Wno-extra-semi-stmt -Wno-float-conversion -Wno-gnu-anonymous-struct -Wno-gnu-zero-variadic-macro-arguments -Wno-missing-prototypes -Wno-nested-anon-types -Wno-option-ignored -Wno-padded -Wno-return-std-move-in-c++11 -Wno-shorten-64-to-32 -Wno-sign-conversion -Wno-unknown-warning-option -Wno-unused-command-line-argument -Wno-weak-vtables -Wno-covered-switch-default -Wno-unused-result -Wno-unsafe-buffer-usage -Wno-deprecated-declarations -Wno-shadow-uncaptured-local -Wno-global-constructors -Wno-reserved-identifier -Wno-zero-as-null-pointer-constant -Wno-ignored-attributes -Wno-deprecated -Wno-incompatible-pointer-types -Wno-old-style-cast -Wno-unknown-attributes -Wno-microsoft-cpp-macro -Wno-microsoft-enum-value -Wno-language-extension-token -Wno-c++11-narrowing -Wno-float-equal -Wno-redundant-parens -Wno-format-nonliteral -Wno-unused-template -Wno-comma -Wno-suggest-destructor-override -Wno-switch-enum -Wno-shift-sign-overflow -Wno-suggest-override -Wno-inconsistent-missing-destructor-override -Wno-cast-function-type -Wno-nonportable-system-include-path -Wno-documentation -Wno-deprecated-builtins -Wno-enum-constexpr-conversion -Wno-unused-value -Wno-unused-parameter -Wno-missing-noreturn -Wno-tautological-constant-out-of-range-compare -Wno-c++20-extensions -Wno-unique-object-duplication -Wno-switch-default -Wno-nontrivial-memcall -Wno-undef -std=c++20 -ferror-limit 19 -fhip-new-launch-api -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.44.35215 -fno-implicit-modules -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -fcolor-diagnostics -vectorize-loops -vectorize-slp --dependent-lib=msvcrt -fcuda-include-gpubinary C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16-809b30.hipfb -cuid=bf2425f0600af3e8 -fcuda-allow-variadic-functions -faddrsig -o test/gtest/CMakeFiles/miopen_gtest.dir/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp.obj -x hip C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest/smoke_solver_ConvAsmImplicitGemmGTCDynamicXdlopsNHWC_bf16.cpp [MIOpen] 1. <eof> parser at end of file [MIOpen] 2. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\gtest_common.hpp:279:6: instantiating function definition 'invoke_with_params<conv2d_driver, GPU_Conv2dTuning_BFP16, void (&)(const std::basic_string<char> &)>' [MIOpen] 3. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:1356:6: instantiating function definition 'test_drive<conv2d_driver>' [MIOpen] 4. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:1337:6: instantiating function definition 'test_drive_impl<conv2d_driver<double>>' [MIOpen] 5. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:1233:6: instantiating function definition 'test_drive_impl_1<conv2d_driver<double>>' [MIOpen] 6. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:938:10: instantiating function definition 'test_driver::base_run<conv2d_driver<double>>' [MIOpen] 7. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\..\conv_common.hpp:1962:10: instantiating function definition 'conv_driver<double>::run' [MIOpen] 8. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:910:10: instantiating function definition 'test_driver::verify<verify_backward_weights_conv<ConvApi::Find_1_0, double>>' [MIOpen] 9. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:798:10: instantiating function definition 'test_driver::verify_impl<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\driver.hpp:913:13), verify_backward_weights_conv<ConvApi::Find_1_0, double> &>' [MIOpen] 10. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\../driver.hpp:746:10: instantiating function definition 'test_driver::run_cpu<verify_backward_weights_conv<ConvApi::Find_1_0, double>>' [MIOpen] 11. C:/home/runner/_work/TheRock/TheRock/rocm-libraries/projects/miopen/test/gtest\..\ford.hpp:56:6: instantiating function definition 'then<tensor<double>, (lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\driver.hpp:768:46)>' [MIOpen] 12. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:1362:81: instantiating function definition 'std::async<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>' [MIOpen] 13. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:1350:41: instantiating function definition 'std::_Get_associated_state<tensor<double>, std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>>' [MIOpen] 14. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:597:5: instantiating function definition 'std::_Deferred_async_state<tensor<double>>::_Deferred_async_state<std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>>' [MIOpen] 15. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\type_traits:53:16: instantiating variable definition 'std::conjunction_v<std::negation<std::is_same<std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>, std::function<tensor<double> ()>>>, std::_Is_invocable_r<tensor<double>, std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)> &>>' [MIOpen] 16. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\type_traits:45:8: instantiating class definition 'std::conjunction<std::negation<std::is_same<std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>, std::function<tensor<double> ()>>>, std::_Is_invocable_r<tensor<double>, std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)> &>>' [MIOpen] 17. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\type_traits:35:8: instantiating class definition 'std::_Conjunction<true, std::negation<std::is_same<std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>, std::function<tensor<double> ()>>>, std::_Is_invocable_r<tensor<double>, std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)> &>>' [MIOpen] 18. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\type_traits:1827:8: instantiating class definition 'std::_Is_invocable_r<tensor<double>, std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)> &>' [MIOpen] 19. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:1341:20: instantiating function definition 'std::_Fake_no_copy_callable_adapter<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>::operator()' [MIOpen] 20. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:1314:16: instantiating function definition 'std::_Invoke_stored<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23)>' [MIOpen] 21. C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\include\future:1308:16: instantiating function definition 'std::_Invoke_stored_explicit<(lambda at C:\home\runner\_work\TheRock\TheRock\rocm-libraries\projects\miopen\test\gtest\..\ford.hpp:59:23), 0ULL>' [MIOpen] Exception Code: 0xC0000005 [MIOpen] #0 0x00007ff64a1882be (C:\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\build\core\clr\dist\lib\llvm\bin\clang++.exe+0x15082be) [MIOpen] #1 0x00007ff64bb9af20 (C:\2E1C510A-F3EC-4287-AB5A-59025DAF1B15\build\core\clr\dist\lib\llvm\bin\clang++.exe+0x2f1af20) ``` It also increased MIOpen test times substantially: Before: https://github.com/ROCm/TheRock/actions/runs/17447546026 * Linux mi325 50m * Linux mi355 1h7m After: https://github.com/ROCm/TheRock/actions/runs/17458318068 * Linux mi325 1h30m * Linux mi355 1h54m (very close to a 2 hour timeout)
Contributor
Author
Interesting, there were no failures shown on PR main page when I merged. |
Member
|
Continuing to see more flaky compiler crashes even after reverting 🤔 filed #1407. I wonder if adding more tests just increased the frequency of the crashes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A lor of tests were added.
However, some of them are failing, so I had to disable them:
negative_filter.append("/GPU_KernelTuningNetTest")
negative_filter.append("DBSync")
negative_filter.append("/GPU_MIOpenDriver")