Fuse GEMM+GEMM with rocMLIR #4261

bdevorem · 2025-08-26T22:33:17Z

Motivation

rocmlir supports GEMM+GEMM fusion, so MIGraphX needs to support the fusion on our side.
Solves #4230

Technical Details

Update:
The g+g match-rewrite pass is now implemented as its own discrete matcher, not a fusion on top of the G->optional reshape->E fusion. It handles the two potential cases of intermediate results needing to be returned from the fused module.

Currently, it is enabled with a user-specified switch (env var). When enabled, it takes priority over G->optional reshape->E. Work is ongoing to turn the enablement into a heuristic-based decision.

This match-rewrite pass is a modular matcher, built off of the G->optional reshape->E pass. In order to match an already-fused module, I have added a tag member to mlir_op that only find_mlir_fused_ops (G->optional reshape->E) makes use of currently. This tag is defaulted to empty string, but all tests that utilize that matcher were required to be updated to show the empty tag.

~~There are two potential multi-out scenarios from the fused G->E module this new rewriter matches on:~~
~~1. the module returns a single value, but it is used by the final GEMM instruction and another op outside of the new fusion~~
~~2. the module returns multiple values, i.e. a tuple that gets extracted by get_tuple_elem ops for each return value~~

As a result, this rewriter needs to handle both multi output scenarios as well as the single output scenario. For both multi-out scenarios, the new fused module needs to propagate every intermediate value that is used outside of the new module.

Testing

Unit tests

dot_add_dot : simple single-output test case
dot_mul_dot : simple single-output test case, different pointwise op
conv_add : simple single-output conv test case for G->optional reshape->E, since we lacked one
conv_add_dot : a test case building off of the previous that utilizes G+G, to make sure conv is supported as first op (rocMLIR supports this case)
dot_multi_user_add : tests the scenario for G->optional reshape->E where the intermediate G result has to be outputted by the fusion, i.e. the intermediate result is used by an op outside of the fusion
dot_add_dot_multi_user : tests the scenario of multi-out where the first gemm has a user outside of the fusion.
dot_add_multi_user_dot : tests the scenario of multi-out where the elementwise op has a user outside of the fusion.

Verify tests

test/verify/test_conv_add_dot.cpp
test/verify/test_dot_add_dot.cpp

Considerations

- this prioritizes G-optional reshape-E fusion ahead of GEG- specifically this means that a G->E->G->E in the graph is currently gonna result in two G-optional reshape-E fusions. I plan on benchmarking how that performs against a G+G kernel followed by a standalone pointwise

benchmarking to decide the order of fusion matches has been done; heuristic will be added in later PR

Upcoming PRs

heuristic for taking this fusion path vs not
making fusion between gemm0 and gemm1 optional
~~mlir tests; these can not be added until the rocmlir commit we build off of has been updated past rocmlir#1960.~~

codecov · 2025-08-26T23:52:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4261   +/-   ##
========================================
  Coverage    92.24%   92.25%           
========================================
  Files          560      560           
  Lines        26302    26310    +8     
========================================
+ Hits         24262    24270    +8     
  Misses        2040     2040

Files with missing lines	Coverage Δ
src/include/migraphx/module.hpp	`100.00% <ø> (ø)`
src/module.cpp	`87.12% <100.00%> (+0.13%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull Request Overview

This PR implements GEMM+GEMM fusion support in MIGraphX to leverage rocMLIR's existing fusion capabilities, addressing issue #4230. The implementation adds a new match-rewrite pass that identifies and fuses sequences of GEMM operations with optional intermediate elementwise operations.

Introduces find_mlir_fused_geg_ops pass to match and fuse GEMM+GEMM patterns
Adds a tag member to mlir_op to identify previously fused operations
Handles multiple output scenarios including single outputs with multiple users and tuple outputs

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
test/gpu/fuse_mlir.cpp	Adds comprehensive test cases covering various GEMM+GEMM fusion scenarios including multi-user cases
src/targets/gpu/fuse_mlir.cpp	Implements the core GEMM+GEMM fusion logic with the new matcher and rewriter
docs/reference/MIGraphX-dev-env-vars.rst	Documents the new environment variable to control GEMM+GEMM fusion
Jenkinsfile	Enables the new fusion feature in the MLIR debug test pipeline

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/targets/gpu/fuse_mlir.cpp

bdevorem · 2025-09-03T03:41:13Z

looking into the CI failures...

Copilot

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/targets/gpu/fuse_mlir.cpp

bdevorem · 2025-09-19T18:06:20Z

Update:
~~there is one failing MLIR CI test, GRU bidirectional, coming from rocMLIR. There is an open ticket on rocMLIR-internal currently, which is the only blocker left for this PR~~

This reverts commit ba57d66.

…ve the first instr in the fused mod

Copilot

Pull Request Overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

pfultz2 · 2025-09-24T15:12:57Z

src/targets/gpu/fuse_mlir.cpp

+        // However, for the single-out case, it is possible for there to be unresolved dependencies
+        // if we place the fused mod at the first instruction, because in some archs, the inputs
+        // of the intermediates may be located in the IR in between the fused ops. As a result, for
+        // the single-out case, we insert the fused op at the last instruction.


We run into a similar(but much simpler) scenario for the gemm/conv fusion and we move the instructions but that is much easier to do for that case. In the future we may need to do this, I am thinking the find_instructions_between might help with this.

Ah, ok. I didn't see that function.

There were kind of 3 paths to handle this that I saw. First, to conditionally place the fused module either at the beginning or end, which I ended up doing. Second, place the fused module at the end, and then handle moving any op that happens to be in the IR before that location that uses a return val of the fused module (specifically, an intermediate return val in the multi-out fused module case). Third, place the fused module at the beginning, and then need to handle moving any external inputs of the fused module that happen to be defined after the first gemm. It seemed like, out of the latter two options, that the second might be the least gnarly, but gnarly regardless. Sounds like find_instructions_between might solve that

Worst case, break that off as a separate issue and tackle it in another PR. Overall for the current cases this appears to work.

src/targets/gpu/fuse_mlir.cpp

test/gpu/fuse_mlir.cpp

migraphx-bot · 2025-10-02T08:46:26Z

Test	Batch	Rate new f43e0a	Rate old 3a688d	Diff	Compare
torchvision-resnet50	64	3,158.81	3,161.63	-0.09%	✅
torchvision-resnet50_fp16	64	6,590.61	6,617.50	-0.41%	✅
torchvision-densenet121	32	2,438.45	2,448.77	-0.42%	✅
torchvision-densenet121_fp16	32	4,125.80	4,138.16	-0.30%	✅
torchvision-inceptionv3	32	1,665.55	1,673.22	-0.46%	✅
torchvision-inceptionv3_fp16	32	2,586.55	2,596.16	-0.37%	✅
cadene-inceptionv4	16	794.02	798.23	-0.53%	✅
cadene-resnext64x4	16	802.76	806.79	-0.50%	✅
slim-mobilenet	64	8,210.76	8,228.75	-0.22%	✅
slim-nasnetalarge	64	221.76	222.90	-0.51%	✅
slim-resnet50v2	64	3,294.49	3,308.78	-0.43%	✅
bert-mrpc-onnx	8	1,133.57	1,144.23	-0.93%	✅
bert-mrpc-tf	1	489.00	488.62	0.08%	✅
pytorch-examples-wlang-gru	1	314.82	310.31	1.45%	✅
pytorch-examples-wlang-lstm	1	434.03	516.61	-15.98%	🔴
torchvision-resnet50_1	1	809.94	805.19	0.59%	✅
cadene-dpn92_1	1	423.54	429.66	-1.42%	✅
cadene-resnext101_1	1	368.37	369.71	-0.36%	✅
onnx-taau-downsample	1	398.07	399.64	-0.39%	✅
dlrm-criteoterabyte	1	31.93	32.05	-0.38%	✅
dlrm-criteoterabyte_fp16	1	51.03	51.07	-0.08%	✅
agentmodel	1	9,912.57	10,121.66	-2.07%	✅
unet_fp16	2	57.57	57.79	-0.38%	✅
resnet50v1_fp16	1	989.09	1,048.13	-5.63%	🔴
resnet50v1_int8	1	993.43	992.53	0.09%	✅
bert_base_cased_fp16	64	1,099.39	1,104.77	-0.49%	✅
bert_large_uncased_fp16	32	343.80	345.66	-0.54%	✅
bert_large_fp16	1	197.82	198.35	-0.27%	✅
distilgpt2_fp16	16	2,093.14	2,107.64	-0.69%	✅
yolov5s	1	582.20	587.88	-0.97%	✅
tinyllama	1	43.80	43.97	-0.39%	✅
vicuna-fastchat	1	44.88	45.19	-0.69%	✅
whisper-tiny-encoder	1	409.94	411.14	-0.29%	✅
whisper-tiny-decoder	1	413.23	414.17	-0.23%	✅
llama2_7b	1	19.06	19.08	-0.11%	✅
qwen1.5-7b	1	23.43	23.54	-0.45%	✅
phi3-3.8b	1	26.58	26.68	-0.39%	✅
mask-rcnn	1	12.36	12.09	2.26%	✅
llama3-8b	1	21.65	21.74	-0.40%	✅
whisper-large-encoder	1	10.17	10.22	-0.48%	✅
whisper-large-decoder	1	99.50	99.48	0.02%	✅
mistral-7b	1	23.68	23.74	-0.26%	✅
FLUX.1-schnell	1	728.00	709.17	2.65%	✅
nan	nan	nan	nan	nan%	❌

This build is not recommended to merge 🔴

migraphx-bot · 2025-10-02T08:46:29Z

✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

✅ agentmodel: PASSED: MIGraphX meets tolerance

✅ unet: PASSED: MIGraphX meets tolerance

✅ resnet50v1: PASSED: MIGraphX meets tolerance

❌bert_base_cased_fp16: ERROR - check error output

�[1;31m2025-10-02 02:53:06.558477396 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184754, index: 64, mask: {65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.567418410 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184758, index: 68, mask: {69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.567450570 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184766, index: 76, mask: {77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568042876 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184757, index: 67, mask: {68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568067703 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184763, index: 73, mask: {74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568076289 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184791, index: 101, mask: {102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568087450 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184772, index: 82, mask: {83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.567418430 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184759, index: 69, mask: {70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568224007 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184764, index: 74, mask: {75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.569828480 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184761, index: 71, mask: {72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572044464 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184760, index: 70, mask: {71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572056537 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184815, index: 125, mask: {126, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568060119 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184770, index: 80, mask: {81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568060139 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184782, index: 92, mask: {93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568063435 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184783, index: 93, mask: {94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568065058 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184779, index: 89, mask: {90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568066240 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184781, index: 91, mask: {92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568075277 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184762, index: 72, mask: {73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568062393 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184785, index: 95, mask: {96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568072873 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184787, index: 97, mask: {98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568046182 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184753, index: 63, mask: {64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.576043038 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184801, index: 111, mask: {112, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.580029066 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184765, index: 75, mask: {76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568096677 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184790, index: 100, mask: {101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568097208 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184780, index: 90, mask: {91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568100204 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184792, index: 102, mask: {103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568100875 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184788, index: 98, mask: {99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572047400 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184809, index: 119, mask: {120, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572050826 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184800, index: 110, mask: {111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572053211 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184805, index: 115, mask: {116, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572047640 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184767, index: 77, mask: {78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572060534 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184812, index: 122, mask: {123, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568060169 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184796, index: 106, mask: {107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572066155 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184794, index: 104, mask: {105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572066015 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184808, index: 118, mask: {119, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572065444 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184769, index: 79, mask: {80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572065424 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184786, index: 96, mask: {97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572072547 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184814, index: 124, mask: {125, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572072247 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184804, index: 114, mask: {115, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572072146 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184807, index: 117, mask: {118, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572073018 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184775, index: 85, mask: {86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572072868 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184773, index: 83, mask: {84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572065464 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184795, index: 105, mask: {106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572077757 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184813, index: 123, mask: {124, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572075122 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184776, index: 86, mask: {87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572075232 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184771, index: 81, mask: {82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572080071 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184774, index: 84, mask: {85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572080843 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184810, index: 120, mask: {121, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572080382 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184793, index: 103, mask: {104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.572086073 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184768, index: 78, mask: {79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.576036956 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184816, index: 126, mask: {127, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.580029156 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184797, index: 107, mask: {108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.583749345 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184806, index: 116, mask: {117, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568048477 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184798, index: 108, mask: {109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568094103 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184789, index: 99, mask: {100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.584028050 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184778, index: 88, mask: {89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.584030184 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184803, index: 113, mask: {114, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568049348 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184784, index: 94, mask: {95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568044349 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184756, index: 66, mask: {67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.584033471 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184799, index: 109, mask: {110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.584034302 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184811, index: 121, mask: {122, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.584032489 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184802, index: 112, mask: {113, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.568049759 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184755, index: 65, mask: {66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:06.588032824 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 184777, index: 87, mask: {88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:53:07.897142674 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_1623for node: Mul_53/SimplifiedLayerNormFusion/
�[m
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 359, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 278, in main
sess = ort.InferenceSession(model_name,
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 473, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 572, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_1623for node: Mul_53/SimplifiedLayerNormFusion/

❌bert_large_uncased_fp16: ERROR - check error output

�[1;31m2025-10-02 02:54:19.159302175 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191739, index: 63, mask: {64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168043192 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191740, index: 64, mask: {65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172046053 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191755, index: 79, mask: {80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172056753 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191745, index: 69, mask: {70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168053422 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191750, index: 74, mask: {75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168049304 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191747, index: 71, mask: {72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.176045287 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191758, index: 82, mask: {83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168076585 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191742, index: 66, mask: {67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168077196 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191751, index: 75, mask: {76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168078379 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191749, index: 73, mask: {74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168087005 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191743, index: 67, mask: {68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172049630 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191760, index: 84, mask: {85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172046934 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191759, index: 83, mask: {84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172046965 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191752, index: 76, mask: {77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172052685 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191757, index: 81, mask: {82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172050521 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191753, index: 77, mask: {78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172057995 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191756, index: 80, mask: {81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172058947 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191746, index: 70, mask: {71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.172061742 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191754, index: 78, mask: {79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185358292 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191782, index: 106, mask: {107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185397676 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191788, index: 112, mask: {113, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185479260 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191801, index: 125, mask: {126, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185504277 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191794, index: 118, mask: {119, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168049795 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191744, index: 68, mask: {69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.180030405 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191748, index: 72, mask: {73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185510088 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191768, index: 92, mask: {93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185639071 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191800, index: 124, mask: {125, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185852634 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191772, index: 96, mask: {97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185871499 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191779, index: 103, mask: {104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185906766 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191786, index: 110, mask: {111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185917536 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191793, index: 117, mask: {118, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.185942613 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191765, index: 89, mask: {90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186200799 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191792, index: 116, mask: {117, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186226608 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191778, index: 102, mask: {103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186241145 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191784, index: 108, mask: {109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186227059 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191777, index: 101, mask: {102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186277384 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191763, index: 87, mask: {88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186286802 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191764, index: 88, mask: {89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186304785 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191790, index: 114, mask: {115, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186336635 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191799, index: 123, mask: {124, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186356002 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191770, index: 94, mask: {95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186339030 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191785, index: 109, mask: {110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186383784 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191769, index: 93, mask: {94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186320996 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191797, index: 121, mask: {122, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186304365 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191771, index: 95, mask: {96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186451852 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191762, index: 86, mask: {87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186510503 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191776, index: 100, mask: {101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186561449 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191783, index: 107, mask: {108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186650577 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191802, index: 126, mask: {127, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186644155 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191795, index: 119, mask: {120, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186850423 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191798, index: 122, mask: {123, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.186893505 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191791, index: 115, mask: {116, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187145068 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191796, index: 120, mask: {121, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187199371 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191767, index: 91, mask: {92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187216122 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191774, index: 98, mask: {99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187264143 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191781, index: 105, mask: {106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187284611 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191789, index: 113, mask: {114, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187562555 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191766, index: 90, mask: {91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187576541 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191787, index: 111, mask: {112, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187647235 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191773, index: 97, mask: {98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187649439 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191761, index: 85, mask: {86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.187834778 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191775, index: 99, mask: {100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.188513877 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191780, index: 104, mask: {105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:19.168050847 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 191741, index: 65, mask: {66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 02:54:22.764617523 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_3183for node: Mul_53/SimplifiedLayerNormFusion/
�[m
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 359, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 278, in main
sess = ort.InferenceSession(model_name,
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 473, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 572, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_3183for node: Mul_53/SimplifiedLayerNormFusion/

✅ bert_large: PASSED: MIGraphX meets tolerance

✅ yolov5s: PASSED: MIGraphX meets tolerance

✅ tinyllama: PASSED: MIGraphX meets tolerance

✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

❌distilgpt2_fp16: ERROR - check error output

�[1;31m2025-10-02 03:00:06.670394847 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207743, index: 63, mask: {64, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684076993 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207766, index: 86, mask: {87, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680043895 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207748, index: 68, mask: {69, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680047842 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207783, index: 103, mask: {104, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680066407 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207778, index: 98, mask: {99, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684050874 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207764, index: 84, mask: {85, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684058248 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207759, index: 79, mask: {80, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680049034 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207754, index: 74, mask: {75, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684071913 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207789, index: 109, mask: {110, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684078886 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207779, index: 99, mask: {100, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684071803 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207776, index: 96, mask: {97, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680043574 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207772, index: 92, mask: {93, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680065255 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207753, index: 73, mask: {74, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680053242 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207773, index: 93, mask: {94, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680065255 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207752, index: 72, mask: {73, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680064694 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207744, index: 64, mask: {65, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680063532 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207755, index: 75, mask: {76, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680074212 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207782, index: 102, mask: {103, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680050367 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207774, index: 94, mask: {95, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.696624491 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207803, index: 123, mask: {124, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684044522 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207757, index: 77, mask: {78, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680047912 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207746, index: 66, mask: {67, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684052387 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207771, index: 91, mask: {92, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680074232 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207781, index: 101, mask: {102, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680069683 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207791, index: 111, mask: {112, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684052457 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207787, index: 107, mask: {108, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680047862 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207745, index: 65, mask: {66, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684056134 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207763, index: 83, mask: {84, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684056594 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207758, index: 78, mask: {79, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684058919 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207750, index: 70, mask: {71, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684056434 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207790, index: 110, mask: {111, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684052647 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207762, index: 82, mask: {83, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684059720 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207765, index: 85, mask: {86, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684055933 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207777, index: 97, mask: {98, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680050086 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207751, index: 71, mask: {72, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.688118767 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207784, index: 104, mask: {105, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684073887 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207786, index: 106, mask: {107, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684073937 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207767, index: 87, mask: {88, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.688120690 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207760, index: 80, mask: {81, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.688123335 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207749, index: 69, mask: {70, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684071152 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207756, index: 76, mask: {77, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697416744 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207806, index: 126, mask: {127, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697474042 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207794, index: 114, mask: {115, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697502495 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207800, index: 120, mask: {121, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697764198 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207793, index: 113, mask: {114, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697786130 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207799, index: 119, mask: {120, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697810716 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207805, index: 125, mask: {126, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.698014891 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207792, index: 112, mask: {113, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.698030019 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207798, index: 118, mask: {119, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.698132582 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207804, index: 124, mask: {125, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.698473665 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207796, index: 116, mask: {117, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.698499123 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207802, index: 122, mask: {123, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680079131 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207780, index: 100, mask: {101, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684077414 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207769, index: 89, mask: {90, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684079127 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207770, index: 90, mask: {91, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684078526 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207788, index: 108, mask: {109, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684091711 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207785, index: 105, mask: {106, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684073867 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207761, index: 81, mask: {82, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680061888 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207747, index: 67, mask: {68, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.680049515 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207775, index: 95, mask: {96, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.684078315 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207768, index: 88, mask: {89, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.696995120 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207795, index: 115, mask: {116, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.697021459 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207801, index: 121, mask: {122, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:06.696771268 [E:onnxruntime:Default, env.cc:226 ThreadMain] pthread_setaffinity_np failed for thread: 207797, index: 117, mask: {118, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2025-10-02 03:00:07.528702985 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_1551for node: Mul_33/SimplifiedLayerNormFusion/
�[m
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 359, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 278, in main
sess = ort.InferenceSession(model_name,
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 473, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 572, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_onnx::Pow_1551for node: Mul_33/SimplifiedLayerNormFusion/

✅ llama2_7b: PASSED: MIGraphX meets tolerance

✅ qwen1.5-7b: PASSED: MIGraphX meets tolerance

✅ phi3-3.8b: PASSED: MIGraphX meets tolerance

🔴mask-rcnn: FAILED: MIGraphX is not within tolerance - check verbose output

✅ llama3-8b: PASSED: MIGraphX meets tolerance

✅ whisper-large-decoder: PASSED: MIGraphX meets tolerance

✅ mistral-7b: PASSED: MIGraphX meets tolerance

✅ FLUX.1-schnell: PASSED: MIGraphX meets tolerance

rocmlir supports GEMM+GEMM fusion, so MIGraphX needs to support the fusion on our side. Solves #4230

* Update build and test script and dockerfile to add in onnxrt pai laun… (#4336) Update build and test script and dockerfile to add in onnxrt pai launcher scripts Push pai launcher scripts to Onnxrutime to let us reuse scripts from prior run Related to change from - #4321 * Bump SQlite3 to 3.50.4 (#4322) * TopK exception bugfix (#4329) * Use --input-dim for specifying dynamic shapes at driver runtime (#4342) * Add video and render groups in docker for CI (#4340) * Fix MXFP4 bugs (#4324) * propagate_constant ignores `unpack_fp4` instructions now * `match_find_mx_quantizable_ops` from `simplify_qdq` updated to not require non-constant scales. The scales can be literals. * transpose, reshape, and broadcast instructions propagated on scale instructions when going to `quant_dot` and `quant_conv` * `raw_data` `operator<<` updated to use `fallback_visit` to also handle non-computable types * Brevitas MXFP4 quantization parse (#4301) * Bump onnx to 1.18.0 (#4323) * Fuse GEMM+GEMM with rocMLIR (#4261) rocmlir supports GEMM+GEMM fusion, so MIGraphX needs to support the fusion on our side. Solves #4230 * Updated SD3 example for change in optimum-onnx[onnxruntime] (#4344) * Lower lrn to pooling (#4294) This PR implements a lowering transformation that converts LRN operations into a series of pooling and arithmetic operations. * Update onnxruntime main be655f69f6c8623eebcac094626d0c8545951e6d (#4348) * Update docs/reference/MIGraphX-dev-env-vars.rst --------- Co-authored-by: Ted Themistokleous <[email protected]> Co-authored-by: Lakhinder Walia <[email protected]> Co-authored-by: kahmed10 <[email protected]> Co-authored-by: Charlie Lin <[email protected]> Co-authored-by: Breanna Devore-McDonald <[email protected]> Co-authored-by: Aarushi Jain <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This was referenced Aug 26, 2025

Refactor and fix creation of ElementWise Region for Gemm+Gemm like ops ROCm/rocMLIR#1960

Merged

[Issue]: GEMM+GEMM fusion fails to compile with assertion failure ROCm/rocMLIR#1961

Closed

bdevorem force-pushed the bdevorem/fuse-geg branch from df3bbbc to 080479b Compare August 29, 2025 07:04

bdevorem changed the title ~~[DRAFT] Bdevorem/fuse geg~~ Fuse GEMM+GEMM with rocMLIR Aug 30, 2025

bdevorem force-pushed the bdevorem/fuse-geg branch 2 times, most recently from 33910a7 to 6c5493b Compare August 30, 2025 08:40

bdevorem marked this pull request as ready for review August 30, 2025 08:42

bdevorem requested review from a team and causten as code owners August 30, 2025 08:42

bdevorem requested a review from Copilot August 30, 2025 08:42

Copilot AI reviewed Aug 30, 2025

View reviewed changes

src/targets/gpu/fuse_mlir.cpp Outdated Show resolved Hide resolved

src/targets/gpu/fuse_mlir.cpp Outdated Show resolved Hide resolved

src/targets/gpu/fuse_mlir.cpp Outdated Show resolved Hide resolved

src/targets/gpu/fuse_mlir.cpp Outdated Show resolved Hide resolved

bdevorem force-pushed the bdevorem/fuse-geg branch from 6c5493b to 713aa1a Compare September 10, 2025 04:18

bdevorem requested a review from Copilot September 12, 2025 05:39

Copilot AI reviewed Sep 12, 2025

View reviewed changes

src/targets/gpu/fuse_mlir.cpp Show resolved Hide resolved

anisha-amd approved these changes Sep 12, 2025

View reviewed changes

pfultz2 mentioned this pull request Sep 17, 2025

Standalone operator Convolution_backwards for MLIR #4292

Merged

bdevorem force-pushed the bdevorem/fuse-geg branch from ca65323 to 8cceb6e Compare September 19, 2025 15:27

bdevorem added 10 commits September 22, 2025 19:12

single output gemm elem gemm fusion w mlir

367f9e2

dead code elim after pass

5e4bd88

make modular; add tag feature to mlir_ops

b6c9b53

working on multi output

917992b

handle multi out case: GE single return val, multi user

6b4b672

fix test case

3211837

multi return from GE fusion supported in GEG fusion

5f1613d

clang format

48bea0c

env var, more formatting, comments, remove change to requirements

84b8953

Revert "multi return from GE fusion supported in GEG fusion"

2d5b25d

This reverts commit ba57d66.

bdevorem added 2 commits September 23, 2025 19:00

handle the case where intermediate external inputs are not placed abo…

510da97

…ve the first instr in the fused mod

add conv test to conv group

fade73a

bdevorem requested review from Copilot and pfultz2 September 24, 2025 03:20

Copilot AI reviewed Sep 24, 2025

View reviewed changes

pfultz2 approved these changes Sep 24, 2025

View reviewed changes

pfultz2 requested review from TedThemistokleous and shivadbhavsar September 24, 2025 15:16

TedThemistokleous approved these changes Sep 24, 2025

View reviewed changes

TedThemistokleous assigned bdevorem Sep 24, 2025

TedThemistokleous added enhancement New feature or request Matchers Updates or adds a change to compile time Matchers rocMLIR labels Sep 24, 2025

shivadbhavsar reviewed Sep 24, 2025

View reviewed changes

bdevorem added 6 commits September 29, 2025 22:10

review comments; adds sort for localized sections of modules

3257bb2

clang-format

a2bc265

make some tests rectangular

7a22ab3

clang-format

6b5d7cb

edit tests to handle both if input fusion enabled or disabled

047d7ac

clang-tidy

699e02b

bdevorem requested a review from shivadbhavsar September 30, 2025 19:06

shivadbhavsar approved these changes Sep 30, 2025

View reviewed changes

bdevorem and others added 2 commits September 30, 2025 21:49

test for localized module sort

35005c3

Merge branch 'develop' into bdevorem/fuse-geg

f43e0a1

causten merged commit b3b6dfd into develop Oct 2, 2025
48 of 51 checks passed

causten deleted the bdevorem/fuse-geg branch October 2, 2025 13:16

shahamed pushed a commit that referenced this pull request Oct 7, 2025

Fuse GEMM+GEMM with rocMLIR (#4261)

22f5227

rocmlir supports GEMM+GEMM fusion, so MIGraphX needs to support the fusion on our side. Solves #4230

Fuse GEMM+GEMM with rocMLIR #4261

Fuse GEMM+GEMM with rocMLIR #4261

Uh oh!

Conversation

bdevorem commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Testing

Unit tests

Verify tests

Considerations

Upcoming PRs

Uh oh!

codecov bot commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bdevorem commented Sep 3, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

bdevorem commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

pfultz2 Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

bdevorem Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

TedThemistokleous Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

migraphx-bot commented Oct 2, 2025

Uh oh!

migraphx-bot commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

bdevorem commented Aug 26, 2025 •

edited

Loading

codecov bot commented Aug 26, 2025 •

edited

Loading

bdevorem commented Sep 19, 2025 •

edited

Loading

TedThemistokleous Sep 24, 2025 •

edited

Loading