Skip to content

Upstream merge dec18#2180

Closed
umangyadav wants to merge 8 commits into
developfrom
upstream_merge_dec18
Closed

Upstream merge dec18#2180
umangyadav wants to merge 8 commits into
developfrom
upstream_merge_dec18

Revert https://github.com/llvm/llvm-project/pull/169219 and mark tes…

f344b30
Select commit
Loading
Failed to load commit list.
ROCm Repo Management API / Jenkins failed Dec 22, 2025 in 41m 51s

Build and Test/Matrix - CODEPATH = 'gfx950'/Matrix row orchestration/Tune selected rocMLIR configs: error in 'sh' step

Build and Test / Matrix - CODEPATH = 'mfma' / Matrix - CODEPATH = 'mfma' / Matrix row orchestration / Tune selected rocMLIR configs / Shell Script

Error in sh step, with arguments ../mlir/utils/tuna/tuna-script.sh -o attention -c ../mlir/utils/jenkins/ci-configs/selected-attention-configs -t /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/MITuna -f tuning_attention.tsv [ -f tuning_attention.tsv ].

Failed in branch Matrix - CODEPATH = 'gfx950'
Build log 
Build log truncated.

[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: FINE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GROUP
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    64(0x40) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Machine Models:          HSA_MACHINE_MODEL_LARGE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Profiles:                HSA_PROFILE_BASE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Fast f16:                TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size:      1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size:           4294967295(0xffffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        2147483647(0x7fffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - FBarrier Max Size:       32
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - *******
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Agent 9
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - *******
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Name:                    gfx90a
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Uuid:                    GPU-0c718fc4253be7b6
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Marketing Name:          AMD Instinct MI250X/MI250
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Vendor Name:             AMD
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Feature:                 KERNEL_DISPATCH
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Profile:                 BASE_PROFILE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Float Round Mode:        NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Queue Number:        128(0x80)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Min Size:          64(0x40)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Max Size:          131072(0x20000)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Type:              MULTI
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Node:                    8
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Device Type:             GPU
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,058 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Cache Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - L1:                      16(0x10) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - L2:                      8192(0x2000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Chip ID:                 29708(0x740c)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ASIC Revision:           1(0x1)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Cacheline Size:          128(0x80)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Clock Freq. (MHz):   1700
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - BDFID:                   36352
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Internal Node ID:        8
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Compute Unit:            104
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - SIMDs per CU:            4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Shader Engines:          8
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Shader Arrs. per Eng.:   1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - WatchPts on Addr. Ranges:4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Coherent Host Access:    FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Memory Properties:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Features:                KERNEL_DISPATCH
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Fast F16 Operation:      TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Wavefront Size:          64(0x40)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size:      1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Waves Per CU:        32(0x20)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Work-item Per CU:    2048(0x800)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size:           4294967295(0xffffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        2147483647(0x7fffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max fbarriers/Workgrp:   32
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Packet Processor uCode:: 92
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - SDMA engine uCode::      9
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - IOMMU Support::          None
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: COARSE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 2
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: FINE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GROUP
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    64(0x40) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Machine Models:          HSA_MACHINE_MODEL_LARGE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Profiles:                HSA_PROFILE_BASE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,059 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Fast f16:                TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size:      1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size:           4294967295(0xffffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        2147483647(0x7fffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - FBarrier Max Size:       32
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - *******
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Agent 10
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - *******
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Name:                    gfx90a
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Uuid:                    GPU-01386683f0d100f3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Marketing Name:          AMD Instinct MI250X/MI250
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Vendor Name:             AMD
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Feature:                 KERNEL_DISPATCH
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Profile:                 BASE_PROFILE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Float Round Mode:        NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Queue Number:        128(0x80)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Min Size:          64(0x40)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Max Size:          131072(0x20000)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Queue Type:              MULTI
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Node:                    9
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Device Type:             GPU
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Cache Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - L1:                      16(0x10) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - L2:                      8192(0x2000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Chip ID:                 29708(0x740c)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ASIC Revision:           1(0x1)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Cacheline Size:          128(0x80)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Clock Freq. (MHz):   1700
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - BDFID:                   37632
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Internal Node ID:        9
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Compute Unit:            104
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - SIMDs per CU:            4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Shader Engines:          8
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Shader Arrs. per Eng.:   1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - WatchPts on Addr. Ranges:4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Coherent Host Access:    FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Memory Properties:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Features:                KERNEL_DISPATCH
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Fast F16 Operation:      TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Wavefront Size:          64(0x40)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size:      1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Waves Per CU:        32(0x20)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max Work-item Per CU:    2048(0x800)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size:           4294967295(0xffffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        2147483647(0x7fffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Max fbarriers/Workgrp:   32
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Packet Processor uCode:: 92
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - SDMA engine uCode::      9
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - IOMMU Support::          None
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: COARSE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 2
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,060 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GLOBAL; FLAGS: FINE GRAINED
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    67092480(0x3ffc000) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:2048KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         4KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Pool 4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Segment:                 GROUP
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Size:                    64(0x40) KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Allocatable:             FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Granule:           0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Recommended Granule:0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Alloc Alignment:         0KB
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Accessible by all:       FALSE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA Info:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - ISA 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Machine Models:          HSA_MACHINE_MODEL_LARGE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Profiles:                HSA_PROFILE_BASE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Default Rounding Mode:   NEAR
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Fast f16:                TRUE
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size:      1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Workgroup Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        1024(0x400)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size:           4294967295(0xffffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - Grid Max Size per Dimension:
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - x                        2147483647(0x7fffffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - y                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - z                        65535(0xffff)
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - FBarrier Max Size:       32
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,061 - Machine_567fb1f7e85d - INFO -  [connection.py:276] - *** Done ***
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,068 - Machine_567fb1f7e85d - INFO -  [machine.py:146] - avail gpus: [0, 1, 2, 3, 4, 5, 6, 7]
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,068 - miopen_utility - INFO -  [miopen_utility.py:67] - Local machine not in database, continue with incomplete details
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,073 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 0
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,088 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 1
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,089 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102609
[2025-12-22T17:19:22.306Z] 171 - 2025-12-22 17:19:22,089 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - No connection for process 102609, creating now
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:No connection for process 102609, creating now
[2025-12-22T17:19:22.306Z] 440 - 2025-12-22 17:19:22,090 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Checking barrier
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Checking barrier
[2025-12-22T17:19:22.306Z] 499 - 2025-12-22 17:19:22,090 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Getting free space
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Getting free space
[2025-12-22T17:19:22.306Z] 519 - 2025-12-22 17:19:22,090 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,090 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.306Z] 269 - 2025-12-22 17:19:22,095 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - new jobs [1]
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:new jobs [1]
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,095 - db_utility - INFO -  [db_utility.py:195] - Query Update: UPDATE rocmlir_attention_job SET state='running' WHERE id=1;
[2025-12-22T17:19:22.306Z] 235 - 2025-12-22 17:19:22,097 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Put job 1 running 2025-12-22T17:19+00:00 attention
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Put job 1 running 2025-12-22T17:19+00:00 attention
[2025-12-22T17:19:22.306Z] 240 - 2025-12-22 17:19:22,097 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Got job 1 running 2025-12-22T17:19+00:00 attention
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Got job 1 running 2025-12-22T17:19+00:00 attention
[2025-12-22T17:19:22.306Z] 127 - 2025-12-22 17:19:22,097 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Acquired new job: job_id=1
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Acquired new job: job_id=1
[2025-12-22T17:19:22.306Z] 314 - 2025-12-22 17:19:22,097 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Setting job id 1 state to running
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Setting job id 1 state to running
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,098 - db_utility - INFO -  [db_utility.py:195] - Query Update: UPDATE rocmlir_attention_job SET state='running',gpu_id=0 WHERE id=1;
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,097 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 2
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,098 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102611
[2025-12-22T17:19:22.306Z] 171 - 2025-12-22 17:19:22,099 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - INFO - No connection for process 102611, creating now
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:No connection for process 102611, creating now
[2025-12-22T17:19:22.306Z] 440 - 2025-12-22 17:19:22,099 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - INFO - Checking barrier
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:Checking barrier
[2025-12-22T17:19:22.306Z] 499 - 2025-12-22 17:19:22,099 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - INFO - Getting free space
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:Getting free space
[2025-12-22T17:19:22.306Z] 519 - 2025-12-22 17:19:22,099 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,100 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,102 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.306Z] 193 - 2025-12-22 17:19:22,103 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.306Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:No new jobs found, session 3
[2025-12-22T17:19:22.306Z] 396 - 2025-12-22 17:19:22,104 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - Running command:  python3 ./bin/tuningRunner.py -q --operation attention --verify-mode none --tuning-space=full                      --config='-t f16 -transQ False -transK True -transV False -transO False -g 256 -seq_len_q 1024 -seq_len_k 1024 -num_heads_q 1 -num_heads_kv 1 -head_dim_qk 128 -head_dim_v 128 -causal False -return_lse False -split_kv 1 -with-attn-scale False -with-attn-bias False' --mlir-build-dir `pwd`                      --output=- --tflops                      --rocmlir_gen_flags='--device=0' 2>/dev/null
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:Running command:  python3 ./bin/tuningRunner.py -q --operation attention --verify-mode none --tuning-space=full                      --config='-t f16 -transQ False -transK True -transV False -transO False -g 256 -seq_len_q 1024 -seq_len_k 1024 -num_heads_q 1 -num_heads_kv 1 -head_dim_qk 128 -head_dim_v 128 -causal False -return_lse False -split_kv 1 -with-attn-scale False -with-attn-bias False' --mlir-build-dir `pwd`                      --output=- --tflops                      --rocmlir_gen_flags='--device=0' 2>/dev/null
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,104 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,106 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102612
[2025-12-22T17:19:22.306Z] 171 - 2025-12-22 17:19:22,106 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - INFO - No connection for process 102612, creating now
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:No connection for process 102612, creating now
[2025-12-22T17:19:22.306Z] 440 - 2025-12-22 17:19:22,106 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - INFO - Checking barrier
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:Checking barrier
[2025-12-22T17:19:22.306Z] 276 - 2025-12-22 17:19:22,106 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - /tuna-venv/bin/python3
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:/tuna-venv/bin/python3
[2025-12-22T17:19:22.306Z] 499 - 2025-12-22 17:19:22,106 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - INFO - Getting free space
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:Getting free space
[2025-12-22T17:19:22.306Z] 519 - 2025-12-22 17:19:22,106 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,107 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,109 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.306Z] 193 - 2025-12-22 17:19:22,110 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.306Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:No new jobs found, session 3
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,111 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 4
[2025-12-22T17:19:22.306Z] 2025-12-22 17:19:22,112 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102618
[2025-12-22T17:19:22.306Z] 171 - 2025-12-22 17:19:22,112 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - INFO - No connection for process 102618, creating now
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:No connection for process 102618, creating now
[2025-12-22T17:19:22.306Z] 440 - 2025-12-22 17:19:22,112 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - INFO - Checking barrier
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:Checking barrier
[2025-12-22T17:19:22.306Z] 499 - 2025-12-22 17:19:22,112 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - INFO - Getting free space
[2025-12-22T17:19:22.306Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:Getting free space
[2025-12-22T17:19:22.307Z] 519 - 2025-12-22 17:19:22,112 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,113 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,115 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.307Z] 193 - 2025-12-22 17:19:22,115 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.307Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:No new jobs found, session 3
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,116 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 5
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,118 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102619
[2025-12-22T17:19:22.307Z] 171 - 2025-12-22 17:19:22,118 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - INFO - No connection for process 102619, creating now
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:No connection for process 102619, creating now
[2025-12-22T17:19:22.307Z] 440 - 2025-12-22 17:19:22,118 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - INFO - Checking barrier
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:Checking barrier
[2025-12-22T17:19:22.307Z] 499 - 2025-12-22 17:19:22,118 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - INFO - Getting free space
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:Getting free space
[2025-12-22T17:19:22.307Z] 519 - 2025-12-22 17:19:22,119 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,119 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,121 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.307Z] 193 - 2025-12-22 17:19:22,122 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.307Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:No new jobs found, session 3
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,124 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 6
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,125 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102620
[2025-12-22T17:19:22.307Z] 171 - 2025-12-22 17:19:22,125 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - INFO - No connection for process 102620, creating now
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:No connection for process 102620, creating now
[2025-12-22T17:19:22.307Z] 440 - 2025-12-22 17:19:22,125 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - INFO - Checking barrier
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:Checking barrier
[2025-12-22T17:19:22.307Z] 499 - 2025-12-22 17:19:22,126 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - INFO - Getting free space
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:Getting free space
[2025-12-22T17:19:22.307Z] 519 - 2025-12-22 17:19:22,126 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,126 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,128 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.307Z] 193 - 2025-12-22 17:19:22,128 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.307Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:No new jobs found, session 3
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,131 - rocmlir - INFO -  [rocmlir_lib.py:152] - launch mid 0, proc 7
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,133 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102621
[2025-12-22T17:19:22.307Z] 171 - 2025-12-22 17:19:22,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - INFO - No connection for process 102621, creating now
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:No connection for process 102621, creating now
[2025-12-22T17:19:22.307Z] 440 - 2025-12-22 17:19:22,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - INFO - Checking barrier
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:Checking barrier
[2025-12-22T17:19:22.307Z] 499 - 2025-12-22 17:19:22,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - INFO - Getting free space
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:Getting free space
[2025-12-22T17:19:22.307Z] 519 - 2025-12-22 17:19:22,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,134 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,135 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.307Z] 193 - 2025-12-22 17:19:22,136 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.307Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:No new jobs found, session 3
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,139 - Machine_567fb1f7e85d - INFO -  [machine.py:152] - Set logger for process 102622
[2025-12-22T17:19:22.307Z] 171 - 2025-12-22 17:19:22,139 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - INFO - No connection for process 102622, creating now
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:No connection for process 102622, creating now
[2025-12-22T17:19:22.307Z] 440 - 2025-12-22 17:19:22,139 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - INFO - Checking barrier
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:Checking barrier
[2025-12-22T17:19:22.307Z] 499 - 2025-12-22 17:19:22,139 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - INFO - Getting free space
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:Getting free space
[2025-12-22T17:19:22.307Z] 519 - 2025-12-22 17:19:22,140 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - INFO - Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:Used space on 567fb1f7e85d : 18
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,140 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE SKIP LOCKED;
[2025-12-22T17:19:22.307Z] 2025-12-22 17:19:22,142 - db_utility - INFO -  [db_utility.py:220] - Query Select: SELECT id,valid,reason,state,retries,result,compile_start,compile_end,gpu_id,machine_name,config,session FROM rocmlir_attention_job WHERE session=3 AND valid=1 AND retries<10 AND state='new' ORDER BY retries,config ASC LIMIT 24 FOR UPDATE;
[2025-12-22T17:19:22.307Z] 193 - 2025-12-22 17:19:22,143 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - WARNING - No new jobs found, session 3
[2025-12-22T17:19:22.307Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:No new jobs found, session 3
[2025-12-22T17:19:23.472Z] 276 - 2025-12-22 17:19:23,048 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0 - INFO - # arch	numCUs	testVector	perfConfig (full)	TFlops
[2025-12-22T17:19:23.472Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/0:# arch	numCUs	testVector	perfConfig (full)	TFlops
[2025-12-22T17:19:23.472Z] 509 - 2025-12-22 17:19:23,144 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - INFO - proc 7 step False
[2025-12-22T17:19:23.472Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:proc 7 step False
[2025-12-22T17:19:23.472Z] 511 - 2025-12-22 17:19:23,144 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7 - WARNING - No more steps, quitting...
[2025-12-22T17:19:23.472Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/7:No more steps, quitting...
[2025-12-22T17:19:24.191Z] 509 - 2025-12-22 17:19:24,105 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - INFO - proc 1 step False
[2025-12-22T17:19:24.191Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:proc 1 step False
[2025-12-22T17:19:24.191Z] 511 - 2025-12-22 17:19:24,105 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1 - WARNING - No more steps, quitting...
[2025-12-22T17:19:24.191Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/1:No more steps, quitting...
[2025-12-22T17:19:24.191Z] 509 - 2025-12-22 17:19:24,138 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - INFO - proc 6 step False
[2025-12-22T17:19:24.191Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:proc 6 step False
[2025-12-22T17:19:24.191Z] 511 - 2025-12-22 17:19:24,139 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6 - WARNING - No more steps, quitting...
[2025-12-22T17:19:24.191Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/6:No more steps, quitting...
[2025-12-22T17:19:25.356Z] 509 - 2025-12-22 17:19:25,113 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - INFO - proc 2 step False
[2025-12-22T17:19:25.356Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:proc 2 step False
[2025-12-22T17:19:25.356Z] 511 - 2025-12-22 17:19:25,116 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2 - WARNING - No more steps, quitting...
[2025-12-22T17:19:25.356Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/2:No more steps, quitting...
[2025-12-22T17:19:29.205Z] 509 - 2025-12-22 17:19:29,121 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - INFO - proc 3 step False
[2025-12-22T17:19:29.205Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:proc 3 step False
[2025-12-22T17:19:29.205Z] 511 - 2025-12-22 17:19:29,121 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3 - WARNING - No more steps, quitting...
[2025-12-22T17:19:29.205Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/3:No more steps, quitting...
[2025-12-22T17:19:30.554Z] 509 - 2025-12-22 17:19:30,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - INFO - proc 5 step False
[2025-12-22T17:19:30.554Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:proc 5 step False
[2025-12-22T17:19:30.554Z] 511 - 2025-12-22 17:19:30,133 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5 - WARNING - No more steps, quitting...
[2025-12-22T17:19:30.554Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/5:No more steps, quitting...
[2025-12-22T17:19:32.369Z] 509 - 2025-12-22 17:19:32,129 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - INFO - proc 4 step False
[2025-12-22T17:19:32.369Z] INFO:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:proc 4 step False
[2025-12-22T17:19:32.369Z] 511 - 2025-12-22 17:19:32,129 - /home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4 - WARNING - No more steps, quitting...
[2025-12-22T17:19:32.369Z] WARNING:/home/jenkins/internal/workspace/MLIR_mlir_PR-2180/tmp/tuna_logs/RocMLIRWorker/567fb1f7e85d_Nonep/4:No more steps, quitting...
[2025-12-22T17:21:24.458Z] Sending interrupt signal to process
[2025-12-22T17:21:25.927Z] Terminated
[2025-12-22T17:21:26.059Z] script returned exit code 143

Output truncated.

Details

  • Set System Property (5.9 sec)
  • Kill old PR builds (1.3 sec)
  • Build and Test (39 min)
    • Matrix - CODEPATH = 'vanilla' (6 ms)
      • Matrix - CODEPATH = 'vanilla' (18 sec)
        • Matrix row orchestration (8 sec)
    • Matrix - CODEPATH = 'mfma' (6 ms)
      • Matrix - CODEPATH = 'mfma' (39 min)
        • Matrix row orchestration (39 min)
          • Health checks on ctr-ubbsmc02 (3 min 3 sec)
          • Node selected (1.2 sec)
          • SCM Checkout (54 sec)
          • Prepare Docker environment (3 min 19 sec)
            • Shared Library: fixed E2E (28 min)
            • Tune selected rocMLIR configs (3 min 29 sec)
              Error: Failed in branch Matrix - CODEPATH = 'gfx950' - logs
    • Matrix - CODEPATH = 'navi21' (6 ms)
      • Matrix - CODEPATH = 'navi21' (18 sec)
        • Matrix row orchestration (8 sec)
    • Matrix - CODEPATH = 'navi3x' (6 ms)
      • Matrix - CODEPATH = 'navi3x' (39 min)
        • Matrix row orchestration (39 min)
          • Health checks on ctr-smc-s39-006.amd.com (36 sec)
          • Node selected (1.3 sec)
          • SCM Checkout (2 min 9 sec)
          • Prepare Docker environment (2 min 31 sec)
            • Shared Library: fixed E2E (14 min)
            • Tune selected rocMLIR configs (14 min)
            • Static Lib: build packages (4 min 44 sec)
              Error: Failed in branch Matrix - CODEPATH = 'gfx950' - logs
    • Matrix - CODEPATH = 'navi4x' (7 ms)
      • Matrix - CODEPATH = 'navi4x' (39 min)
        • Matrix row orchestration (39 min)
          • Health checks on navi4x-hw-31.amd.com (33 sec)
          • Node selected (1.4 sec)
          • SCM Checkout (2 min 26 sec)
          • Prepare Docker environment (9.6 sec)
            • Shared Library: fixed E2E (23 min)
            • Tune selected rocMLIR configs (8 min 33 sec)
            • Static Lib: build packages (3 min 40 sec)
              Error: Failed in branch Matrix - CODEPATH = 'gfx950' - logs
    • Matrix - CODEPATH = 'gfx950' (39 min)
      • Matrix - CODEPATH = 'gfx950' (39 min)
        • Matrix row orchestration (39 min)
          • Health checks on k8s-mi355-1-r2v0k (23 sec)
          • Node selected (1.3 sec)
          • SCM Checkout (2 min 29 sec)
          • Prepare Docker environment (1 min 55 sec)
            • Shared Library: fixed E2E (13 min)
            • Tune selected rocMLIR configs (19 min)
              Error: script returned exit code 1 - logs
  • Parameter sweeps (18 sec)
    • Matrix - CODEPATH = 'mfma' (7 ms)
      • Matrix - CODEPATH = 'mfma' (11 sec)
        • Matrix row orchestration (5.4 sec)
    • Matrix - CODEPATH = 'vanilla' (7 ms)
      • Matrix - CODEPATH = 'vanilla' (11 sec)
        • Matrix row orchestration (5.4 sec)
    • Matrix - CODEPATH = 'navi21' (6 ms)
      • Matrix - CODEPATH = 'navi21' (11 sec)
        • Matrix row orchestration (5.4 sec)
    • Matrix - CODEPATH = 'gfx950' (16 sec)
      • Matrix - CODEPATH = 'gfx950' (11 sec)
        • Matrix row orchestration (5.4 sec)
  • Tune MLIR kernels (26 sec)
    • Matrix - CHIP = 'gfx908' (7 ms)
      • Matrix - CHIP = 'gfx908' (16 sec)
        • Matrix row orchestration (8.1 sec)
    • Matrix - CHIP = 'gfx90a' (7 ms)
      • Matrix - CHIP = 'gfx90a' (16 sec)
        • Matrix row orchestration (8.1 sec)
    • Matrix - CHIP = 'gfx942' (9 ms)
      • Matrix - CHIP = 'gfx942' (16 sec)
        • Matrix row orchestration (8.1 sec)
    • Matrix - CHIP = 'gfx950' (8 ms)
      • Matrix - CHIP = 'gfx950' (16 sec)
        • Matrix row orchestration (8.1 sec)
    • Matrix - CHIP = 'gfx1100' (6 ms)
      • Matrix - CHIP = 'gfx1100' (16 sec)
        • Matrix row orchestration (8.1 sec)
    • Matrix - CHIP = 'gfx1201' (24 sec)
      • Matrix - CHIP = 'gfx1201' (16 sec)
        • Matrix row orchestration (8 sec)
  • Archive weekly tuning perfDB (1.5 sec)
  • Benchmark and Report Performance (29 sec)
    • Matrix - CHIP = 'gfx908' (7 ms)
      • Matrix - CHIP = 'gfx908' (18 sec)
        • Matrix row orchestration (8.8 sec)
    • Matrix - CHIP = 'gfx90a' (9 ms)
      • Matrix - CHIP = 'gfx90a' (18 sec)
        • Matrix row orchestration (8.9 sec)
    • Matrix - CHIP = 'gfx942' (8 ms)
      • Matrix - CHIP = 'gfx942' (18 sec)
        • Matrix row orchestration (8.9 sec)
    • Matrix - CHIP = 'gfx950' (6 ms)
      • Matrix - CHIP = 'gfx950' (18 sec)
        • Matrix row orchestration (8.9 sec)
    • Matrix - CHIP = 'gfx1030' (7 ms)
      • Matrix - CHIP = 'gfx1030' (18 sec)
        • Matrix row orchestration (8.9 sec)
    • Matrix - CHIP = 'gfx1100' (8 ms)
      • Matrix - CHIP = 'gfx1100' (18 sec)
        • Matrix row orchestration (9 sec)
    • Matrix - CHIP = 'gfx1201' (27 sec)
      • Matrix - CHIP = 'gfx1201' (18 sec)
        • Matrix row orchestration (9 sec)
  • MIGraphX (19 sec)
    • Matrix - CODEPATH = 'mfma' (9 ms)
      • Matrix - CODEPATH = 'mfma' (11 sec)
        • Matrix row orchestration (5.1 sec)
    • Matrix - CODEPATH = 'navi21' (10 ms)
      • Matrix - CODEPATH = 'navi21' (11 sec)
        • Matrix row orchestration (5.2 sec)
    • Matrix - CODEPATH = 'navi4x' (6 ms)
      • Matrix - CODEPATH = 'navi4x' (11 sec)
        • Matrix row orchestration (5.2 sec)
    • Matrix - CODEPATH = 'gfx950' (17 sec)
      • Matrix - CODEPATH = 'gfx950' (11 sec)
        • Matrix row orchestration (5.2 sec)
  • Code coverage (13 sec)
    • Matrix - CODEPATH = 'mfma' (7 ms)
      • Matrix - CODEPATH = 'mfma' (8 sec)
        • Matrix row orchestration (3.8 sec)
    • Matrix - CODEPATH = 'navi4x' (7 ms)
      • Matrix - CODEPATH = 'navi4x' (8 sec)
        • Matrix row orchestration (3.8 sec)
    • Matrix - CODEPATH = 'gfx950' (12 sec)
      • Matrix - CODEPATH = 'gfx950' (8 sec)
        • Matrix row orchestration (3.8 sec)
View more details on ROCm Repo Management API