Warning when running PyTorch with ROCm

[succo104]

My system:
AMD Ryzen 5 7600X
32 GB RAM DDR5 6000 Mhz
NVME SSD 256 GB (For Ubuntu) + 2 TB (For Windows)
AMD Radeon RX 6800 RDNA 2

I'm using the latest AMD drivers on Ubuntu 22.04 LTS and getting this warning when running my PyTorch deep learning AI model:

MIOpen(HIP): Warning [FindSolutionImpl] Invalid config loaded from Perf Db: ConvBinWinogradRxSf3x2: 72. Performance may degrade.

What does this mean and how can i fix this?

[AtiqurRahmanAni]

I am facing the same warning

[ppanchad-amd]

Hi @succo104, an internal ticket has been created to investigate your issue. Thanks!

[jamesxu2]

Hi @succo104

I'm using the latest AMD drivers on Ubuntu 22.04 LTS and getting this warning when running my PyTorch deep learning AI model:

Can you provide more information on what PyTorch deep learning AI model you're running? If you have a specific model, tutorial, or can share source code that would really help to reproduce this issue. Also, do you have ROCm installed, and which version? Please provide as much detail as you can!

@AtiqurRahmanAni If you can provide the same information or just follow along on this ticket that would be great. We usually recommend that you open a new ticket rather than adding on to an existing issue, because they may have separate root causes. Also, we don't have any information about your hardware, OS, ROCm version which makes this hard to diagnose.

Some conjecture: At this point, I suspect the issue is just because you are both probably not running an officially supported GPU and MIOpen does not have a precompiled kernel that's tuned for your specific hardware, resulting in a less-than-optimal model performance. However, we can confirm this with more information on how to reproduce the problem.

[AtiqurRahmanAni]

Thank you so much your response. I am running on RX6600, Ubuntu 24.04.1 LTS. ROCm version is 6.2.0. Basically, when I use BatchNorm layer in CNN model, I get this warning.