after forge-extension update: error message with local path (which is not mine) when "background removal" activated

[dermesut]

• title says it, just updated the ic-light extension in forge and this is the error message i get in the cmd-window:
  "
  2024-06-08 11:30:02.0788894 [E:onnxruntime:Default, provider_bridge_ort.cc:1534 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\ai_gh_repos\webui_forge_cu121_torch21\system\python\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"

*************** EP Error ***************
EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:456 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

---

"
still, rendering is being done, but unsable if background removal is activated.
if not, everything seems to be working as expected.

• not sure, if related to that:
  in img2img, i see barely a difference between the the generated image and the init-image, if "Reinforce Foreground" is activated. as though ic-light didn't do any relighting at all.

• more of a question:
  prior to this update, txt2img in fbc mode had gradients. now there only the option to use a custom image. is there s.th. wrong on my side or have those been removed?

[huchenlei]

*************** EP Error ***************
EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:456 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

This is a ONNX related issue. @Haoming02 do you know if we need to add any extra dep requirement to the extension?

not sure, if related to that:
in img2img, i see barely a difference between the the generated image and the init-image, if "Reinforce Foreground" is activated. as though ic-light didn't do any relighting at all.

Can you share your generation setup?

more of a question:
prior to this update, txt2img in fbc mode had gradients. now there only the option to use a custom image. is there s.th. wrong on my side or have those been removed?

Yes. They are removed. Previously they are just short-hand for testing purpose.

[dermesut]

This is a ONNX related issue. @Haoming02 do you know if we need to add any extra dep requirement to the extension?
so, i guess, i'll just wait and see... or is there sth on my side, that i could do to make it work?

Can you share your generation setup?
sure, here are 2 screenshots, one with and the other without "Reinforce Foreground" activated:

Yes. They are removed. Previously they are just short-hand for testing purpose.
ok. thx.

[Haoming02]

The Onnx issue should already be fixed by the install.py though.
It should automatically install a onnxruntime package that uses CPU.

[dermesut]

The Onnx issue should already be fixed by the install.py though. It should automatically install a onnxruntime package that uses CPU.

looked at the path, that is given in the error message "2024-06-08 18:29:40.9022887 [E:onnxruntime:Default, provider_bridge_ort.cc:1534 onnxruntime::TryGetProviderInfo_TensorRT] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "E:\ai_gh_repos\webui_forge_cu121_torch21\system\python\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll""

so, it is there.

[Haoming02]

Hmm. By default, it should be using the CPU one. Why is it loading the tensorrt one?

I also had this issue before, but after I manually pip install onnxruntime, the issue went away...

[Haoming02]

And yeah, I just checked my installation. My onnxruntime does not have those .dll files.

I guess the problem is, the Webui checks that onnxruntime is already installed, so it didn't install it again.

Hence why yours is still using providers other than CPU.

[Haoming02]

Others also had the same issue: danielgatis/rembg#312

The simplest solution is to uninstall the gpu version and install the cpu version...

[dermesut]

yes, i just installed the rembg extension and it shows the same behaviour.
not sure, if uninstalling the current gpu-version and installing the cpu-version would have an impact on other installed extensions.
just for the fun of it, i renamed the folder of that runtime and started the server, obviously resulting in a list of error message, starting from "insightface" to whatnot...
i wonder if there'd be a more convenient solution to this.

[dan4ik94]

Others also had the same issue: danielgatis/rembg#312

The simplest solution is to uninstall the gpu version and install the cpu version...

from what I remember the cpu version is pretty slow even on my 14700k and consumes a lot of power. Maybe we should revert to the old rembg implementation that didn't have any issues running on GPU.

The current one doesn't use the embedded python but searches for dependencies system wide for some reason and fails if you don't have cudnn etc installed and added to path.

[Haoming02]

It's instant on 13700 though?

How large of an image were you processing?

[dan4ik94]

It's instant on 13700 though?

How large of an image were you processing?

strange, it was probably around 5-7 secs last time I checked with a 900px-1024px image

[Haoming02]

btw, can anyone try

pip uninstall rembg
pip install rembg[gpu]

to see if it can work with non-CPU version of onnxruntime

[dan4ik94]

btw, can anyone try

pip uninstall rembg
pip install rembg[gpu]

to see if it can work with non-CPU version of onnxruntime

No differences for me, even if I completely remove the rembg though venv I get the same error.

pip list

[Haoming02]

Pain...

So you either get fricked by HuggingFace or by onnxruntime 🤣

[dan4ik94]

Pain...

So you either get fricked by HuggingFace or by onnxruntime 🤣

yup these dependency/package compatibility issues are a huge pain, fortunately I don’t get them so often.

I’ll keep trying, the harder the better. Giving up is the last thing I can do.

[dan4ik94]

@dermesut @huchenlei @Haoming02

lil update from me

A)
I've tried installing cuda, cudnn and tersorrt stuff as suggested here danielgatis/rembg#312 (comment)

I got rid of all the errors and warnings, however the bg extraction speed incresed drastically and took 1 minute for 800px image. :O

B) I've removed the tensorrt libs, replaced the onnxruntime with onnxruntime-gpu in the install.py file. Also removed the rembg and onnxruntime packages from the \system\python\Lib\site-packages folder to make sure Forge does a clean install on startup.

rembg finally started to use cuda instead of tensorrt and the bg extraction speed went down to 1-2 secs.

So to summarize:

tensorrt (onnxruntime): 60 secs
default (unknown): 7 secs
cuda (onnxruntime-gpu): 1-2 secs

N.B. also to mention that I had a newer driver with CUDA 12.5, that onnxruntime didn't support yet. Make sure you have the right cuda and cudnn versions. https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements or just wait for a newer version (1.18.1) later this month.

[Haoming02]

I just checked on my Automatic1111 installation, which has both onnxruntime and onnxruntime-gpu, and there is also onnxruntime_providers_tensorrt.dll inside ~webui\venv\Lib\site-packages\onnxruntime\capi.

But the IC-Light still works fine with no error.

Tested on Forge as well, still no error.

Also tested by renaming the TensorRT folder (to break the SYSTEM PATH), still no error.

Guess I will test with a clean install later.

[dan4ik94]

I just checked on my Automatic1111 installation, which has both onnxruntime and onnxruntime-gpu, and there is also onnxruntime_providers_tensorrt.dll inside ~webui\venv\Lib\site-packages\onnxruntime\capi.

But the IC-Light still works fine with no error.

Tested on Forge as well, still no error.

Also tested by renaming the TensorRT folder (to break the SYSTEM PATH), still no error.

Guess I will test with a clean install later.

Could just be a driver & cuda/onnx compatibility issue, like in my case.

[dan4ik94]

After updating onnxruntime-gpu to 1.18.1 and installing the latest rembg version the issue went away (bg removal is super fast again)

rembg di per se is not so accurate and good at removing backgrounds, but that's another question.

[hben35096]

I happened to encounter that after installing onnxruntime-gpu, an error would be reported. I uninstalled onnxruntime and installed it again, and the error disappeared.