Speed To Detect very SLOW

[hohoangan]

First, Thank you for your source, It has helped me a lot.
But speed to detect the object in an image is slow,
How can faster do it such as use GPU or custom list object, Because I need counting vehicle from the camera.
Please help!

[orielswisa]

+1

[Transigent]

This has GPU support:
https://github.com/wang-xinyu/tensorrtx
Some benchmarks at the page end.

[BobLd]

Hi,

There is already an open issue about that here: dotnet/machinelearning#5597 Unfortunatly, and as explained here, this doesn't come from this code but from the ML.Net/Onnx step.

You can use the GPU. If I remember well you will need to use the following package: https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.Gpu/
I did some tests in the past and it worked, but you might have error messages that are linked to nuget packages conflicts. You will need to uninstall the cpu package.

[BobLd]

Try updating to NL.Net v1.6.0, they have improved image handling and it's much faster.
I've updated the repos with 5cab380

[orielswisa]

Hi,

There is already an open issue about that here: dotnet/machinelearning#5597 Unfortunatly, and as explained here, this doesn't come from this code but from the ML.Net/Onnx step.

You can use the GPU. If I remember well you will need to use the following package: https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.Gpu/
I did some tests in the past and it worked, but you might have error messages that are linked to nuget packages conflicts. You will need to uninstall the cpu package.

Hi,

I have nvidia rtx 2080 super i haved try to do that in past but it not work for me,
Are you know which CUDA and cuDNN version i need?

[BobLd]

From this recent article, I see that they use

• Install CUDA v10.1
• Install cuDNN v7.6.4 for CUDA 10.1

But I haven't tried yet

[orielswisa]

Tnx for your answer!
The Model Builder recognized the GPU but how can it help me with your code?

[Eclipi]

I found that the inference speed is a lot faster on .Net Core 5.0 w/o GPU. It took around ~8000ms on .NET Framework 4.7.2 and ~600s on Core.