This sample shows how to use the oneAPI Video Processing Library (oneVPL) to perform a single and mulit-source video decode and preprocess and inference using OpenVINO to show the device surface sharing (zero copy), modified from the example in (oneVPL)[https://github.com/oneapi-src/oneVPL/tree/master/examples].
Optimized for | Description |
---|---|
OS | Ubuntu* 20.04 |
Hardware | Compatible with Intel® oneAPI Video Processing Library(oneVPL) GPU implementation, which can be found at https://github.com/oneapi-src/oneVPL-intel-gpu |
Software | Intel® oneAPI Video Processing Library(oneVPL) CPU implementation and Intel® - OpenVINO 2022.2 |
- Download and install OpenVINO C++ runtime: https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_from_archive_linux.html
- Configurations for Intel® Processor Graphics (GPU): https://docs.openvino.ai/latest/openvino_docs_install_guides_configurations_for_intel_gpu.html#gpu-guide
- Install the OpenVINO Developtment tools:
pip install openvino-dev[tensorflow2,onnx]
apt update && \
apt install --no-install-recommends -q -y software-properties-common gnupg wget sudo unzip libnss3-tools ncurses-term python3-pip
wget https://repositories.intel.com/graphics/intel-graphics.key && \
apt-key add intel-graphics.key && \
apt-add-repository 'deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu focal main' && \
apt update && \
apt install -y libmfx1 libmfxgen1 vainfo intel-opencl-icd intel-level-zero-gpu level-zero intel-media-va-driver-non-free python3.9
apt install -y cmake build-essential libva-dev libdrm-dev net-tools pkg-config libigc-dev intel-igc-cm libigdfcl-dev libigfxcmrt-dev level-zero-dev opencl-headers build-essential
Or you can follow this instruction to install the driver package for a discrete GPU device. https://dgpu-docs.intel.com/index.html
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB && \
apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB && \
echo "deb https://apt.repos.intel.com/oneapi all main" | tee /etc/apt/sources.list.d/oneAPI.list && \
apt update && apt install -y intel-oneapi-onevpl-devel
- OpenVINO:
source <INSTALL_DIR>/setupvars.sh
- oneAPI:
source /opt/intel/oneapi/setvars.sh
3) Build the source code
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
Tips: To trigger dGPU, you have to switch the code as below: https://github.com/OpenVINO-dev-contest/decode-infer-on-GPU/blob/main/multi_src/decode_vpp.h#L252
from
fd = open("/dev/dri/renderD128", O_RDWR);
to
fd = open("/dev/dri/renderD129", O_RDWR);
Download the vehicle detection model from OpenVINO model zoo
omz_downloader --name vehicle-detection-0200
- For single source
./single_src/single_source -i ../content/cars_320x240.h265 -m ~/vehicle-detection-0200/FP32/vehicle-detection-0200.xml
- For multiple source
./multi_src/multi_source -i ../content/cars_320x240.h265,../content/cars_320x240.h265,../content/cars_320x240.h265 -m ~/vehicle-detection-0200/FP32/vehicle-detection-0200.xml -bs 2 -nr 4
- -i = Path to one or multiple input video files, separated by comma;
- -m = Path to IR .xml file;
- -bs = Batch size;
- -nr = Number of inference requests;
- -ns = Number of GPU streams;
- -fr = Number of frame to be decoded for each input source;
Tips: Since the sample has been set the number of stream as 1, the number of infer request should be larger than 1.
In this sample, you will get the inference result according to stream id of input source.
libva info: VA-API version 1.12.0
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_12
libva info: va_openDriver() returns 0
Implementation details:
ApiVersion: 2.7
Implementation type: HW
AccelerationMode via: VAAPI
Path: /usr/lib/x86_64-linux-gnu/libmfx-gen.so.1.2.7
Frames [stream_id=1] [stream_id=0]
image0: bbox 204.99, 49.43, 296.43, 144.56, confidence = 0.99805
image0: bbox 91.26, 115.56, 198.41, 221.69, confidence = 0.99609
image0: bbox 36.50, 44.75, 111.34, 134.57, confidence = 0.98535
image0: bbox 77.92, 72.38, 155.06, 164.30, confidence = 0.97510
image1: bbox 204.99, 49.43, 296.43, 144.56, confidence = 0.99805
image1: bbox 91.26, 115.56, 198.41, 221.69, confidence = 0.99609
image1: bbox 36.50, 44.75, 111.34, 134.57, confidence = 0.98535
image1: bbox 77.92, 72.38, 155.06, 164.30, confidence = 0.97510
Frames [stream_id=1] [stream_id=0]
image0: bbox 206.96, 50.41, 299.54, 146.23, confidence = 0.99805
image0: bbox 93.81, 115.29, 200.86, 222.94, confidence = 0.99414
image0: bbox 84.15, 92.91, 178.14, 191.82, confidence = 0.99316
image0: bbox 37.78, 45.82, 113.29, 132.28, confidence = 0.98193
image0: bbox 75.96, 71.88, 154.31, 164.54, confidence = 0.96582
image1: bbox 206.96, 50.41, 299.54, 146.23, confidence = 0.99805
image1: bbox 93.81, 115.29, 200.86, 222.94, confidence = 0.99414
image1: bbox 84.15, 92.91, 178.14, 191.82, confidence = 0.99316
image1: bbox 37.78, 45.82, 113.29, 132.28, confidence = 0.98193
image1: bbox 75.96, 71.88, 154.31, 164.54, confidence = 0.96582
...
decoded and infered 60 frames
Time = 0.328556s