Skip to content

innerlee/cvbenchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

800c879 · Jun 19, 2020

History

73 Commits
Jun 19, 2020
May 8, 2020
May 8, 2020
Feb 4, 2020
Feb 20, 2020
Feb 14, 2020
Jun 19, 2020

Repository files navigation

CV Benchmark

Who is fast, and who is the fastest.

Results

  • Read a .jpeg color image as an numpy array.
./mmbench benchmarks/run_resize.py
# resize cv2  50 loops, best of 5: 5.13 msec per loop
# resize np   20 loops, best of 5: 9.05 msec per loop

./mmbench jpg2np_color
# jpg2np opencv     200 loops, best of 5: 1.51 msec per loop
# jpg2np turbojpeg  500 loops, best of 5: 624 usec per loop
  • Read a .jpeg gray scale image as an numpy array.
./mmbench jpg2np_gray
# jpg2np_gray opencv     500 loops, best of 5: 658 usec per loop
# jpg2np_gray turbojpeg  500 loops, best of 5: 451 usec per loop
  • Read a .jpeg color image from bytes to an numpy array.
./mmbench jpgbytes2np_color
# jpgbytes2np_color opencv     200 loops, best of 5: 1.39 msec per loop
# jpgbytes2np_color turbojpeg  500 loops, best of 5: 608 usec per loop
  • Read a .jpeg gray scale image from bytes to an numpy array.
./mmbench jpgbytes2np_gray
# jpgbytes2np_gray opencv     500 loops, best of 5: 629 usec per loop
# jpgbytes2np_gray turbojpeg  500 loops, best of 5: 438 usec per loop
  • Image array normalization
./mmbench img_array_normalize
# img_array_normalize cv2_div    200 loops, best of 5: 1.81 msec per loop
# img_array_normalize cv2_mult   200 loops, best of 5: 1.68 msec per loop
# img_array_normalize cv2_mult2  200 loops, best of 5: 1.33 msec per loop
# img_array_normalize cv2_mult3  200 loops, best of 5: 1.25 msec per loop
# img_array_normalize cv2_mult4  500 loops, best of 5: 594 usec per loop
# img_array_normalize np         100 loops, best of 5: 2.43 msec per loop
  • Convert numpy array from unit8 to float32
./mmbench np_uint8_to_float32
# np_uint8_to_float32 astype       5000 loops, best of 5: 69 usec per loop
# np_uint8_to_float32 constructor  5000 loops, best of 5: 67.8 usec per loop
  • Stack a list of np array (non-contiguous) along axis 0. This test is unstable. Not sure why, yet
./mmbench img_list_stack
# img_list_stack constructor  100 loops, best of 5: 2.12 msec per loop
# img_list_stack stack        200 loops, best of 5: 1.96 msec per loop
  • Clip bbox
./mmbench bbox_clip
# bbox_clip clip  50000 loops, best of 5: 4.09 usec per loop
# bbox_clip fast  100000 loops, best of 5: 3.49 usec per loop
# bbox_clip slow  50000 loops, best of 5: 5.71 usec per loop
  • Permute channel dim before spatial at cpu or gpu.
./mmbench -n 500 benchmarks/run_permute_at.py
# permute_at cpu  500 loops, best of 5: 1.22 msec per loop
# permute_at gpu  500 loops, best of 5: 478 usec per loop
  • Flip image array using np or cv2.
./mmbench benchmarks/run_flip.py
# flip cv2  500 loops, best of 5: 915 usec per loop
# flip np   50 loops, best of 5: 7.66 msec per loop
  • Pipeline: load -> resize -> crop -> flip -> normalize -> transpose
## color

# color 400x300 -> x1.41 -> 224x224
./mmbench -n 200 pipeline_color
# pipeline_color opencv          200 loops, best of 5: 4.85 msec per loop
# pipeline_color opencv_fast     200 loops, best of 5: 2.81 msec per loop
# pipeline_color opencv_fastest  200 loops, best of 5: 1.8 msec per loop
# pipeline_color pil             200 loops, best of 5: 2.6 msec per loop
# pipeline_color pil_fast        200 loops, best of 5: 1.85 msec per loop

# color 400x300 -> x0.80 -> 224x224
./mmbench -n 200 pipeline_color_shrink
# pipeline_color_shrink opencv          200 loops, best of 5: 3.26 msec per loop
# pipeline_color_shrink opencv_fast     200 loops, best of 5: 2.13 msec per loop
# pipeline_color_shrink opencv_fastest  200 loops, best of 5: 1.89 msec per loop
# pipeline_color_shrink pil             200 loops, best of 5: 1.81 msec per loop
# pipeline_color_shrink pil_fast        200 loops, best of 5: 1.86 msec per loop

# color 800x600 -> x0.426 -> 224x224
./mmbench -n 200 pipeline_color_shrink_big
# pipeline_color_shrink_big opencv          200 loops, best of 5: 7.22 msec per loop
# pipeline_color_shrink_big opencv_fast     200 loops, best of 5: 3.1 msec per loop
# pipeline_color_shrink_big opencv_fastest  200 loops, best of 5: 2.65 msec per loop
# pipeline_color_shrink_big pil             200 loops, best of 5: 2.7 msec per loop
# pipeline_color_shrink_big pil_fast        200 loops, best of 5: 2.87 msec per loop

# color 133x100 -> x2.56 -> 224x224
./mmbench -n 200 pipeline_color_small
# pipeline_color_small opencv          200 loops, best of 5: 1.91 msec per loop
# pipeline_color_small opencv_fast     200 loops, best of 5: 1.4 msec per loop
# pipeline_color_small opencv_fastest  200 loops, best of 5: 1.14 msec per loop
# pipeline_color_small pil             200 loops, best of 5: 1.23 msec per loop
# pipeline_color_small pil_fast        200 loops, best of 5: 1.18 msec per loop


## gray

# gray 400x300 -> x1.41 -> 224x224
./mmbench -n 200 pipeline_gray
# pipeline_gray opencv          200 loops, best of 5: 1.63 msec per loop
# pipeline_gray opencv_fast     200 loops, best of 5: 1.33 msec per loop
# pipeline_gray opencv_fastest  200 loops, best of 5: 865 usec per loop
# pipeline_gray pil             200 loops, best of 5: 2.31 msec per loop
# pipeline_gray pil_fast        200 loops, best of 5: 1.19 msec per loop

# gray 400x300 -> x0.80 -> 224x224
./mmbench -n 200 pipeline_gray_shrink
# pipeline_gray_shrink opencv          200 loops, best of 5: 1.59 msec per loop
# pipeline_gray_shrink opencv_fast     200 loops, best of 5: 1.28 msec per loop
# pipeline_gray_shrink opencv_fastest  200 loops, best of 5: 898 usec per loop
# pipeline_gray_shrink pil             200 loops, best of 5: 2.3 msec per loop
# pipeline_gray_shrink pil_fast        200 loops, best of 5: 1.22 msec per loop

# gray 133x100 -> x2.56 -> 224x224
./mmbench -n 200 pipeline_gray_small
# pipeline_gray_small opencv          200 loops, best of 5: 445 usec per loop
# pipeline_gray_small opencv_fast     200 loops, best of 5: 434 usec per loop
# pipeline_gray_small opencv_fastest  200 loops, best of 5: 428 usec per loop
# pipeline_gray_small pil             200 loops, best of 5: 942 usec per loop
# pipeline_gray_small pil_fast        200 loops, best of 5: 727 usec per loop


## batch

./mmbench -n 10 pipeline_color_batch
# pipeline_color_batch opencv          10 loops, best of 5: 265 msec per loop
# pipeline_color_batch opencv_fast     10 loops, best of 5: 145 msec per loop
# pipeline_color_batch opencv_fastest  10 loops, best of 5: 88.4 msec per loop
# pipeline_color_batch pil             10 loops, best of 5: 99.8 msec per loop
# pipeline_color_batch pil_fast        10 loops, best of 5: 96 msec per loop

./mmbench -n 10 pipeline_color_shrink_batch
# pipeline_color_shrink_batch opencv          10 loops, best of 5: 187 msec per loop
# pipeline_color_shrink_batch opencv_fast     10 loops, best of 5: 119 msec per loop
# pipeline_color_shrink_batch opencv_fastest  10 loops, best of 5: 100 msec per loop
# pipeline_color_shrink_batch pil             10 loops, best of 5: 90.7 msec per loop
# pipeline_color_shrink_batch pil_fast        10 loops, best of 5: 101 msec per loop

./mmbench -n 10 pipeline_color_small_batch
# pipeline_color_small_batch opencv          10 loops, best of 5: 103 msec per loop
# pipeline_color_small_batch opencv_fast     10 loops, best of 5: 67.1 msec per loop
# pipeline_color_small_batch opencv_fastest  10 loops, best of 5: 46.2 msec per loop
# pipeline_color_small_batch pil             10 loops, best of 5: 52.4 msec per loop
# pipeline_color_small_batch pil_fast        10 loops, best of 5: 49.7 msec per loop


## gpu

./mmbench -n 100 img_array_normalize_at
# img_array_normalize_at cpu  100 loops, best of 5: 2.57 msec per loop
# img_array_normalize_at gpu  100 loops, best of 5: 486 usec per loop

Setup

conda uninstall -y --force pillow pil jpeg libtiff libjpeg-turbo
pip   uninstall -y         pillow pil jpeg libtiff libjpeg-turbo
conda install -yc conda-forge libjpeg-turbo
CFLAGS="${CFLAGS} -mavx2" pip install --upgrade --no-cache-dir --force-reinstall --no-binary :all: --compile pillow-simd

pip install -U git+git://github.com/lilohuang/PyTurboJPEG.git

About

pytest-like benchmark tool

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published