Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Commit

Permalink
improve README
Browse files Browse the repository at this point in the history
  • Loading branch information
@wuxun-zhang
wuxun-zhang committed Jun 25, 2019
1 parent 8a21876 commit b265dca
Showing 1 changed file with 20 additions and 24 deletions.
44 changes: 20 additions & 24 deletions cpp-package/example/inference/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,36 +32,32 @@ This directory contains following examples. In order to run the examples, ensure

## [imagenet_inference.cpp](<https://github.com/apache/incubator-mxnet/blob/master/cpp-package/example/inference/imagenet_inference.cpp>)

This example demonstrates image classification workflow with pre-trained models using MXNet C++ API. Now this script also supports inference with quantized CNN models generated by Intel® MKL-DNN (see this [quantization flow](https://github.com/apache/incubator-mxnet/blob/master/example/quantization/README.md)). By using C++ API, the latency of most models can get **2%~17%** speedup compared with current Python implementation. The below tables show accuracy and latency of several CNN models.
This example demonstrates image classification workflow with pre-trained models using MXNet C++ API. Now this script also supports inference with quantized CNN models generated by Intel® MKL-DNN (see this [quantization flow](https://github.com/apache/incubator-mxnet/blob/master/example/quantization/README.md)). By using C++ API, the latency of most models will be reduced to some extent compared with current Python implementation.

The following models have been tested on Linux systems. And 50000 images are used to collect the following accuracy numbers.
Most of CNN models have been tested on Linux systems. And 50000 images are used to collect accuracy numbers. Please refer to this [README](https://github.com/apache/incubator-mxnet/blob/master/example/quantization/README.md) for more details about accuracy.

| Model | Source | Dataset | FP32 Accuracy (top-1)| INT8 Accuracy (top-1)|
|:---|:---|---|:---:|:---:|
| [ResNet18-V1](#3) | [Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |70.15%|69.92%|
| [ResNet50-V1](#3) | [Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |76.34%| 75.91% |
| [ResNet101-V1](#3) | [Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 77.33% | 77.05% |
|[Squeezenet 1.0](#4)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)| 56.98%| 52.98%|
|[MobileNet 1.0](#5)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|72.23%|72.03%|
|[MobileNetV2 1.0](#6)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|70.27%|69.70%|
|[Inception V3](#7)|[Gluon-CV](https://gluon-cv.mxnet.io/model_zoo/classification.html)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|77.76% |77.87% |
|[ResNet152-V2](#8)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/resnet/152-layers/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|76.65%|76.36%|
|[Inception-BN](#9)|[MXNet ModelZoo](http://data.mxnet.io/models/imagenet/inception-bn/)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|72.28%|72.20%|
The following performance numbers are collected via using C++ inference API on AWS EC2 C5.12xlarge. The environment variables are set like below:

The following performance numbers are collected by using AWS EC2 C5.18xlarge.
```
export KMP_AFFINITY=granularity=fine,noduplicates,compact,1,0
export OMP_NUM_THREADS=$(vCPUs/2)
export MXNET_SUBGRAPH_BACKEND=MKLDNN
export MXNET_ENGINE_TYPE=NaiveEngine
```
Also users are recommended to use ```numactl``` or ```taskset``` to bind a running process to the specified cores.

| Model | Dataset |C++ latency (imgs/sec) |Python latency (imgs/sec) |
| Model | Dataset |BS=1 (imgs/sec) |BS=64 (imgs/sec) |
|:---|:---|:---:|:---:|
| | |FP32 / INT8 | FP32 / INT8 |
| [ResNet18-V1](#3) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |207.38 / 303.918|197.98 / 271.01|
| [ResNet50-V1](#3) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |96.67 / 165.54|93.20 / 156.65 |
| [ResNet101-V1](#3) | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 54.34 / 85.89| 52.79 / 83.24|
|[Squeezenet 1.0](#4)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)| 182.20 / 421.79| 161.59 / 362.26|
|[MobileNet 1.0](#5)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|300.86 / 344.59|284.16 / 295.25|
|[MobileNetV2 1.0](#6)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|166.05 / 421.25|150.40 / 360.92|
|[Inception V3](#7)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|64.49 / 101.22 | 62.05 / 95.71 |
|[ResNet152-V2](#8)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)|33.08 / 33.64|32.39 / 32.97 |
|[Inception-BN](#9)|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec)| 125.60 / 161.46| 121.47 / 147.63|
| ResNet18-V1 | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |369.00 / 778.82|799.7 / 2598.04|
| ResNet50-V1 | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |160.72 / 405.84|349.73 / 1297.65 |
| ResNet101-V1 | [Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 89.56 / 197.55| 193.25 / 740.47|
|Squeezenet 1.0|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 294.46 / 899.28| 857.70 / 3065.13|
|MobileNet 1.0|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |554.94 / 676.59|1279.44 / 3393.43|
|MobileNetV2 1.0|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |303.40 / 776.40|994.25 / 4227.77|
|Inception V3|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |108.20 / 219.20 | 232.22 / 870.09 |
|ResNet152-V2|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) |52.28 / 64.62|107.03 / 134.04 |
|Inception-BN|[Validation Dataset](http://data.mxnet.io/data/val_256_q90.rec) | 211.86 / 306.37| 632.79 / 2115.28|

The command line to launch inference by this script can accept are as shown below:
```
Expand Down

0 comments on commit b265dca

Please sign in to comment.