Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-1249] Fix Object Detector Performance with GPU #13522

Merged
merged 4 commits into from
Dec 5, 2018
Merged

[MXNET-1249] Fix Object Detector Performance with GPU #13522

merged 4 commits into from
Dec 5, 2018

Conversation

lanking520
Copy link
Member

@lanking520 lanking520 commented Dec 4, 2018 •
edited
Loading

Description

Found Object Detector contains performance problem with GPU which is slower than CPU. The root cause is the long time taken in the post-processing.
@piyushghai @andrewfayres @zachgk

Batchsize: 5
Num of runs: 5

ObjectDetectionbatch_inference_p99 1175.227477ms
ObjectDetectionbatch_inference_p90 1175.227477ms
ObjectDetectionbatch_inference_p50 914.688555ms
ObjectDetectionbatch_inference_average 985.95ms

Num of runs: 5

ObjectDetectionsingle_inference_p99 168.997921ms
ObjectDetectionsingle_inference_p90 168.997921ms
ObjectDetectionsingle_inference_p50 154.573480ms
ObjectDetectionsingle_inference_average 157.10ms

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

piyushghai
piyushghai reviewed Dec 4, 2018
View reviewed changes
scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
@@ -94,13 +94,13 @@ class ObjectDetector(modelPathPrefix: String,
def objectDetectWithNDArray(input: IndexedSeq[NDArray], topK: Option[Int])
: IndexedSeq[IndexedSeq[(String, Array[Float])]] = {

val predictResult = predictor.predictWithNDArray(input)(0)
val predictResult = predictor.predictWithNDArray(input)(0).asInContext(Context.cpu())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, can we apply this to ImageClassifier class as well ?
Maybe this can provide a speed boost there as well ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same nitpick I had on imageclassifier. A comment explaining this wouldn't hurt since it's not obvious why we're doing this.

piyushghai
piyushghai approved these changes Dec 5, 2018
View reviewed changes
Copy link
Contributor

@piyushghai piyushghai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me. 💯 🥇
Please also add comments where you are the parArray. That's a nice optimization that might not be very obvious while reading the code :)

andrewfayres
andrewfayres reviewed Dec 5, 2018
View reviewed changes
scala-package/infer/src/main/scala/org/apache/mxnet/infer/Classifier.scala Show resolved Hide resolved
scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
@@ -94,13 +94,13 @@ class ObjectDetector(modelPathPrefix: String,
def objectDetectWithNDArray(input: IndexedSeq[NDArray], topK: Option[Int])
: IndexedSeq[IndexedSeq[(String, Array[Float])]] = {

val predictResult = predictor.predictWithNDArray(input)(0)
val predictResult = predictor.predictWithNDArray(input)(0).asInContext(Context.cpu())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same nitpick I had on imageclassifier. A comment explaining this wouldn't hurt since it's not obvious why we're doing this.

andrewfayres
andrewfayres approved these changes Dec 5, 2018
View reviewed changes
Copy link
Contributor

@andrewfayres andrewfayres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lanking520 lanking520 merged commit 1c8972c into apache:master Dec 5, 2018
TaoLv added a commit that referenced this pull request Dec 6, 2018
@TaoLv
Revert "[MXNET-1249] Fix Object Detector Performance with GPU (#13522)"
30f2bde 
This reverts commit 1c8972c.
TaoLv added a commit that referenced this pull request Dec 6, 2018
@TaoLv
Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated L…
29885c5 
…icense file" (#13558)

* Revert "Chi_square_check for discrete distribution fix (#13543)"

This reverts commit cf6e8cb.

* Revert "Updated docs for randint operator (#13541)"

This reverts commit e0ff3c3.

* Revert "Simplifications and some fun stuff for the MNIST Gluon tutorial (#13094)"

This reverts commit 8bbac82.

* Revert "Fix #13521 (#13537)"

This reverts commit f6b4665.

* Revert "Add a retry to qemu_provision (#13551)"

This reverts commit f6f8401.

* Revert "[MXNET-769] Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (#13531)"

This reverts commit bd8e0f8.

* Revert "[MXNET-1249] Fix Object Detector Performance with GPU (#13522)"

This reverts commit 1c8972c.

* Revert "Fixing a 404 in the ubuntu setup doc (#13542)"

This reverts commit cb0db29.

* Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file (#13478)"

This reverts commit 40db619.
@lanking520 lanking520 deleted the fixSSD branch December 13, 2018 01:08
zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@lanking520 @zhaoyao73
[MXNET-1249] Fix Object Detector Performance with GPU (apache#13522)
229f7f6 
* Reduce post processing time

* fix ssd

* fix the CI

* add comments
zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@TaoLv @zhaoyao73
Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated L…
ad7ba29 
…icense file" (apache#13558)

* Revert "Chi_square_check for discrete distribution fix (apache#13543)"

This reverts commit cf6e8cb.

* Revert "Updated docs for randint operator (apache#13541)"

This reverts commit e0ff3c3.

* Revert "Simplifications and some fun stuff for the MNIST Gluon tutorial (apache#13094)"

This reverts commit 8bbac82.

* Revert "Fix apache#13521 (apache#13537)"

This reverts commit f6b4665.

* Revert "Add a retry to qemu_provision (apache#13551)"

This reverts commit f6f8401.

* Revert "[MXNET-769] Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (apache#13531)"

This reverts commit bd8e0f8.

* Revert "[MXNET-1249] Fix Object Detector Performance with GPU (apache#13522)"

This reverts commit 1c8972c.

* Revert "Fixing a 404 in the ubuntu setup doc (apache#13542)"

This reverts commit cb0db29.

* Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file (apache#13478)"

This reverts commit 40db619.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@zachgk zachgk zachgk approved these changes

@piyushghai piyushghai piyushghai approved these changes

@andrewfayres andrewfayres andrewfayres approved these changes

@nswamy nswamy Awaiting requested review from nswamy nswamy is a code owner

@yzhliu yzhliu Awaiting requested review from yzhliu yzhliu is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lanking520 @zachgk @piyushghai @andrewfayres