enable TensorRT integration with cpp api

[haohuanw]

Description

This PR enables using TensorRT through cpp api

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• [ x ] Changes are complete (i.e. I finished coding on this PR)
• [ x ] All changes have test coverage:
• [ x ] Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • I have updated the inference test but I don't think this is part of the ci task since I am not sure what is the easiest way to add this test in ci.
• [ x ] Code is well-documented:
• [ x ]For new C++ functions in header files, their functionalities and arguments are documented.
• [ x ]For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
• [ x ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

• [ X ] inception_inference.cpp : update example for bind with TRT

./inception_inference --symbol "./model/Inception-BN-symbol.json" --params "./model/Inception-BN-0126.params" --synset "./model/synset.txt" --mean "./model/mean_224.nd" --image "./model/dog.jpg" --enableTRT
...
[17:53:45] /opt/workspace/incubator-mxnet/cpp-package/example/inception_inference.cpp:377: The model predicts the input image to be a [ pug, pug-dog ] with Accuracy = 0.987554

root@0f0b3e022627:/opt/workspace/incubator-mxnet/build/private/cmake/cpp-package/example# LD_LIBRARY_PATH=/opt/workspace/incubator-mxnet/build/lib/:$LD_LIBRARY_PATH ./inception_inference --symbol "./model/Inception-BN-symbol.json" --params "./model/Inception-BN-0126.params" --synset "./model/synset.txt" --mean "./model/mean_224.nd" --image "./model/dog.jpg"            
...
[17:53:52] /opt/workspace/incubator-mxnet/cpp-package/example/inception_inference.cpp:377: The model predicts the input image to be a [ pug, pug-dog ] with Accuracy = 0.986955

root@0f0b3e022627:/opt/workspace/incubator-mxnet/build/private/cmake/cpp-package/example# LD_LIBRARY_PATH=/opt/workspace/incubator-mxnet/build/lib/:$LD_LIBRARY_PATH ./inception_inference --symbol "./model/Inception-BN-symbol.json" --params "./model/Inception-BN-0126.params" --synset "./model/synset.txt" --mean "./model/mean_224.nd" --image "./model/dog.jpg" --gpu
...
[17:54:03] /opt/workspace/incubator-mxnet/cpp-package/example/inception_inference.cpp:377: The model predicts the input image to be a [ pug, pug-dog ] with Accuracy = 0.986955

• [ x ] symbol.hpp: added some api, these are tested when running inception_inference
• [ x ] contrib.h: new file, added functions for binding TRT, these are tested when running inception_inference

Comments

• If this change is a backward incompatible change, why must this change be made.
  • This change is for enabling ability to use TensorRT integration through C++ API

[haohuanw]

@KellenSunderland @Caenorst

[haohuanw]

ping! anyone could do a quick review?

[Caenorst]

We should probably add those constants in src/operator/subgraph/tensorrt/tensorrt-inl.h to keep track

[haohuanw]

You mean duplicate them in the tensorrt-inl.h as well?

[Caenorst]

I was more thinking putting them in tensorrt-inl.h and including the file in this file (in case we need it somewhere).

[haohuanw]

hmmmm, tensorrt-inl.h is not in the default header folder. if we do that we probably need an new header file?

[Caenorst]

I see, ignore my original comment then

[KellenSunderland]

@szha Hey Sheng, have we had contrib namespaces in C++ before? The intent of this code basically aligns with the intent of having a contrib level python API. Does namespacing it out like this make sense to you?