Deploy BERT model - Script (#1237) (#1357)

* Add example script to deploy BERT

* Add options to better measure performance

* Allow specification of path for exported model

* Add option to use custom graph pass

* Add optimization for MHA in custom graph pass

* Correct bug with input shapes in optimize_for

* correct typo

* fix lint

* fix lint

* Add documentation

* Add documentation for using deploy script

* Correct typo/add spaces in documentation

* Add setup.py to compile pass, update documentation

* Fix bug in path to include dir & fix pylint

* Add unitest for deploy bert script

* change CUDA version in wheel

* test latest wheel

* change path to custom pass library

* fixing trigger custom pass compilation

* fix lint

* fix lint

* Update mxnet pip version

* Only GPU versions changed

* fix lint

* change wheel to include mkl headers

* lint docstring

* remove debug print

* change include paths

* lint

* debugging lib_api.h

* debugging lib_api.h

* debugging

* Disable test for now

* skip test if mxnet_version < 1.7.0

* use pytest.mark.skipif to skip test

* test only BERT-base (fp16/fp32, SST/QA, embeddings) to avoid timeout

Co-authored-by: Leonard Lausen <[email protected]>

Co-authored-by: Leonard Lausen <[email protected]>

env/gpu/py3-master.yml

@@ -33,7 +33,7 @@ dependencies:
     - flaky==3.6.1
     - flake8==3.7.9
     - mock<3
-    - https://repo.mxnet.io/dist/python/cu100/mxnet_cu100-1.6.0-py2.py3-none-manylinux1_x86_64.whl
+    - https://repo.mxnet.io/dist/python/cu100/mxnet_cu100-1.7.0b20200809-py2.py3-none-manylinux2014_x86_64.whl
     - scipy==1.3.2
     - regex==2020.4.4
     - nltk==3.4.5