* Improve order of execution of install scripts.
Remove uneccesary environment variable. /usr/loca/lib already in ld.so.conf*

* Fix

* Add primitive cache for mkldnn sum

* fix cpp test failure

* update to mkldnn v0.19-rc

* update cpp test

* update mklml library

* bump mkldnn version to v0.19

* fix cmake for win and mkl

* Update build_windows.py

update MKL_ROOT

* Update Jenkins_steps.groovy

fix nomkldnn issue.

* Update build_windows.py

change MKL_ROOT

* msshadow_blas

* update mshasow

* Update DownloadMKLML.cmake

windows md5

* Update DownloadMKLML.cmake

windows md5

* Update Jenkins_steps.groovy

export MKLROOT

* Update Jenkins_steps.groovy

 powershell 'cd $Env:MKL_ROOT'

* Update Jenkins_steps.groovy

trigger MKL_ROOT test

* Update Jenkins_steps.groovy

test MKL_ROOT line 552

* Update Jenkins_steps.groovy

update 538 $Env:MKL_ROOT

* update mshadow

* revert gitmodules

* remove comments

* remove test code

* remove test powershell

* add 3d softmax

* fix

* handle req type

* clean code

* remove check

* check axis

* retrigger ci

* conv+act fusion

* fix mobilenetv2

* fix lint

* Add comment

* Fix build

* trigger ci

* Fix CutGraphInputs

* Run CI

* run ci

* trigger

* bump up the cuDNN version

* fall back to fp32

* add fallback data type

* remove the duplicate one

* reenable the memory check

* add comment

* fix the linter

* merge the conflict

* retrigger CI

…e#14748)

* sync examples with latest changes in Horovod

* update README

…#14830)

* add env var to control accumulation option

* trial

* try with ci change

* recover all test_operator

* add gpu coverage

* fix order

* address comments

* extra disclaimer on model accuracies

* improve mkldnn document

* fix

* enable fusion

* adjust table

* fix comments

* promote mxnet-mkl package

* Update docs/tutorials/mkldnn/MKLDNN_README.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/install/index.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/install/index.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/install/index.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/install/index.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/tutorials/mkldnn/operator_list.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/faq/perf.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/faq/perf.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/tutorials/mkldnn/operator_list.md

Co-Authored-By: TaoLv <[email protected]>

* Update docs/tutorials/mkldnn/operator_list.md

Co-Authored-By: TaoLv <[email protected]>

* fix markdown table

* fix comments

* Update docs/faq/env_var.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/tutorials/mkldnn/MKLDNN_README.md

Co-Authored-By: Aaron Markham <[email protected]>

* change name of env variable

* retrigger ci

* Update env_var.md

* Silence excessive mkldnn logging output on tests.
http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-14940/runs/1/nodes/283/steps/749/log/?start=0

* As per Peng Zhao and Tao Lv suggestion

* [MXNet-1334][Fit API]base class for estimator and eventhandler (apache#14346)

* base class for estimator and eventhandler

* add license

* add event handlers

* fix pylint

* improve arg check

* fix pylint

* add unit tests

* Fixed issue where the estimator was printing beyond the dataset size … (apache#14464)

* Fixed issue where the estimator was printing beyond the dataset size for the last batch

* Added comments

* Nudge to CI

* [MXNet-1349][Fit API]Add validation support and unit tests for fit() API (apache#14442)

* added estimator unittests

* add more tests for estimator

* added validation logic

* added error handlers, unittests

* improve val stats

* fix pylint

* fix pylint

* update unit test

* fix tests

* fix tests

* updated metrics, val logic

* trigger ci

* trigger ci

* update metric, batch_fn error handler

* update context logic, add default metric

* [MXNet-1340][Fit API]Update train stats (apache#14494)

* add train history

* update history

* update test

* avoid calling empty methods

* remove train history object

* fix pylint

* add unit test

* fix test

* update categorize handlers

* [MXNet-1375][Fit API]Added RNN integration test for fit() API (apache#14547)

* Added RNN integration test for fit() API

* Addressed review comments: change in JenkinFile, tmp directory, ctx with condense if/else, renamed imports

* CPU test doesn't require nvidiadocker container

* Modified the structure by removing the redundant code

* [MXNet-1343][Fit API]Add CNN integration test for fit() API (apache#14405)

* added cnn intg tests for fit api

* updated cnn intg tests

* added functions for nightly test

* updated runtime_function

* updated intg tests

* updated init, datapath, refs

* added validation data

* update cpu test

* refactor code

* updated context

* [MXNET-1344, 1346][FIT API] Retrieve Batch size and Logging verbose support for Gluon fit() API (apache#14587)

* Retrieve Batch size and Logging verbose support for Gluon fit() API

* NIT changes

* Addressed review comments: shifted the batch size code to a separate method, sentence correction

* Modified unittest

* removed redundant parameter

* Resolve CI test failure

* only support DataLoader for now, future PRs will include DataIter to DataLoader converter

* Get the number of samples from shape attribute instead of length due to low space complexity

* Simplified batch size retrieval code

* removed batch_size parameter from fit() method and fixed the tests

* Verbose exception handling

* Assigning constant to a verbose

* Modified exception message

* Resolved undefined class reference

* Addressed review comments: Modified verbose level names, docs, variable names

* Update estimator.py

* move estimator to contrib (apache#14633)

* move to gluon contrib (apache#14635)

* [Fit API] improve event handlers (apache#14685)

* improve event handlers

* update tests

* passing weakref of estimator

* fix unit test

* fix test

* fix pylint

* fix test

* fix pylint

* move default metric logic

* combine nightly tests

* [MXNET-1396][Fit-API] Update default handler logic (apache#14765)

* move to nightly for binaries

* update default handler

* fix pylint

* trigger ci

* trigger ci

* [Fit API] update estimator (apache#14849)

* address comments

* add comment

* check available context

* fix bug

* change cpu check

* [Fit-API] Adress PR comments (apache#14885)

* address comments

* update checkpoint

* test symbol save

* address comments

* add resume

* update doc and resume checkpoint

* update docs

* trigger ci

* trigger ci

…4952)

* update mklndd.md

* Update MKLDNN_README.md

simplify the windows build instruction

* Update MKLDNN_README.md

simply steps.

* Update MKLDNN_README.md

Simply VS 2017 part

* modify vs2017

* Update MKLDNN_README.md

remove numpy hardcode

* LF replace

* -DUSE_MKLDNN=1

* Update MKLDNN_README.md

resolve the conflict

* quantization

…g process (apache#14683)

…opied to MXNet (apache#14948)

* Initial commit

* update

* Update test_ndarray.py

* Retrigger

* add inverse cpu

* add comment

* add inverse backward cpu

* add inverse gpu

* able to compile

* fix

* fix

* guard for lower version cuda

* update docs

* update docs

* fix misaligned memory

* add test

* fix lint

* fix android

* fix indent

* change transfer gradient

* fix

* refactor test

* delete unnecessary copy

* trigger CI

* fix test

* Initial commit

* Website stuff

* Address comments from Tao

* Update docs/_static/mxnet-theme/index.html

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Update docs/install/index.md

Co-Authored-By: Aaron Markham <[email protected]>

* Retrigger

)

This reverts commit 9f451fb.

…e#14346)

* base class for estimator and eventhandler

* add license

* add event handlers

* fix pylint

* improve arg check

* fix pylint

* add unit tests

apache#14464)

* Fixed issue where the estimator was printing beyond the dataset size for the last batch

* Added comments

* Nudge to CI

…API (apache#14442)

* added estimator unittests

* add more tests for estimator

* added validation logic

* added error handlers, unittests

* improve val stats

* fix pylint

* fix pylint

* update unit test

* fix tests

* fix tests

* updated metrics, val logic

* trigger ci

* trigger ci

* update metric, batch_fn error handler

* update context logic, add default metric

* add train history

* update history

* update test

* avoid calling empty methods

* remove train history object

* fix pylint

* add unit test

* fix test

* update categorize handlers

…#14547)

* Added RNN integration test for fit() API

* Addressed review comments: change in JenkinFile, tmp directory, ctx with condense if/else, renamed imports

* CPU test doesn't require nvidiadocker container

* Modified the structure by removing the redundant code

…4405)

* added cnn intg tests for fit api

* updated cnn intg tests

* added functions for nightly test

* updated runtime_function

* updated intg tests

* updated init, datapath, refs

* added validation data

* update cpu test

* refactor code

* updated context

…upport for Gluon fit() API (apache#14587)

* Retrieve Batch size and Logging verbose support for Gluon fit() API

* NIT changes

* Addressed review comments: shifted the batch size code to a separate method, sentence correction

* Modified unittest

* removed redundant parameter

* Resolve CI test failure

* only support DataLoader for now, future PRs will include DataIter to DataLoader converter

* Get the number of samples from shape attribute instead of length due to low space complexity

* Simplified batch size retrieval code

* removed batch_size parameter from fit() method and fixed the tests

* Verbose exception handling

* Assigning constant to a verbose

* Modified exception message

* Resolved undefined class reference

* Addressed review comments: Modified verbose level names, docs, variable names

* Update estimator.py

* improve event handlers

* update tests

* passing weakref of estimator

* fix unit test

* fix test

* fix pylint

* fix test

* fix pylint

* move default metric logic

* combine nightly tests

* move to nightly for binaries

* update default handler

* fix pylint

* trigger ci

* trigger ci

* address comments

* add comment

* check available context

* fix bug

* change cpu check

* address comments

* update checkpoint

* test symbol save

* address comments

* add resume

* update doc and resume checkpoint

* update docs

* trigger ci

* trigger ci

* Add dtype for large int dtypes

* Fix bug

fix lint

fix lint

fix bug

further accelerate

fix

fix bug

fix bug

[Dependency Update] CUDA10.1 Support

* update

* modify the link

* Add STL checks via -D_GLIBCXX_ASSERTIONS in debug mode

* Also add to Makefile

* Fix warning (fpermissive) sleep definition not available

* Beginning of AMP

* Optimize noop cast

* More operations added

* Backward cast

* Adding AMPCast and AMPMultiCast

* Fix some of lint

* Changed symbol wrapper to handle hidden inputs
Added PoC of dynamic loss scaling

* Moved back to dmlc/tvm repo

* fix counter reset to increase loss scale every 2k iterations

* Fix indentation

* Add contrib from symbol and ndarray to symbol list

* Adding where to widest type cast

* Do not cast in imperative mode on CPU context

* Update dmlc-core to fix unittests

* Fix wrapper metadata, fix self handling

* Blacklist sync batchnorm (since its implementation is FP32 only)

* Fix lint

* Enable losses to be tuple

* Get rid of AMP handle

* Add scaling to Output functions

* Fix pylint

* Update dmlc-core

* Changing prints in AMP to logging.info

* NNVM -> MXNet for FInferShape

* Bring the inplaceidentity fix to copied pass from NNVM

* Added tutorial for AMP

* Making Windows compiler happy

* Fixes to tutorial

* More fixes

* Fix lint

* Fix

* Add amp/index.md to whitelist for tutorial tests

* Whitelisting cuDNN RNN

* Manual unscale

* _internal functions wrapping

* Make SymbolFunctor from Symbol

* Fix the type infer function of AMP multicast

* Added ability to override casting lists

* Making clang-tidy and pylint happy

* More cleaning

* Making clang-tidy really happy

* remove amp_cast and amp_multicast before saving the model

* Changes from review

* Add RemoveAmpCast in a separate c_api function, add the option in symbol.save

* add remove_amp_cast option (True by default) to everyway of saving symbol

* Fix

* First stab at adding the gray list

* More ops added

* Adding the rest of the functions

* Improvements to AMP test

* Changing of names and changing wrapping

* Moving to contrib

* Modifying tutorial for contrib AMP

* Removing non existent functions

* Fix import in test

* Fix lint

* Added new functions

* Added assert

* Fix the unknown ndim in PlanMemory pass

* Moving back to FP16_FUNCS and FP16_FP32_FUNCS

* Removing unnecessary ops

* Adding ops that exist only in some build configurations and removing
tests checking that AMP lists contain only existing ops

* Removing warning when not every function was found during AMP init
because of functions being available only in specific configurations

* Add tests and doc

* Fix the CPU version of all_finite

* Adding test cases for all_finite operator

* Add new operators

* Fix

* change check and shape_is_known

* rever some changes

* revert

* revert

* revert

* add test

* add more tests

* update test dot

* fix test

* update reduce axes

* fix lint

* update check

* fix lint

* address comments

* remove invalid test case

* run all tests

* update test case

* Fix

* Fix

* Retrigger

…pache#14944)

* use float32 to store the reduction result of float16

enable safe accumulation

fix bug

fix

* update test for safe_accumulate

* fix

…ised) mathematic computation (apache#14893)

* mkl_func test with erf&log op, build success~

* fix lint and build issues

* Try to add support to sparse array

* fix build

* add functions

* Fix review comments

* remove unecessary code

* Update test case

* minor fix

* move the position of MKL_Compute

* mkl_func test with erf&log op, build success~

* fix lint and build issues

* Try to add support to sparse array

* fix build

* Fix review comments

* remove unecessary code

* Update test case

* minor fix

* add functions

* move the position of MKL_Compute

* fix cpplint

* cpp lint

* trigger ci

* address comments

* coding style

* enable layernorm

* fix windows build

* revert changes to FComputeEx

* int -> index_t

* remove workspace

* fix lint

* clean code

…pache#15031)

* [Quantization] Support zero-size tensor input for quantization flow

* Comment out quantized_act and quantized_sum

* retrigger CI

* Add test cases

…#14095)

* Optimize move semantics of NodeEntry
apache/tvm#2576
Making copies of shared_ptr is more expensive than moving.
This PR reduces lock contention by using move semantics in NNVM nodes
making also more convenient to construct NodeEntry classes in the code
due to the added ctors.

Update NDarray with NodeEntry constructors and refine initializer lists.

Sync gradient.cc with tvm

* Remove additional calls to NodeEntry in emplace_back

* refine patch

* Fix lint

)

* Fix warning

* CR comments

* Fix windows CI

* as per Haojin2 request, moving to op namespace

* fix crash in random_shuffle caused by int overflow

* add unit test

* add comment

* remove small random test to avoid CI failure

* add cpu deformable_im2col

* fix doc string

* fix lint

* add consistency check

* fix code style

* add cpu deformable_col2im, deformable_col2im_coord

* update consistency test

* unify variable names

* fix lint

* update to index_t

…pache#15038)

* fix type inconsistent when using C++ API to load params file

* add test case

* fix cpplint

* address comment

* retrigger CI

* fix comments

* modify ci_test

* fix indentation

* add cpu deformable_psroi_pooling forward

* add cpu deformable_psroi_pooling backward

* add consistency checks

* fix nullptr

* fix code style

* fix lint

* fix code style

* update to index_t

* fix lint

* fix compile

* Fix broken build with cython 0.28

* Fix setup.py to be compatible with cython 0.28

* Fix broken cython ndarray module

* Revised comments

* Replace hard coded library path with one obtained by find_lib_path

* Add documentation for MXNET_ENABLE_CYTHON and MXNET_ENFORCE_CYTHON

* Add cython build to CI

* Fix for cython CI

* Adjust python environment for cython CI

* Add make variables to set python executable

* Fix typo

* Fix nnvm include path

* Does not use ccache for cython

* Fix issues with the wildcards in the library list in Jenkinsfile

* Fix issues with the wildcards in the library list in Jenkinsfile (continued)

* Fix issues with the wildcards in the library list in Jenkinsfile (continued)

* Intentionally introduce a bug to check that the tests actually runwith cython

* Remove the intentionally introduced bug

* Update installation doc

* Retrigger CI

* Run cython CI in ubuntu environment instead of CentOS environment

* Commit a missed file

* Fix a bug in check_cython

* Refine environments for cython CI

* Restore unrelated changes

* Fix a problem occurring when the cython modules for python 2 and 3 are built successively

* Trigger CI

* Pin the cython version in the CI

* Catch up apache#11320

* Remove optional arguments unused after apache#11320

* Trigger CI

* Trigger CI

* Trigger CI

* Remove unnecessary stype argument from the NDArrayBase constructor

* Revise confusing initialization of `_ndarray_cls`

* Add cython build for python3 in CI

* Fix misplaced cython build in CI

* Adjust CI environments for cython

* Fix invalid path for cython generated .so files in cmake build

* Revert invalid fix

* Revise docs

* Revise check_cython

* Temporaily use make instead of cmake for debugging

* Temporal changes for debugging

* Temporal changes for debugging

* Temporaily use ctypes instead of cython modules for debugging

* Temporaily disable ccache for debugging

* Temporaily use make (DEV = 0) instead of cmake for debugging

* Temporaily disable cudnn for debugging

* Restore temporal changes

* Temporarily disable coverage report

* Adapt to Jenkinsfile_utils

* Adapt to Jenkinsfile_utils (cont.)

* Restore unrelated changes

* Restore temporal changes

* Resolving conflict

* Test with the cmake build is removed

* Add MXNET_ENABLE_CYTHON=0 to tensorrt test

* Fix typo

* Trigger CI

* Trigger CI

* Adapt to Jenkinsfile refactoring

* Adapt to Jenkinsfile refactoring (cont.)

* Trigger CI

* Trigger CI

* Stash missing cython modules

* Trigger CI

* CMake build of cython modules without unit tests

* Fix typo

* Trigger CI

* Fix a mistake introduced in merging process

* trigger test

* Update Jenkinsfile_utils.groovy

* Trigger CI

* Trigger CI

* Trigger CI

* Trigger tests

* Trigger tests

* Trigger tests

* Trigger tests

* Trigger tests

* Trigger tests

* Trigger tests

* Trigger tests

…u) (apache#14713)

* trigger the ci

* integrate mkldnn rnn fp32 inference(LSTM and vRNN with tanh and relu)

* fix bug about comparison between signed and unsigned integer expressions

* fix unix-gpu issue

* fix unix gpu bug

* fix unix-gpu issues

* fix some comments

* fix issue

* fix comment

* rename `cached` to `initialized`

* support IType

* TODO for MKLDNN GRU

* fix bugs in memory adjustment

* Reformat TODO for MKLDNN GRU

* Reserve original RNN path

* Remove MKLDNN GRU

* Fix bug for rnn forward

* Remove `__CUDAACC__`

* Move `RNNStatefulComputeCPU` to rnn.cc

* Remove redundent macro of `__CUDACC__`

* Remove the last macro `__CUDACC__` from rnn*

* Allow releasing all gpu memory

* fix white space

* stuck ci checks

* Fix whitespace

* Rename release_all -> empty_cache and provide documentation

* fix indentation

* Rename c_api's MXStorageReleaseAll -> MXStorageEmptyCache and clarify documention

* nudge ci

* Update context.py

* GPU RNN to use TempSpace resource for workspace.

* Trigger CI.

* Fix syntax error after merge.

* Implement the index_array operator

* Add index_array operator tests

* Add index_array operator GPU tests

* Add the index_array operator to the Python docs autosummary

* Add the author of the index_array operator to CONTRIBUTORS.md

* Make index_array compatible with zero-dim and zero-size arrays

Changes the implementation of index_array to be compatible with the
recently merged support for zero-dim and zero-size arrays. Resolves the
incompatibilities with apache#14661.

* Fix the index_array gradient checks in the unit tests

In the previous implementation, the output gradient had an incorrect
shape. This commit fixes the shapes and makes the tests more readable.

* Add zero-dim and zero-size array tests for index_array

* Use mxnet::Tuple<int> instead of TShape for the axes parameter

* Fix incorrect array indexing in index_array

Solves access violations when compiling with MSVC++ 14.0.

* Avoid copying the input shape array in the index_array shape function

* Add unknown shape handling to index_array

* Use SHAPE_ASSIGN_CHECK to assign the shape in index_array

* Remove the redundant index_array GPU tests from test_operator_gpu.py

* Move the index_array tests into a single function (test_index_array)

* Use @mx.use_np_compat instead of mx.np_compat in index_array op tests

* Remove the use of template specialization for IndexArrayForward

* Add the index_array operator to the AMP symbol list

* Retrigger CI

* Change np_compat to np_shape

* Fix scala

* Fix pylint

* Add examples and fix documentation

* Fix doc

* More doc

* Rename np_compat to np_shape in test_operatory.py

* Rename in ndarray.cc

* Improve FC perf when no_bias=False

* Add Issue number in comment

* Correct req

* add higher order gradient support for log, log10, log2

* add tests

* address comments

* simplify NodeEntry creation.

* address comments

* update comment to avoid confusion.

…pache#15065)

* changed http://data.dmlc.ml/ links to http://data.mxnet.io/

* Update get_data.R

* Update get_inception_model.sh

* Update CatsDogsFinetune.Rmd

* Update README.md

Updated pretrained ImageNet model links

)

This reverts commit 6cf964a.

* fix mkldnn deconvolution forward with bias

* Rerun CI

…pache#14946)

* Change mx_test_utils.list_gpus to mx.context.num_gpus where possible
apache#14926

* Remove mx.test_utils.list_gpus

* revert accidental unrelated dmlc-core changes

* Fix basic_layers.py import

* Fix typo

* restore list_gpus() in tests_utils for compat

* restore dmlc-core

* nag_mp

* doc

* reuse sgd updates where convenient