MKLDNN can be turned off with env var #12058

azai91 · 2018-08-07T03:17:55Z

Description

MKLDNN can be turned off with USE_MKLDNN env flag

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

add fallback logic in storagetype to fallback if USE_MKLDNN var set

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

azai91 · 2018-08-07T18:42:04Z

python example/image-classification/benchmark_score.py
INFO:root:batch size  1, dtype float32, images/sec: 18.603268
INFO:root:batch size  2, dtype float32, images/sec: 28.085829
INFO:root:batch size  4, dtype float32, images/sec: 40.856433
INFO:root:batch size  8, dtype float32, images/sec: 47.194993
INFO:root:batch size 16, dtype float32, images/sec: 50.532706
INFO:root:batch size 32, dtype float32, images/sec: 51.747587

USE_MKLDNN=0 python example/image-classification/benchmark_score.py
INFO:root:batch size  1, dtype float32, images/sec: 5.174663
INFO:root:batch size  2, dtype float32, images/sec: 5.668587
INFO:root:batch size  4, dtype float32, images/sec: 5.831225
INFO:root:batch size  8, dtype float32, images/sec: 5.870498
INFO:root:batch size 16, dtype float32, images/sec: 5.856749
INFO:root:batch size 32, dtype float32, images/sec: 5.871009

haojin2 · 2018-08-07T21:52:52Z

src/operator/tensor/elemwise_binary_op_basic.cc

-  if (dev_mask == mshadow::cpu::kDevMask
+  if (!MKLDNNEnvSet()) {
+    *dispatch_mode = DispatchMode::kFComputeFallback;
+  } else if (dev_mask == mshadow::cpu::kDevMask


Please also align the 2 lines below with this one.

pengzhao-intel · 2018-08-08T00:04:52Z

@TaoLv @ZhennanQin

TaoLv · 2018-08-08T01:06:30Z

Maybe the env variable should start with MXNET_, like what we did for other MXNet env variables. Please also add document for it in this page: https://mxnet.incubator.apache.org/faq/env_var.html

Another question, what's the behavior of enabling this variable when MKL-DNN is not compiled into MXNet? Should we give a proper warning or tip for that?

haojin2 · 2018-08-08T01:14:18Z

src/operator/nn/mkldnn/mkldnn_base-inl.h

@@ -137,6 +137,10 @@ static inline bool SupportMKLDNN(const NDArray &input) {
      && SupportStorageMKLDNN(input.storage_type());
 }

+static inline bool MKLDNNEnvSet() {
+  return dmlc::GetEnv("USE_MKLDNN", true);


Seems like you're setting the default to be true, would you please also add corresponding documentation for this env var on the webpage so that we're not causing surprise for users?

ZhennanQin

If we assume mkldnn is the only backend for kFComputeEx on cpu, shouldn't we make the change inside the caller of InferStorageType? Then we don't need to change it in every ops.

ZhennanQin · 2018-08-08T04:07:13Z

src/operator/nn/activation.cc

@@ -116,6 +116,10 @@ inline static bool ActivationStorageType(const nnvm::NodeAttrs& attrs,
  if (dev_mask == mshadow::cpu::kDevMask && SupportMKLDNNAct(param)) {
    *dispatch_mode = DispatchMode::kFComputeEx;
  }
+  if (!MKLDNNEnvSet()) {


Don't we need to check (dev_mask == mshadow::cpu::kDevMask) here?

lupesko

Looking good. Some notes:

I suggest we align the env var name with the conventional prefix "MXNET_"
As part of the PR, please also update env_var.md where all env vars are documented.

lupesko · 2018-08-08T05:00:07Z

src/operator/nn/activation.cc

@@ -158,6 +162,10 @@ inline static bool BackwardActStorageType(const nnvm::NodeAttrs& attrs,
  if (dev_mask == mshadow::cpu::kDevMask && SupportMKLDNNAct(param)) {
    *dispatch_mode = DispatchMode::kFComputeEx;
  }
+  if (!MKLDNNEnvSet()) {
+    *dispatch_mode = DispatchMode::kFComputeFallback;
+    return ret;


Is this return statement redundant due to the statement in line 170?

azai91 · 2018-08-08T20:10:25Z

@ZhennanQin there are some fcomputeex ops that are not mkldnn (customop is one example). something I did in another PR (#12019) was set an attr on all MKLDNN operators. if we wait for that merge we could then check if the operator is mkldnn and if the flag is set and fallback before calling inferstorage type.

azai91 · 2018-08-14T04:50:11Z

@pengzhao-intel review / approve please

pengzhao-intel · 2018-08-15T04:27:52Z

Looks good.

One tip: Could we add an info message to let the user know the MKLDNN is disabled now by "MXNET_MKLDNN_ENABLED=0" and it can be switched on with env setting again?

I afraid the user will forget this env setting in some bash file and then can't get the correct performance.

azai91 · 2018-08-15T04:38:05Z

No problem. Do you know the best place to put that info? I asked around last week and there doesn't seem to be a consensus.

pengzhao-intel · 2018-08-15T05:27:38Z

I look into the code. The message should be in the top level of OP but seem no proper place now.
So, how about waiting the #12019 is merged and then we only need to change InferStorageType where we can write the message too?

pengzhao-intel · 2018-08-15T15:35:46Z

Looks fine now even the fallback log is appeared many times :)
The good thing is the user would not miss this important info.

[patric@mlt-skx084 image-classification]$ export MXNET_MKLDNN_ENABLED=0
[patric@mlt-skx084 image-classification]$ python benchmark_score.py
/home/patric/.local/lib/python2.7/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
INFO:root:network: alexnet
INFO:root:device: cpu(0)
/home/patric/develop/mxnet-mkldnn-env/python/mxnet/module/base_module.py:66: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
  warnings.warn(msg)
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_filter" : 96, "stride" : (4, 4), "kernel" : (11, 11), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Activation
input storage types = [default, ]
output storage types = [default, ]
params = {"act_type" : relu, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = LRN
input storage types = [default, ]
output storage types = [default, default, ]
params = {"knorm" : 2, "beta" : 0.75, "nsize" : 5, "alpha" : 0.0001, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Pooling
input storage types = [default, ]
output storage types = [default, default, ]
params = {"stride" : (2, 2), "pool_type" : max, "kernel" : (3, 3), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_filter" : 256, "pad" : (2, 2), "kernel" : (5, 5), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_filter" : 384, "pad" : (1, 1), "kernel" : (3, 3), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_filter" : 256, "pad" : (1, 1), "kernel" : (3, 3), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = FullyConnected
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_hidden" : 4096, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../operator/../common/utils.h:447:
Storage type fallback detected:
operator = FullyConnected
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_hidden" : 1000, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:26:40] src/executor/../common/utils.h:447: MXNET_MKLDNN_ENABLED flag is off. You can re-enable by setting MXNET_MKLDNN_ENABLED=1
INFO:root:batch size  1, dtype float32, images/sec: 82.074852
INFO:root:batch size  2, dtype float32, images/sec: 79.050781
INFO:root:batch size  4, dtype float32, images/sec: 96.430868
INFO:root:batch size  8, dtype float32, images/sec: 110.427058
INFO:root:batch size 16, dtype float32, images/sec: 117.611687
INFO:root:batch size 32, dtype float32, images/sec: 114.142417

eric-haibin-lin · 2018-08-15T20:11:18Z

src/executor/attach_op_execs_pass.cc

+                                       "You can re-enable by setting MXNET_MKLDNN_ENABLED=1");
+#endif
+
+


This warning only gets printed for symbolic. What about imperative/Gluon?

eric-haibin-lin · 2018-08-15T20:13:02Z

src/operator/nn/mkldnn/mkldnn_base-inl.h

@@ -137,6 +137,10 @@ static inline bool SupportMKLDNN(const NDArray &input) {
      && SupportStorageMKLDNN(input.storage_type());
 }

+static inline bool MKLDNNEnvSet() {


Can we declare a static var so the linear time env lookup only happen once ?

azai91 · 2018-08-15T23:32:51Z

ubuntu@ip-172-31-11-93:~/incubator-mxnet-2/example/image-classification$ python benchmark_score.py
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Assertion failure at kmp_runtime.cpp(2582): __kmp_init_serial.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(2582).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
INFO:root:network: resnet-50
INFO:root:device: cpu(0)
/home/ubuntu/incubator-mxnet-2/python/mxnet/module/base_module.py:66: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
  warnings.warn(msg)
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = BatchNorm
input storage types = [default, default, default, default, default, ]
output storage types = [default, default, default, ]
params = {"fix_gamma" : True, "momentum" : 0.9, "eps" : 2e-05, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450: MXNET_MKLDNN_ENABLED flag is off. You can re-enable by setting MXNET_MKLDNN_ENABLED=1
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (7, 7), "stride" : (2, 2), "pad" : (3, 3), "num_filter" : 64, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = BatchNorm
input storage types = [default, default, default, default, default, ]
output storage types = [default, default, default, ]
params = {"fix_gamma" : False, "momentum" : 0.9, "eps" : 2e-05, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Activation
input storage types = [default, ]
output storage types = [default, ]
params = {"act_type" : relu, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Pooling
input storage types = [default, ]
output storage types = [default, default, ]
params = {"pad" : (1, 1), "stride" : (2, 2), "pool_type" : max, "kernel" : (3, 3), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 64, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (1, 1), "pad" : (1, 1), "num_filter" : 64, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 256, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"no_bias" : True, "kernel" : (1, 1), "workspace" : 256, "stride" : (1, 1), "num_filter" : 256, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = elemwise_add
input storage types = [default, default, ]
output storage types = [default, ]
params = {}
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 128, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (2, 2), "pad" : (1, 1), "num_filter" : 128, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 512, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"no_bias" : True, "kernel" : (1, 1), "workspace" : 256, "stride" : (2, 2), "num_filter" : 512, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (1, 1), "pad" : (1, 1), "num_filter" : 128, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (2, 2), "pad" : (1, 1), "num_filter" : 256, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 1024, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"no_bias" : True, "kernel" : (1, 1), "workspace" : 256, "stride" : (2, 2), "num_filter" : 1024, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (1, 1), "pad" : (1, 1), "num_filter" : 256, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (2, 2), "pad" : (1, 1), "num_filter" : 512, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (1, 1), "stride" : (1, 1), "pad" : (0, 0), "num_filter" : 2048, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"no_bias" : True, "kernel" : (1, 1), "workspace" : 256, "stride" : (2, 2), "num_filter" : 2048, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Convolution
input storage types = [default, default, ]
output storage types = [default, ]
params = {"workspace" : 256, "no_bias" : True, "kernel" : (3, 3), "stride" : (1, 1), "pad" : (1, 1), "num_filter" : 512, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = Pooling
input storage types = [default, ]
output storage types = [default, ]
params = {"global_pool" : True, "pool_type" : avg, "kernel" : (7, 7), }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
[23:16:54] ../src/executor/../operator/../common/utils.h:450:
Storage type fallback detected:
operator = FullyConnected
input storage types = [default, default, default, ]
output storage types = [default, ]
params = {"num_hidden" : 1000, }
context.dev_mask = cpu
The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning.
INFO:root:batch size  1, dtype float32, images/sec: 5.386060
INFO:root:batch size  2, dtype float32, images/sec: 5.743624
INFO:root:batch size  4, dtype float32, images/sec: 5.823662
INFO:root:batch size  8, dtype float32, images/sec: 5.858996
INFO:root:batch size 16, dtype float32, images/sec: 5.866382
INFO:root:batch size 32, dtype float32, images/sec: 5.865309

azai91 · 2018-08-15T23:33:35Z

moved the log into the fallback code so it should work for gluon / imperative as well. that being said, we should probably clean up the logs in a future cause it's becoming messy.

azai91 · 2018-08-15T23:33:40Z

@eric-haibin-lin <3

* fallback when env set * fix mkdnn default env * update docs to include desc about MXNET_MKLDNN_ENABLED * update env name to MXNET_MKLDNN_ENABLED * check dev_mask is cpu before fallback * log if flag is off * cache mkldnn check output * move logonce inside fallback * retrigger

fallback when env set

5080446

azai91 requested a review from anirudh2290 as a code owner August 7, 2018 03:17

fix mkdnn default env

03bd210

azai91 force-pushed the fix/mkldnn-env-flag branch from 0d1829f to 03bd210 Compare August 7, 2018 19:04

haojin2 reviewed Aug 7, 2018

View reviewed changes

haojin2 reviewed Aug 8, 2018

View reviewed changes

ZhennanQin reviewed Aug 8, 2018

View reviewed changes

lupesko suggested changes Aug 8, 2018

View reviewed changes

azai91 added 2 commits August 8, 2018 12:38

update docs to include desc about MXNET_MKLDNN_ENABLED

3b80836

update env name to MXNET_MKLDNN_ENABLED

3ab08fa

azai91 requested a review from szha as a code owner August 8, 2018 19:39

check dev_mask is cpu before fallback

06b2a5c

nswamy added MKLDNN pr-awaiting-review PR is waiting for code review labels Aug 9, 2018

log if flag is off

3ad3ec3

azai91 force-pushed the fix/mkldnn-env-flag branch from a3c94fd to 3ad3ec3 Compare August 15, 2018 05:30

eric-haibin-lin reviewed Aug 15, 2018

View reviewed changes

cache mkldnn check output

9e53fdc

azai91 force-pushed the fix/mkldnn-env-flag branch 2 times, most recently from 3bc949f to e4bda35 Compare August 15, 2018 22:58

move logonce inside fallback

65184d3

azai91 force-pushed the fix/mkldnn-env-flag branch from e4bda35 to 65184d3 Compare August 15, 2018 22:59

eric-haibin-lin approved these changes Aug 16, 2018

View reviewed changes

retrigger

ca3efb1

anirudh2290 merged commit 6843914 into apache:master Aug 17, 2018

TaoLv mentioned this pull request Apr 11, 2020

Confusing warning message when mkldnn is turned off #18019

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MKLDNN can be turned off with env var #12058

MKLDNN can be turned off with env var #12058

azai91 commented Aug 7, 2018 •

edited

Loading

azai91 commented Aug 7, 2018

haojin2 Aug 7, 2018

pengzhao-intel commented Aug 8, 2018

TaoLv commented Aug 8, 2018

haojin2 Aug 8, 2018

ZhennanQin left a comment

ZhennanQin Aug 8, 2018

lupesko left a comment

lupesko Aug 8, 2018

azai91 commented Aug 8, 2018

azai91 commented Aug 14, 2018

pengzhao-intel commented Aug 15, 2018

azai91 commented Aug 15, 2018

pengzhao-intel commented Aug 15, 2018

pengzhao-intel commented Aug 15, 2018

eric-haibin-lin Aug 15, 2018

eric-haibin-lin Aug 15, 2018

azai91 commented Aug 15, 2018

azai91 commented Aug 15, 2018

azai91 commented Aug 15, 2018

		"You can re-enable by setting MXNET_MKLDNN_ENABLED=1");
		#endif

MKLDNN can be turned off with env var #12058

MKLDNN can be turned off with env var #12058

Conversation

azai91 commented Aug 7, 2018 • edited Loading

Description

Checklist

Essentials

Changes

Comments

azai91 commented Aug 7, 2018

haojin2 Aug 7, 2018

Choose a reason for hiding this comment

pengzhao-intel commented Aug 8, 2018

TaoLv commented Aug 8, 2018

haojin2 Aug 8, 2018

Choose a reason for hiding this comment

ZhennanQin left a comment

Choose a reason for hiding this comment

ZhennanQin Aug 8, 2018

Choose a reason for hiding this comment

lupesko left a comment

Choose a reason for hiding this comment

lupesko Aug 8, 2018

Choose a reason for hiding this comment

azai91 commented Aug 8, 2018

azai91 commented Aug 14, 2018

pengzhao-intel commented Aug 15, 2018

azai91 commented Aug 15, 2018

pengzhao-intel commented Aug 15, 2018

pengzhao-intel commented Aug 15, 2018

eric-haibin-lin Aug 15, 2018

Choose a reason for hiding this comment

eric-haibin-lin Aug 15, 2018

Choose a reason for hiding this comment

azai91 commented Aug 15, 2018

azai91 commented Aug 15, 2018

azai91 commented Aug 15, 2018

azai91 commented Aug 7, 2018 •

edited

Loading