Fused RNN Operators have nonsupport of add grad_req with mkl-dnn

[zixuanweeei]

Currently, we have not integrated the add grad_req routine in FusedRNNCell with mkl-dnn fusion. It will be highly appreciated if anyone could tell us the application scenario using add during the training process.

FYI, #568 and #725. When weights are updated every batch, add and write will produce the same results.

[ddavydenko]

@mxnet-label-bot add [MKLDNN]

[ddavydenko]

@TaoLv , I suggest you got this issue verified once you have [Upgrade MKL-DNN dependency to v1.0 #16555] merged.

[zixuanweeei]

@TaoLv , I suggest you got this issue verified once you have [Upgrade MKL-DNN dependency to v1.0 #16555] merged.

Thanks for your suggestion. MKL-DNN RNN operators do have this issue, and it will terminate the program when meets add. I have not met some cases that use add for RNN training. It will be highly appreciated if you can provide some information.
Currently, we need to deliver the gradients from mkl-dnn space to MXNet native space. It requires further design to guarantee performance and accuracy. We prefer to have #16555 merged, and then accomplish the add operation.

[pengzhao-intel]

@zixuanweeei does the issue resolve by MKLDNN upgrade PR?

[zixuanweeei]

@zixuanweeei does the issue resolve by MKLDNN upgrade PR?

This issue exists in the upgrade PR. It will give an error message when uses add.

[pengzhao-intel]

OK, is there a workaround for the user and what's the plan for the next step?

[zixuanweeei]

OK, is there a workaround for the user and what's the plan for the next step?

It will be appreciated that users could report their usage with add. But we couldn't force everyone to do this. So I will put it on the list. I plan to fix it in one week.

[TaoLv]

Should be fixed via #17075. Feel free to re-open if the problem is still there. @zixuanweeei