Need a debugging utility to dump out intermediate values of Tensor object

[apeforest]

Currently, when debugging a MXNet operator, there is no existing utility function to dump out intermediate Tensor and NDArray object.

Developers have to write their own debugging code which is inefficient and error prone.

We need an easy to use C++ debugging utility to help developers debug MXNet operators in backend.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[anirudh2290]

Does set_monitor_callback in executor work for your use case ?

[ptrendx]

Monitor callback is good (although not really documented properly - I had to look at the internals of monitor implementation in Module API to know what the Python function should do to access the NDArray), but it only works for Symbol API, not for Gluon, which makes it really hard to debug backward pass/both passes of hybridized models.

[larroy]

FYI I'm working on this. Ok, to be precise I want to dump the graph in the backward pass, getting value of the tensors would be secondary. How would monitor callback work if there's no executor? @anirudh2290 had the same question in the PR linked here.