[ambiguity] Does DLManagedTensor::deleter delete its function argument?

[junrushao]

It seems to me an undefined behavior in our protocol whether the deleter below deletes self as well. The inconsistency causes a recent crash when using MXNet as a backend for DGL, reported by @zheng-da.

dlpack/include/dlpack/dlpack.h

Lines 163 to 167 in 5c792ce

	/*! \brief Destructor signature void (*)(void*) - this should be called
	* to destruct manager_ctx which holds the DLManagedTensor. It can be NULL
	* if there is no way for the caller to provide a reasonable destructor.
	*/
	void (*deleter)(struct DLManagedTensor * self);

I did a quick check and found that the deleters in TVM and DGL do free self, but haven't verified it on PyTorch & Chainer side. Could someone help with this issue? Thanks!

[junrushao]

I sent a fix on the MXNet side that allows both
apache/mxnet#15016

[tqchen]

The deleter may not its function argument, but simply implies that the DLManagedTensor no longer want the object. The behavior could be decref if self is also reference counted and shared with others, or delete self if self is a new object and DLManagedTensor is the sole owner.

To elaborate further, the deleter should be sufficient to delete self and no further deletion is needed

[junrushao]

@tqchen I agree with your points and suggest we explicitly state that in our protocol.

The bug I introduced in MXNet is that I didn't know the deleter will free the opaque pointer, which causes a double free in DGL.

[junrushao]

For example, the deleter here doesn't delete the given handle because the handle is transient.

[tqchen]

a PR to clarify the protocol is more than welcomed

[junrushao]

Will do