[Feature Request] Release gpu memory by API.

[ducheng678]

Description

When it is required to do a forward propagation at first, and then do another one in the same process,
if gpu memory malloced by the first propagation can not be released, the second one will not have enough memory to use.

[vrakesh]

@DuCheng2018 Thank you for the feature, request.

[vrakesh]

@mxnet-label-bot add [Feature Request]

[zhreshold]

I am getting multiple(> 5 times) requests similar to this one, I think we need a formal C API as well as python API so user can explicitly release gpu memory pool without waiting for program to exit.

[haoxintong]

+1, for some application it is necessary to release gpu memory.

[relaxli00]

+1, yes, this is a function needed badly

[ghost]

+1

[hdjsjyl]

+1

[kohillyang]

+1

[HermanLiu1]

+1

[fallingdust]

+1

[fe-codes]

+1

[cgraywang]

+1

[Shicc]

+1

[yuyijie1995]

+1

[qdLMF]

+1

[evi-Genius]

+1

[Wondersui]

+1

[anonymouslycn]

+1

[khui]

+1

[hgt312]

+1

[smartwell]

+1

[Kongsea]

+1

[ducnguyen96]

+1

[wkcn]

Hi, there.
Thank @vladoovtcharov for adding this API for GPU memory release.
The API is mx.gpu(0).empty_cache().
Firstly, we should set the environment variable MXNET_GPU_MEM_POOL_TYPE to use GPU memory pool.

Here is an example:

# MXNET_GPU_MEM_POOL_TYPE=Round python
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mxnet as mx
>>> a = mx.nd.zeros((10000, 30000), ctx=mx.gpu(0)) + 1
[14:56:53] src/storage/storage.cc:110: Using GPUPooledRoundedStorageManager.
>>> del a
>>> mx.gpu(0).empty_cache()

In my test, the most GPU memory will be released after calling this API, although there is still around 600Mib memory cache.

[szha]

This API only releases the cache in the GPU memory pool. It doesn't release the NDArray cache held by CachedOp

[ymzx]

+1

[zhreshold]

Closing this issue as #14252 is merged, please feel to ping me if you have other thoughts.