Various issues on Python 3.8

[leezu]

CI doesn't test Python 3.8 and in fact some things are broken. What I noticed:

• Use of time.clock() which is deprecated since Python 3.3 and not available in 3.8 in some files breaks those functionalities on 3.8

To prevent such issues, suggest to test Python 3.8 on the unix pipelines (and test the minimum supported Python version on centos).

[ma-hei]

@leezu can this be reproduced locally by running python tests via nosetest?

[leezu]

Yes. You'll see an exception due to accessing an undefined function. Though note that you should use pytest instead of nosetest. For the tests, only tests/python/unittest/test_profiler.py is affected by the time.clock issue. But there may be more issues.

To enforce that there are no remaining issues, eventually we should switch the Ubuntu CI to Python 3.8 and ensure that all tests pass (at which stage the PR can be merged). This can be achieved by switching https://github.com/apache/incubator-mxnet/blob/master/ci/docker/Dockerfile.build.ubuntu to Ubuntu 20.04, as 20.04 ships with Python 3.8 by default.

The base image for above dockerifle is specified in https://github.com/apache/incubator-mxnet/blob/master/ci/docker/docker-compose.yml Currently there are no nvidia cuda images based on 20.04 yet. So when updating the CPU image to 20.04, the GPU images would need to remain on 18.04.

Switching the CI can be done separately from fixing the bugs.

[ma-hei]

I see, so as part of this fix we would update the base image of ubuntu_cpu to 20.04. I will create a first PR in which the base image of ubuntu_cpu is updated to 20.04, to see which tests are failing.

[ekdnam]

@szha I would love to work on this issue. Can you assign me to this?

[szha]

@ekdnam thanks for offering to help! let me know if you have any question or need any pointer.

[ekdnam]

Yes surely!

[ekdnam]

@szha is this a unix specific issue?

[szha]

@ekdnam apologies, I didn't realize that the issue has already been fixed in https://github.com/apache/incubator-mxnet/pull/18619/files#diff-9b62bcdde840f4769a6f1e1cf1e664a7L55 by @leezu. I think we still need to add the python3.8 pipeline, for which @ma-hei opened #18445.

[szha]

@ma-hei would you be able to complete #18445?

[ekdnam]

Oh okay, its no big deal. Till then, I shall try to solve some other issue

[ma-hei]

Hi, sorry I haven't looked at #18445 in a while. Will take a look again this week.

[leezu]

Python 3.8 is now tested on CI as part of #19588