Mxnet allclose

[drivanov]

Description

• The analog of numpy.allclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False) is implemented as a MxNet operator:

    mx.nd.contrib.allclose(a, b, rtol, atol, equal_nan)

• For now, besides the unit test test_allclose_function(), this method is used only in

def assert_almost_equal(a, b, rtol=None, atol=None, names=('a', 'b'), equal_nan=False)

where parameter a or/and b could be defined also as mx.nd.array(s):

    Parameters
    ----------
    a : np.ndarray or mx.nd.array
    b : np.ndarray or mx.nd.array

• When calling assert_almost_equal, no more asnumpy() conversion needed. It will be done automatically (in assert_almost_equal), if
  (a) a or b has no attribute "context" OR
  (b) these attributes are different.

• The elimination of asnumpy() conversions and the use of mx.nd.contrib.allclose(...) for mx.nd.array's allows to achieve 5-7x speedup for GPU tests that use long arrays.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage:
• Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
• Code is well-documented:
• For user-facing API changes, API doc string has been updated.
• For new C++ functions in header files, their functionalities and arguments are documented.
• To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Comments

Most of the changed files, were changed because of the eliminations of asnumpy() conversions which were used when assert_almost_equal function is called.

[karan6181]

@mxnet-label-bot add [Operator, Test, pr-awaiting-review]

[chinakook]

It this op only support float32?

[drivanov]

It this op only support float32?

Actually, only the tolerances (rtol, atol) are supposed to be float32. The elements of tensors could be any type.

[drivanov]

I am not sure that this fail is related to my changes made to this PR. I will make an empty commit for triggering CI.

[drivanov]

Failing tests are not related to the changes of this PR. I will launch CI one more time.

[wkcn]

Sometimes the CI may fails and raise an irrelated exception. We can disable the using of mx.nd.contrib.allclose firstly to check it.

[drivanov]

As I could see that _test_bulking() is not there and I need to fix it.

[wkcn]

Glad to see the success of CI : )

@chinakook @szha @mseth10
Could you help take a review? Thank you!

[abhinavs95]

@drivanov Could you do a rebase to resolve the merge conflicts?

[ChaiBapchya]

@szha @mseth10 Good to merge? Pl review

[szha]

there're merge conflicts that need to be resolved first.

[drivanov]

Just in case, I rebased. CI is ok and should be no merge conflicts.

[wkcn]

Hi @mseth10 and @szha , the conflict has been resolved.

This PR accelerates MXNet allclose, and it accepts mx.nd.NDArray and np.ndarray without any conversion.

Could you please help take a review? Thank you so much!

[ptrendx]

LGTM