[flaky] test_higher_order_grad.test_tanh

[reminisce]

======================================================================

FAIL: test_higher_order_grad.test_tanh

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/work/mxnet/tests/python/unittest/common.py", line 221, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/unittest/test_higher_order_grad.py", line 133, in test_tanh

    array, tanh, grad_grad_op, rtol=1e-6, atol=1e-6)

  File "/work/mxnet/tests/python/unittest/test_higher_order_grad.py", line 466, in check_second_order_unary

    check_nth_order_unary(x, op, grad_grad_op, 2, rtol, atol)

  File "/work/mxnet/tests/python/unittest/test_higher_order_grad.py", line 536, in check_nth_order_unary

    expected_grad, computed_grad.asnumpy(), rtol=rtol, atol=atol)

  File "/work/mxnet/python/mxnet/test_utils.py", line 627, in assert_almost_equal

    raise AssertionError(msg)

AssertionError: 

Items are not equal:

Error 1.130471 exceeds tolerance rtol=1.000000e-06, atol=1.000000e-06 (mismatch 0.185185%).

Location of maximum error: (0, 1, 4, 2), a=-0.09405555, b=-0.09405431

 ACTUAL: array([[[[ 0.33143836, -0.05487783, -0.06937446,  0.00169096,

           0.7525308 ,  0.5379254 ],

         [ 0.25605297, -0.03728245,  0.37168074,  0.7428499 ,...

 DESIRED: array([[[[ 0.33143836, -0.05487783, -0.06937446,  0.00169096,

           0.7525309 ,  0.5379254 ],

         [ 0.256053  , -0.03728245,  0.37168077,  0.7428501 ,...

-------------------- >> begin captured stdout << ---------------------



*** Maximum errors for vector of size 540:  rtol=1e-06, atol=1e-06



  1: Error 1.130471  Location of error: (0, 1, 4, 2), a=-0.09405555, b=-0.09405431

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17249/5/pipeline

[apeforest]

@kshitij12345 Would you mind taking a look at this test? Thanks! It looks like we can increase the precision tolerance to avoid this flakiness in test. Please let me know if you need help