Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[Flaky Test] Master Python2: MKLDNN-GPU test_operator_gpu.test_multinomial_generator #14158

Closed
Chancebair opened this issue Feb 14, 2019 · 4 comments · Fixed by #14475
Closed

Comments

@Chancebair
Copy link
Contributor

Chancebair commented Feb 14, 2019

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/master/326/pipeline

Disabled test: #14161


======================================================================

FAIL: test_operator_gpu.test_multinomial_generator

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest

    self.test(*self.arg)

  File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc

    return func(*arg, **kw)

  File "/work/mxnet/tests/python/gpu/../unittest/common.py", line 173, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/gpu/../unittest/test_random.py", line 651, in test_multinomial_generator

    nsamples=samples, nrepeat=trials)

  File "/work/mxnet/python/mxnet/test_utils.py", line 1981, in verify_generator

    str(buckets), str(probs)))

AssertionError: Generator test fails, Chi-square p=[0.045214571915128336, 0.9597478987060984, 0.017198384622679492, 0.04226990178352275, 0.01943621392672103], obs_freq=[array([ 99983, 200177, 299875,  50655, 150022, 199288]), array([ 99905, 199840, 300120,  49933, 150306, 199896]), array([ 99994, 199806, 300681,  50542, 149041, 199936]), array([100751, 199764, 298839,  50043, 150295, 200308]), array([100030, 200434, 299542,  50247, 150836, 198911])], expected_freq=[array([100000, 200000, 300000,  50000, 150000, 200000], dtype=int32), array([100000, 200000, 300000,  50000, 150000, 200000], dtype=int32), array([100000, 200000, 300000,  50000, 150000, 200000], dtype=int32), array([100000, 200000, 300000,  50000, 150000, 200000], dtype=int32), array([100000, 200000, 300000,  50000, 150000, 200000], dtype=int32)].

buckets=[0, 1, 2, 3, 4, 5], probs=[0.1  0.2  0.3  0.05 0.15 0.2 ]

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=2030134936 to reproduce.

--------------------- >> end captured logging << ---------------------

@mxnet-label-bot
Copy link
Contributor

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Test, Flaky

@Chancebair
Copy link
Contributor Author

@mxnet-label-bot add [Test, Flaky]

@apeforest
Copy link
Contributor

Based on the master branch CI log, the test started to become flaky after #14136 is merged.

@yuxihu
Copy link
Member

yuxihu commented Feb 15, 2019

The test I added (test_dataloader_context) in #14136 is not part of GPU CI. I do not see how it causes the flakiness for the test mentioned here.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants