Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Fix flaky test test_random:test_randint_generator #13498

Merged
merged 7 commits into from
Dec 7, 2018
Merged

Fix flaky test test_random:test_randint_generator #13498

merged 7 commits into from
Dec 7, 2018

Conversation

ChaiBapchya
Copy link
Contributor

@ChaiBapchya ChaiBapchya commented Dec 1, 2018 •
edited
Loading

Description

Fixes #13446 flaky test test_random:test_randint_generator
Uses the PR #13543 to get rid of the fix seed and make the test robust

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

@vandanavk
Copy link
Contributor

@mxnet-label-bot add [Flaky, pr-awaiting-review]

@marcoabreu marcoabreu added Flaky pr-awaiting-review PR is waiting for code review labels Dec 1, 2018
@anirudh2290
Copy link
Member

Can you run this test repeatedly 1k times with different seeds to check flakiness.

@ChaiBapchya
Copy link
Contributor Author

@anirudh2290 Have tried it 1000 times

MXNET_TEST_COUNT=1000 python -m nose -v --nocapture tests/python/unittest/test_random.py:test_randint_generator

Python3 CPU Linux

@anirudh2290 anirudh2290 merged commit 0011ab2 into apache:master Dec 7, 2018
zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@ChaiBapchya @zhaoyao73
Fix flaky test test_random:test_randint_generator (apache#13498)
0598064 
* updated seed, alpha value, comments

* typo in comment fix

* added nrepeat

* removed unusued variable, added link for scipy alpha, rephrased the sentence for discrete distribution buckets

* removed fixed seed, alpha
zhaoyao73 added a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@zhaoyao73
Merge remote-tracking branch 'upstream/master'
79de25d 
* upstream/master: (54 commits)
  Add notes about debug with libstdc++ symbols (apache#13533)
  add cpp example inception to nightly test (apache#13534)
  Fix exception handling api doc (apache#13519)
  fix link for gluon model zoo (apache#13583)
  ONNX import/export: Size (apache#13112)
  Update MXNetTutorialTemplate.ipynb (apache#13568)
  fix the situation where idx didn't align with rec (apache#13550)
  Fix use-before-assignment in convert_dot (apache#13511)
  License update  (apache#13565)
  Update version to v1.5.0 including clojure package (apache#13566)
  Fix flaky test test_random:test_randint_generator (apache#13498)
  Add workspace cleaning after job finished (apache#13490)
  Adding test for softmaxoutput (apache#13116)
  apache#13441 [Clojure] Add Spec Validations for the Random namespace (apache#13523)
  Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file" (apache#13558)
  Chi_square_check for discrete distribution fix (apache#13543)
  Updated docs for randint operator (apache#13541)
  Simplifications and some fun stuff for the MNIST Gluon tutorial (apache#13094)
  Fix apache#13521 (apache#13537)
  Add a retry to qemu_provision (apache#13551)
  ...
@stu1130 stu1130 mentioned this pull request Mar 21, 2019
Merged
5 tasks
@stu1130 stu1130 mentioned this pull request Mar 30, 2019
Merged
5 tasks
@ChaiBapchya ChaiBapchya deleted the fix_flaky_randint_generator branch July 29, 2019 22:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mseth10 mseth10 mseth10 left review comments

@anirudhacharya anirudhacharya anirudhacharya left review comments

@yzhliu yzhliu yzhliu left review comments

@anirudh2290 anirudh2290 anirudh2290 approved these changes

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
Flaky pr-awaiting-review PR is waiting for code review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

flaky test: test_random.test_randint_generator
7 participants
@ChaiBapchya @vandanavk @anirudh2290 @yzhliu @anirudhacharya @mseth10 @marcoabreu