Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Flaky test test_gluon_trainer.test_trainer_reset_kv #11353

Closed
reminisce opened this issue Jun 21, 2018 · 3 comments
Closed

Flaky test test_gluon_trainer.test_trainer_reset_kv #11353

reminisce opened this issue Jun 21, 2018 · 3 comments
Labels
Bug Flaky Gluon

Comments

@reminisce
Copy link
Contributor

Have seen this in a few PRs before. Now it's always failing in my PR.
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11251/9/pipeline/749

test_gluon_trainer.test_trainer_reset_kv ... 

(py2) C:\jenkins_slave\workspace\ut-python-cpu>IF -1073741676 NEQ 0 exit /b 1 

script returned exit code 1
@zheng-da
Copy link
Contributor

The same problem shows up in my very old branch (about one month old). So if there is a bug in the code, the bug exists a long time ago. If it's not a bug, it might be something in the Windows testing system (it only fails in Windows as far as I know).
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10893/5/pipeline

@marcoabreu
Copy link
Contributor

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/master/1042/pipeline/752

@marcoabreu marcoabreu mentioned this issue Jun 22, 2018
Merged
7 tasks
@eric-haibin-lin
Copy link
Member

I'm able to reproduce it on windows. Still investigating

@eric-haibin-lin eric-haibin-lin mentioned this issue Jun 22, 2018
Merged
7 tasks
@marcoabreu marcoabreu mentioned this issue Jun 25, 2018
Closed
@szha szha closed this as completed in 619e4bd Jun 25, 2018
XinYao1994 pushed a commit to XinYao1994/incubator-mxnet that referenced this issue Aug 29, 2018
@eric-haibin-lin @szha
Fix apache#11353 (apache#11360)
1c7e9b6 
* Update test_gluon_trainer.py

* Update test_gluon_trainer.py

* Update test_gluon_trainer.py

* Update test_gluon_trainer.py

* Update test_gluon_trainer.py

* trigger

* Run 100000 times

* Update test_gluon_trainer.py

* run 10K times

* test_trainer_reset_kv didn't fail for 10K time . 2nd Trigger.

* test_trainer_reset_kv didn't fail for 10K times. 3rd Trigger.

* remove for loop
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Bug Flaky Gluon
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zheng-da @reminisce @eric-haibin-lin @marcoabreu