Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Fix timeout 1200sec issue for large tensor tests #18201

Merged
merged 1 commit into from
Apr 30, 2020

Conversation

ChaiBapchya
Copy link
Contributor

Description

Large tensor tests on nightly fail due to timeout issue. Removing the timeout limit for these tests

@szha @leezu @access2rohit
@ciyongch I will cherrypick both the large tensor fixes [once this gets merged]

@mxnet-bot
Copy link

Hey @ChaiBapchya , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [unix-gpu, windows-gpu, windows-cpu, sanity, centos-gpu, unix-cpu, clang, edge, website, centos-cpu, miscellaneous]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@leezu leezu merged commit b8f78c9 into apache:master Apr 30, 2020
@ChaiBapchya ChaiBapchya deleted the timeout_large_tensor_nightly branch April 30, 2020 18:57
@ChaiBapchya
Copy link
Contributor Author

While this fix makes sure timeout doesn't happen, it results in few large tensor tests getting skip, few tests pass

test session starts ==============================

[2020-04-30T21:01:52.958Z] platform linux -- Python 3.5.2, pytest-5.3.5, py-1.8.1, pluggy-0.13.1

[2020-04-30T21:01:52.958Z] rootdir: /work/mxnet, inifile: pytest.ini

[2020-04-30T21:01:52.958Z] plugins: timeout-1.3.4, xdist-1.31.0, forked-1.1.3, env-0.6.2, cov-2.8.1

[2020-04-30T21:01:52.958Z] timeout: 1200.0s

[2020-04-30T21:01:52.958Z] timeout method: signal

[2020-04-30T21:01:52.958Z] timeout func_only: False

[2020-04-30T21:01:52.958Z] collected 1 item

[2020-04-30T21:01:52.958Z] 

[2020-04-30T21:05:14.349Z] tests/nightly/test_large_vector.py s                                     [100%]

[2020-04-30T21:05:14.349Z] 

[2020-04-30T21:05:14.349Z] ======================== 1 skipped in 188.48s (0:03:08) ========================

[2020-04-30T21:05:14.349Z] + pytest tests/nightly/test_large_vector.py::test_nn

[2020-04-30T21:05:14.349Z] ============================= test session starts ==============================

[2020-04-30T21:05:14.349Z] platform linux -- Python 3.5.2, pytest-5.3.5, py-1.8.1, pluggy-0.13.1

[2020-04-30T21:05:14.349Z] rootdir: /work/mxnet, inifile: pytest.ini

[2020-04-30T21:05:14.349Z] plugins: timeout-1.3.4, xdist-1.31.0, forked-1.1.3, env-0.6.2, cov-2.8.1

[2020-04-30T21:05:14.349Z] timeout: 1200.0s

[2020-04-30T21:05:14.349Z] timeout method: signal

[2020-04-30T21:05:14.349Z] timeout func_only: False

[2020-04-30T21:05:14.349Z] collected 1 item

[2020-04-30T21:05:14.349Z] 

[2020-04-30T21:53:21.695Z] tests/nightly/test_large_vector.py .                                     [100%]

[2020-04-30T21:53:21.695Z] 

[2020-04-30T21:53:21.695Z] ======================== 1 passed in 2898.25s (0:48:18) ========================

Here : http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/detail/master/675/pipeline/100

@ChaiBapchya
Copy link
Contributor Author

It works as expected. The skip command in the test_large_vector.py is the cause.

AntiZpvoh pushed a commit to AntiZpvoh/incubator-mxnet that referenced this pull request Jul 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants