Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

pytest worker crash seen on newly introduced unittest test_profiler_gpu.py::test_aggregate_duplication #18756

Open
DickJC123 opened this issue Jul 19, 2020 · 1 comment
Labels
Bug Flaky Profiler MXNet profiling issues

Comments

@DickJC123
Copy link
Contributor

Description

tests/python/gpu/test_profiler_gpu.py has recently started importing unittests/test_profiler.py, and so now runs those tests for the first time with a gpu default context. With that change, I have seen on centos-gpu:

worker 'gw3' crashed while running 'tests/python/gpu/test_profiler_gpu.py::test_aggregate_duplication'

Occurrences

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-gpu/detail/PR-18694/11/pipeline

@leezu You might want to look into this or watch for more occurences.

What have you tried to solve it?

  1. I've bypassed this test in my current PR with a del test_aggregate_duplication after the newly introduced import.
@leezu
Copy link
Contributor

leezu commented Jul 20, 2020

@DickJC123 I'm not familiar with the feature, but @Zha0q1 added the feature in #15240 and may help. Thanks for disabling the flaky test

@leezu leezu added Bug Profiler MXNet profiling issues labels Jul 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug Flaky Profiler MXNet profiling issues
Projects
None yet
Development

No branches or pull requests

2 participants