Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

mxnet blocked when binding gpu device #8656

Closed
wzl12356 opened this issue Nov 15, 2017 · 0 comments
Closed

mxnet blocked when binding gpu device #8656

wzl12356 opened this issue Nov 15, 2017 · 0 comments

Comments

@wzl12356
Copy link
Contributor

mxnet:0.12.0
cuda:7.5
mxnet blocked python codes:
train_exec = _bind_exec(sym, ctxi, data_shapes, self.param_names,
need_grad=True, base_exec=shared_exec,
shared_data_arrays=self.shared_data_arrays[i],
input_types=data_types)
Stack info:
#1 0x00007f363621dea7 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#2 0x00007f363623e9ea in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#3 0x00007f3636246fef in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#4 0x00007f36367ef3dd in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#5 0x00007f36362319df in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#6 0x00007f36362360a5 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#7 0x00007f36367ef3dd in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#8 0x00007f3636247bc0 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#9 0x00007f36361f4415 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#10 0x00007f36367ef3dd in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#11 0x00007f36361f7b94 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#12 0x00007f36361f92e9 in ?? () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#13 0x00007f36361efabc in __cuda_CallJitEntryPoint () from /usr/lib64/libnvidia-ptxjitcompiler.so.375.20
#14 0x00007f392fb63582 in fatBinaryCtl_Compile () from /usr/lib64/libnvidia-fatbinaryloader.so.375.20
#15 0x00007f3937f52e42 in ?? () from /usr/lib64/libcuda.so.1
#16 0x00007f3937f539c3 in ?? () from /usr/lib64/libcuda.so.1
#17 0x00007f3937eac35e in ?? () from /usr/lib64/libcuda.so.1
#18 0x00007f3937eac640 in ?? () from /usr/lib64/libcuda.so.1
#19 0x00007f39448cd52d in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#20 0x00007f39448c1ba0 in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#21 0x00007f39448cc796 in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#22 0x00007f39448d0ed1 in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#23 0x00007f39448c445e in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#24 0x00007f39448b22ee in ?? () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#25 0x00007f39448e6194 in cudaStreamCreate () from /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.7.5
#26 0x00007f3947383e7c in mshadow::Streammshadow::gpu* mshadow::NewStreammshadow::gpu(bool, bool, int) () from /search/odin/mxnet_wzl/train/python/mxnet/../../lib/libmxnet.so
#27 0x00007f394739fe4f in void mxnet::engine::ThreadedEnginePerDevice::GPUWorker<(dmlc::ConcurrentQueueType)0>(mxnet::Context, bool, mxnet::engine::ThreadedEnginePerDevice::ThreadWorkerBlock<(dmlc::ConcurrentQueueType)0>*, std::shared_ptrmxnet::engine::ThreadPool::SimpleEvent) () from /search/odin/mxnet_wzl/train/python/mxnet/../../lib/libmxnet.so

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant