-
Notifications
You must be signed in to change notification settings - Fork 6.8k
IOError: [Errno 32] Broken pipe in Windows version #10562
Comments
@cjolivier01 : Please label : Windows, Operator |
same issue. It seems that MXNet doesn't support I set |
@Feywell Did this resolve the issue? Or should the docs be updated to talk about this limitation? |
Yes, MXNet doesn't support num_workers on Windows, because |
The fix of the multiple workers issue for Windows is here - #13686 |
Closing this since the fix is in. |
I am with mxnet 1.5.0 and have this problem.. |
Windows 10 Enterprise v 20H2, Python 3.6, Cuda 10, mxnet 1.7.0 - I have this problem too. It does work with >>> train_dataloader = mx.gluon.data.DataLoader(data_train, batch_size=batch_size, num_workers=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\site-packages\mxnet\gluon\data\dataloader.py", line 620, in __init__
initargs=[self._dataset, is_np_shape(), is_np_array()])
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\pool.py", line 174, in __init__
self._repopulate_pool()
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\pool.py", line 239, in _repopulate_pool
w.start()
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\zion\Anaconda3\envs\glu_env\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe |
@aaronmarkham @Ishitori Also having this problem on 1.5 and others clearly still are on later versions. Can anyone confirm that this this is actually fixed on >1.4. Should I create another issue or will someone reopen this one since the fix is clearly not in? |
I'd recommend opening a new issue if it can be triggered on the latest 1.9 version |
@szha So, I found a work around. It appears that there has been similar issues with multiprocessing on Windows in pytorch and pymc3. The data-loader must initialized inside of |
Description
I use mx.gluon.data.DataLoader will meet this trouble:
Code is here:
throw a trouble at
Environment info (Required)
----------Python Info----------
('Version :', '2.7.13')
('Compiler :', 'MSC v.1500 64 bit (AMD64)')
('Build :', ('default', 'May 11 2017 13:17:26'))
('Arch :', ('64bit', 'WindowsPE'))
------------Pip Info-----------
('Version :', '9.0.1')
('Directory :', 'E:\Anaconda2\lib\site-packages\pip')
Package used (Python/R/Scala/Julia):
Python 2.7
The text was updated successfully, but these errors were encountered: