DataLoader cannot use multithreading on windows？

[tongyuhome]

Hi，Thank you very much for sharing。
I have a problem. When I run the code on the Windows10 system, I always get an error when setting ‘num_workers’ to a number >0.
The error location is on line 115:

for i, data in enumerate(loader):

The error message is:

File "E:\Software\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 819, in iter
return _DataLoaderIter(self)
File "E:\Software\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 560, in init
w.start()
File "E:\Software\anaconda3\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "E:\Software\anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\Software\anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "E:\Software\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "E:\Software\anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'TrainAugmentation.init..'

Can you give me some solutions or suggestions?
Thank you！

[qa-majeed]

You can passed '--num_workers 0' as a command line argument.
It will work using 01 thread or not exploiting multi threading.

[tongyuhome]

You can passed '--num_workers 0' as a command line argument.
It will work using 01 thread or not exploiting multi threading.

Thanks for replying, using single thread is very slow, I just want to use multithreading, how do I need to modify it?

[andreistirb]

in order to use multithreading in windows you should try changing the lambda transformation into a function (callable class, to be more specific).
Something like this:
class NormalizesStd(object): def __init__(self, std): self.std = np.array(std, dtype=np.float32) def __call__(self, image, boxes=None, labels=None): image = image.astype(np.float32) image /= self.std return image.astype(np.float32), boxes, labels

[MakingisAwesome]

Hi, i'm having the same problem and would like to use multithreading. Can someone here show an example of how to implement these lines:

class NormalizesStd(object): def init(self, std): self.std = np.array(std, dtype=np.float32) def call(self, image, boxes=None, labels=None): image = image.astype(np.float32) image /= self.std return image.astype(np.float32), boxes, labels

Thank you very much!