Faster preprocessing#18

Merged

thomasw21 merged 15 commits intobigscience-workshop:mainfrom

thomasw21:faster_preprocessing

Aug 4, 2021

Commits on Jul 28, 2021

Propose a faster preprocessing mechanim by reducing the interprocesses communications
Thomas
committed

Commits on Jul 29, 2021

Add flush in order to force print
Thomas
committed

Commits on Jul 30, 2021

Try to prevent dead locks
Thomas
committed
Woops
Thomas
committed
Trying to figure out what causes deadlock
Thomas
committed

Commits on Jul 31, 2021

Commits on Aug 3, 2021

Document high number of worker preprocessing script
thomasw21
committed