Creating a multiprocessing pool while PyTorch Lightning is imported leads to an exception

[senneberden]

Bug description

When both Python's multiprocessing package and PyTorch Lightning are imported, and a multiprocessing pool is made, the following exception is thrown:

Exception ignored in: <function Pool.__del__ at 0x7fd193562280>
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 268, in __del__
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

How to reproduce the bug

import pytorch_lightning as pl
import multiprocessing as mp

pool = mp.Pool(1)

Error messages and logs

Exception ignored in: <function Pool.__del__ at 0x7fd193562280>
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 268, in __del__
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

Environment

* CUDA:
	- GPU:               None
	- available:         False
	- version:           10.2
* Lightning:
	- pytorch-lightning: 1.7.7
	- torch:             1.12.1
	- torchmetrics:      0.10.0
* Packages:
	- absl-py:           1.3.0
	- aiohttp:           3.8.3
	- aiosignal:         1.2.0
	- async-timeout:     4.0.2
	- attrs:             22.1.0
	- cachetools:        5.2.0
	- certifi:           2022.9.24
	- charset-normalizer: 2.1.1
	- frozenlist:        1.3.1
	- fsspec:            2022.10.0
	- google-auth:       2.13.0
	- google-auth-oauthlib: 0.4.6
	- grpcio:            1.50.0
	- idna:              3.4
	- importlib-metadata: 5.0.0
	- markdown:          3.4.1
	- markupsafe:        2.1.1
	- multidict:         6.0.2
	- numpy:             1.23.4
	- oauthlib:          3.2.2
	- packaging:         21.3
	- pip:               20.0.2
	- pkg-resources:     0.0.0
	- protobuf:          3.19.6
	- pyasn1:            0.4.8
	- pyasn1-modules:    0.2.8
	- pydeprecate:       0.3.2
	- pyparsing:         3.0.9
	- pytorch-lightning: 1.7.7
	- pyyaml:            6.0
	- requests:          2.28.1
	- requests-oauthlib: 1.3.1
	- rsa:               4.9
	- setuptools:        44.0.0
	- six:               1.16.0
	- tensorboard:       2.10.1
	- tensorboard-data-server: 0.6.1
	- tensorboard-plugin-wit: 1.8.1
	- torch:             1.12.1
	- torchmetrics:      0.10.0
	- tqdm:              4.64.1
	- typing-extensions: 4.4.0
	- urllib3:           1.26.12
	- werkzeug:          2.2.2
	- wheel:             0.37.1
	- yarl:              1.8.1
	- zipp:              3.9.0
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- ELF
	- processor:         x86_64
	- python:            3.8.10
	- version:           #58~20.04.1-Ubuntu SMP Thu Oct 13 13:09:46 UTC 2022

More info

No response

[awaelchli]

Hi @ijskar

Thanks for reporting. This is a very confusing issue! After a quick look, could not figure out what is going on, but can confirm it reproduces.
I was able to reduce this to lightning_lite (a subpackage in Lightning).

import lightning_lite
import multiprocessing as mp
pool = mp.Pool(1)

So far, I have tried to disable all multiprocessing related imports but no success.