fix AutoTP in deepspeed could not work for bloom#22196

sgugger merged 2 commits into

huggingface:mainhuggingface/transformers:mainfrom

sywangyi:auto_tp_bloomsywangyi/transformers:auto_tp_bloomCopy head branch name to clipboard