don't gather partitioned activations for mp size 1 (#2454)#1

delock merged 1 commit intodelock:gma/snapshot20221103delock/DeepSpeedSYCLSupport:gma/snapshot20221103from

guoyejun:forgmaguoyejun/DeepSpeed:forgmaCopy head branch name to clipboard