Fix hang in DDP HPC accelerators

[ananthsub]

What does this PR do?

init_device was not called, so the root_gpu is 0 throughout. This leads to a hang later on in ddp_train when configure_ddp was called, as the LightningDistributedDataParallel call never completes with the redundant device ids set.

This is a recent change from #5016 and there aren't tests in OSS for this particular accelerator

2 main questions:

• Here I'm calling init_device immediately after set_world_ranks - is that correct?
• What should be done in the DDP HPC CPU accelerator which extends this? Setting the root gpu there doesn't make sense so for now I've set init_device as a no-op, but I'm not sure if that's the most robust.

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #5157 (67c2c60) into master (b4d926b) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #5157   +/-   ##
======================================
  Coverage      93%     93%           
======================================
  Files         134     134           
  Lines        9905    9905           
======================================
  Hits         9204    9204           
  Misses        701     701           

[Borda]

lgtm, mind add chlog?

[SeanNaren]

I think the changes are fine apologies for the mistake. This is correct, devices just now have to be initialised separately from the model due to changes for the RPC plugin, and before initializing any model/connections.