Remove Strategy.on_tpu property

[ananthsub]

What does this PR do?

Part of #10416

Reasons for removal:

• This can be deduced from the root_device property. The property on_tpu is redundant yet it's also an abstractmethod. Requiring every strategy class to implement this is boilerplate.
• We don't offer on_ipu or on_X for existing accelerator implementations
• Given the Accelerator is also customizable/pluggable, the pattern of offering on_gpu and on_tpu flags doesn't scale.

Does your PR introduce any breaking changes? If yes, please list them.

Yes: removes the on_tpu property from the Strategy base class.
Reason: The strategy is still an experimental API, and this is part of making it stable

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

[four4fish]

+1 to this change. I realized the on_tpu is not actually used anywhere, only in tests. Other than on_tpu(), the _XLA_AVAILABLE check is only used at import time. Should we have device_availablity check at X_accelerator initialization time?

[codecov]

Codecov Report

Merging #11536 (4ea1dff) into master (f41d1e5) will decrease coverage by 4%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           master   #11536    +/-   ##
========================================
- Coverage      92%      88%    -4%     
========================================
  Files         194      194            
  Lines       16972    16961    -11     
========================================
- Hits        15611    14964   -647     
- Misses       1361     1997   +636     

[ananthsub]

+1 to this change. I realized the on_tpu is not actually used anywhere, only in tests. Other than on_tpu(), the _XLA_AVAILABLE check is only used at import time. Should we have device_availablity check at X_accelerator initialization time?

@kaushikb11 what do you think^?

In general I am in favor of those checks running as early as possible to avoid wasting users' time. In the GPU accelerator, we do check this here: https://github.com/PyTorchLightning/pytorch-lightning/blob/f41d1e5e5ebb7040a39d137695e818cada9a9234/pytorch_lightning/accelerators/gpu.py#L36-L44

But we don't assert that cuda is available - should we?
And should we do that at the init time vs in setup_environment ?

@four4fish Would this also simplify the accelerator connector initialization logic?

Regardless, I'd like to handle it in a separate issue & PR 😄

[four4fish]

@ananthsub @kaushikb11 I think we can introduce a new function called _device_check() in X_Accelerator, which will be called by accelerator_connector right after accelerator initialization.
We don't have consistent device availibliies check in neither accelerator class, strategies class nor accelerator_connector at this point, so I introduced it in the new accelerator_connector rewrite . But it's not ideal, I think it's a better fit in accelerator API. I will create a issue, let's discuss in the issue

[kaushikb11]

@four4fish @ananthsub I agree. We could add device_check functionality in the Accelerator Class. I had planned to add it after starting to work on #10687

[tchaton]

LGTM !