-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove Strategy.on_gpu
#11537
Remove Strategy.on_gpu
#11537
Conversation
@@ -296,7 +296,7 @@ def _register_ddp_hooks(self) -> None: | |||
# In 1.8, DDP communication hooks only work with NCCL backend and SPSD (single process single device) mode | |||
# Since 1.9, DDP communication hooks can work on all backends. | |||
if _TORCH_GREATER_EQUAL_1_9 or ( | |||
_TORCH_GREATER_EQUAL_1_8 and self.on_gpu and self._is_single_process_single_device | |||
_TORCH_GREATER_EQUAL_1_8 and self.root_device.type == "cuda" and self._is_single_process_single_device |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did you choose to check it this way? in some places it may be necessary to differentiate between the model already being on gpu or not yet, but here and in most cases it is only about which accelerator is selected. So an isinstance check would have been more appropriate I believe.
What does this PR do?
Part of #10416
Reasons for removal:
root_device
property. The propertyon_gpu
is redundant yet it's also anabstractmethod
. Requiring every strategy class to implement this is boilerplate.on_ipu
oron_X
for existing accelerator implementationson_gpu
andon_tpu
flags doesn't scale.Does your PR introduce any breaking changes? If yes, please list them.
Yes: removes the
on_gpu
property from the Strategy base class.Reason: The strategy is still an experimental API, and this is part of making it stable
See #11536 for the equivalent PR removing
on_tpu
Before submitting
PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃