Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Mixin for block backward sync and deprecate block_backward_sync from Parallel Plugin #9234

Closed
four4fish opened this issue Aug 31, 2021 · 0 comments
Labels
feature Is an improvement or enhancement help wanted Open to be worked on refactor

Comments

@four4fish
Copy link
Contributor

Proposed refactoring or deprecation

Discussed in #9101

Motivation

Parallel plugin should be generic and self-contained. Move reference to DistributedDataParallel from parallel block_backward_sync

Pitch

Discussed in #9101
@ananthsub suggested create interface for this trait
`class PluginWithBlockBackwardSync(ABC):

@contextmanager
@abstractmethod
def block_backward_sync(self) -> Generator:`

Additional context


If you enjoy Lightning, check out our other projects! ⚡

  • Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

  • Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning

  • Bolts: Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

  • Lightning Transformers: Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

@four4fish four4fish added feature Is an improvement or enhancement help wanted Open to be worked on refactor labels Aug 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement help wanted Open to be worked on refactor
Projects
None yet
Development

No branches or pull requests

2 participants