Future state of lightning-bolts!

[SeanNaren]

🚀 Feature

After discussion with the Lightning team and the piling up issues I think it's important to address the library and some of its pain points. Overall the goal is to improve the libraries functionality whilst honing exactly what this library will deliver!

Motivation

The Lightning team doesn't have time to assist with issues/PRs around specific research pieces. This primarily is the SSL implementations/pieces. This leads to a lot of smelly code, and horrible import errors as we've seen so far.

Pitch

Focus on general applied research components; callbacks, datamodules, datasets

Move away from specific model implementations and writing applied research. As we've seen these deprecate very quickly, and probably require a standalone repository to ensure support.

Deprecate SSL implementations, encourage Lightning Flash VISSL

Since we'll not be able to support the SSL implementations in bolts, I suggest we make it clear that any issues around these may be stale. We'll continue to encourage contributions to fix any issues with them for users who still use them, but strongly recommend looking at Lightning Flash VISSL integration (once integrated Lightning-Universe/lightning-flash#682).

Re-write README/documentation to be component focused, deprecate/drop model specific implementations

Should be fairly explanatory. Try to hone in and make clear what the library is about.

Additional context

We'll be introducing more third-party callbacks, and introducing more components to help research :) Before doing this, we want to clear up the repository to support this!

cc @Borda @akihironitta @tchaton

[Borda]

Yep, I agree with almost all these points + cut out datasets/data modules they are very hard to reliable testing without downloading the data and downloading is space/connection and time demanding...

The Lightning team doesn't have time to assist with issues/PRs around specific research pieces.

the original idea was to find and establish ambassadors/code contributors for each domain...

[tchaton]

Thanks @SeanNaren for documenting the following process. Definitely the right way forward with Bolts !

@Borda Do you have candidate for ambassadors/code contributors ?

[zippeurfou]

It makes sense to me. I would suggest making sure that we don't "remove" capability but take the opportunity to provide them to our other libraries if they don't already have them and provide a way for our community to transition to the other framework to do the same thing they were doing for the deprecated part providing a smooth transition.

Focus on general applied research components; callbacks, datamodules, datasets
Move away from specific model implementations and writing applied research. As we've seen these deprecate very quickly, and probably require a standalone repository to ensure support.

I would also suggest having a bit more specific in terms of what would be the first new set of features you want to implement.
If we are taking a new direction, it is important to have a notion of the scope so you can think of the design of it first that would encompass most of your ideas.

[akihironitta]

@SeanNaren Thank you for bringing up this issue! I agree with you on all the points listed there.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.