Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Built-in Support for Model Stacking in PyTorch Tabular #519

Closed
taimo3810 opened this issue Dec 7, 2024 · 4 comments
Closed

Add Built-in Support for Model Stacking in PyTorch Tabular #519

taimo3810 opened this issue Dec 7, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@taimo3810
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Model stacking is widely used in Kaggle competitions as an effective technique to boost model performance. However, PyTorch Tabular currently does not support stacking out of the box. As a Kaggler, I believe adding this feature would enhance the library's usability, making stacking more accessible and streamlined for users.

Describe the solution you'd like
I propose creating a Config class and a Model class specifically for the stacking model. Through the Config class, users will have the flexibility to customize the stacking process, including specifying which models to stack and their configurations. I am actively working on a pull request to implement this feature and plan to push the changes soon.

Describe alternatives you've considered
A custom implementation of stacking is possible; however, it requires time and effort to design, implement. This is why I believe adding this feature directly to the library is a better and more efficient solution.

Additional Context
To aid visual understanding, I have included a diagram illustrating the proposed stacking mechanism. This shows how different model backbones, such as CategoryEmbedding, FTTransformer, TabTransformer and more, can be combined through concatenation and passed to a linear layer for final predictions.

Screenshot 2024-12-08 at 1 37 04
@taimo3810 taimo3810 changed the title Add Model Stacking Support to PyTorch Tabular Add Built-in Support for Model Stacking in PyTorch Tabular Dec 7, 2024
@dosubot dosubot bot added the enhancement New feature or request label Dec 7, 2024
@manujosephv
Copy link
Owner

This looks like a great feature! Eagerly waiting for the PR. Let me know if you need any help.

@taimo3810
Copy link
Contributor Author

taimo3810 commented Dec 10, 2024

@manujosephv
I've just pushed pull request, but it's still in draft. There are some tasks remaining to be done. I’ll get it done soon!

@taimo3810
Copy link
Contributor Author

@manujosephv
Thank you for waiting. I have completed my pull request. Could you review it? If you find any issues, let me know!

@manujosephv
Copy link
Owner

Thanks for the great PR. I've merged it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants