Group feature selection

[kerrymcgowan]

Hello Sebastian,

Is it possible to perform feature selection with groups of features instead of individual features with mlxtend? So for example, if you have 100 features that could be partitioned into groups 1-10, is there a way to run feature selection to select which combination of those 10 groups produces the best model instead of selecting from the 100 features individually?

[rasbt]

Good question! The answer is yes and no 😅. Right now, it doesn't work out of the box, but I remember doing something similar once where I had 3 features, I did a one-hot encoding on the third feature, which became 20 new features due to the one hot encoding, leaving me with 22 features in total. I wrote some custom code / modification so that the the 20 new features were treated as a group rather than 20 individual ones.

I should have that code somewhere and will try to see if I can find it in the next couple of days. I should still have it somewhere since that was for a research project 1-2 years ago that I should have archived.

[NimaSarajpoor]

@rasbt
Is that okay if I start working on this? do you have time to help throughout the process? If you think it is too much and probably not useful for most users, please feel free to let me know :)

[rasbt]

Absolutely! That would be amazing. I am happy to try to help along the way when I can

[rasbt]

Btw I think it can probably be implemented similar to the feature_permutation_importance (https://github.com/rasbt/mlxtend/blob/master/mlxtend/evaluate/feature_importance.py#L116, this can be refactored, but currently no time, just wanted to get a correctly working base version done, and refactoring is a future problem 😅)

[NimaSarajpoor]

Btw I think it can probably be implemented similar to the feature_permutation_importance

Cool. Thanks for letting me know. I will take a look!

[kerrymcgowan]

Thank you for checking @rasbt. For anyone looking into doing something similar, I was able to find a group forward feature selection tutorial using mlxtend and eobox here.