A few extra questions on FLAML

[flippercy]

Hi Team FLAML:

I have a few general questions regarding FLAML:

1. Could I use a customized learner for the ensemble? For example, for a linear combination, I want to make sure all the coefficients are negative.

2. I noticed that the search of automl is much more biased when running distributedly on compute clusters, which means that it focuses much more on 'strong' learners. For example, this is the summary from my recent search:
   
   Obviously FLAML spent the majority of time on LearnerE, which is the optimal one; it is understandable, however, the distribution of attempts on different learners is much more balanced when running FLAML on a VM. Could you explain this phenomenon, please? Usually we use FLAML not just to look for the overall optimal solution, but also the best result per learner.

Thank you!

[sonichi]

1. Yes. Please take a look at https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML#ensemble and let me know if it's what you want.
2. Is the option [blendsearch] installed when you install flaml?

[flippercy]

Hi @sonichi:

1. Yes that's what I want.
2. In the docker file flaml is installed as: 'RUN pip install flaml[blendsearch,ray]'

[sonichi]

I created an issue #789. Would you please test it after it's fixed? Welcome to join our discord to discuss further.

[flippercy]

Thanks @sonichi . Back to question #1 on ensemble; my team created the following codes to use ElasticNet as a customized learner for the ensemble; however, it does not work. Could you tell me what the issues are, please?

from flaml.model import SKLearnEstimator
from sklearn.linear_model import ElasticNet

class MyElasticNetClassifier(SKLearnEstimator):

def __init__(self, positive = True, **params):

    super().__init__(task, **params)

    self.estimator_class = ElasticNet

    # convert to int for integer hyperparameters
    self.params = {
        'random_state': params['random_state'] if 'random_state' in params else randomseed,
        'positive': params['positive'] if 'positive' in params else True,   
        'alpha': params['alpha'], 
   }   

@classmethod
def search_space(cls, data_size, task):
    space = {
        "alpha": {"domain": tune.uniform(lower=0, upper=1), "init_value": 0.5},
    }
    return space

automl.add_learner(learner_name = 'MyElasticNet', learner_class = MyElasticNetClassifier)

settings = {
"ensemble": {"final_estimator": MyElasticNetClassifier,
"passthrough": False,
}
}

Thank you!

[sonichi]

The "final_estimator" expects an instance of the estimator instead of the class. Also, the estimator doesn't need to be registered using add_learner. It can be any scikit-learn style estimator.

[flippercy]

So I guess I can simply use:

"final_estimator": ElasticNet(positive=True),

correct?

[sonichi]

Yes, assuming ElasticNet(positive=True) is a valid sklearn classifier.