Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change cost_attr when max_iter is used as the stopping criterion instead of time_budget in AutoML.fit #789

Closed
sonichi opened this issue Nov 2, 2022 Discussed in #779 · 1 comment
Labels
enhancement New feature or request

Comments

@sonichi
Copy link
Contributor

sonichi commented Nov 2, 2022

Discussed in #779

Originally posted by flippercy October 28, 2022
Hi Team FLAML:

I have a few general questions regarding FLAML:

  1. Could I use a customized learner for the ensemble? For example, for a linear combination, I want to make sure all the coefficients are negative.

  2. I noticed that the search of automl is much more biased when running distributedly on compute clusters, which means that it focuses much more on 'strong' learners. For example, this is the summary from my recent search:
    image
    Obviously FLAML spent the majority of time on LearnerE, which is the optimal one; it is understandable, however, the distribution of attempts on different learners is much more balanced when running FLAML on a VM. Could you explain this phenomenon, please? Usually we use FLAML not just to look for the overall optimal solution, but also the best result per learner.

Thank you!

If we set cost_attr for BlendSearch as None, then the number of iterations will be used as cost measurement. It's supposed to allocate # trials to different learners in a more balanced manner in the parallel setting.

@sonichi sonichi added the enhancement New feature or request label Nov 2, 2022
@sonichi
Copy link
Contributor Author

sonichi commented Dec 11, 2022

closed by #837

@sonichi sonichi closed this as completed Dec 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant