Auto FLAML code/api generation

[qingyun-wu]

Use LLM or LLM-based agents to generate FLAML code for users who need to perform automl or tuning tasks.

May need more input from @thinkall

Tasks

Beta Give feedback

1. Survey potential solutions from existing work. E.g., function call of GPT, Gorilla, TaskMatrix.AI. See if we can achieve it during a straightforward solution, e.g., GPT4 + function call, or solutions from existing work.

[thinkall]

Maybe we can organize flaml's APIs and leverage function call of GPT and the existing AssistantAgent and UserProxyAgent.

A use case could be like:

from flaml.autogen.agent import AssistantAgent, UserProxyAgent
assistant = AssistantAgent("assistant", request_timeout=600, seed=42, config_list=config_list)
user = UserProxyAgent(
    "user",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE") or x.get("content", "").rstrip().endswith('"TERMINATE".'),
    work_dir="coding",
    use_docker=False,  # set to True if you are using docker
)

assistant.receive(
    """Generate AutoML code with FLAML for my data x_train and y_train, it's a classification task and I want to train the model in 30 mins.""",
    user,
)

An acceptable result could be like:

import flaml

# Define the search space for FLAML
settings = {
    "time_budget": 1800,  # Time budget in seconds (30 minutes)
    "task": "classification",
    "eval_metric": "accuracy",
}

# Create an AutoML object
automl = flaml.AutoML()

# Train the model using FLAML
automl.fit(x_train=x_train, y_train=y_train, **settings)

# Get the best model found by FLAML
best_model = automl.model

# Train the best model on the entire dataset
best_model.fit(x_train, y_train)

# Save the best model for future use
best_model.save_model("best_model.pkl")

The current version of ChatGPT may provide a more comprehensive code for this query, but it might be excessive. Moreover, new APIs in FLAML is not included in ChatGPT.