[tune] Support for multi-objective optimization?

[bitosky]

What is your question?

In the documents, "metric" refers to "the training result objective value attribute".It should be set to a single string, such as "mean_loss" or "episode_reward_mean" etc.
It's there any way to tackle multi-objective optimization problem? For example, ("mean_loss", "accuracy" , "something else") as a metric.

[richardliaw]

One way you can do this is by having a weighted sum of the objective values -

result = {"multiobjective": a * mean_loss + b * accuracy}

Some underlying packages such as Dragonfly (https://dragonfly-opt.readthedocs.io/en/master/getting_started_py/) and Ax (facebook/Ax#185) also have multi-objective optimization support. You should hypothetically be able to get them working with RayTune.

[bitosky]

@richardliaw Thanks for answering my question. But I still have some problem.
The schedulers are also use "metric", and there is no underlying packages. What should I do to tackle multi-objective optimization problem if I don't use the weighted sum trick?

async_hb_scheduler = AsyncHyperBandScheduler(
    time_attr='training_iteration',
    metric='episode_reward_mean',
    mode='max',
    max_t=100,
    grace_period=10,
    reduction_factor=3,
    brackets=3)

Besides, it's there a plan for supporting multi-objective optimization?

[richardliaw]

For existing schedulers, I have no immediate plan to support multi-objective optimization. However, I'm happy to take any contributions. Note that this is not hard to do - adding multi-objective optimization does not require any lower-level modifications to the framework.

I'll close this for now, but feel free to reopen if you have any questions.

[aswanthkrishna]

were you able to get this to work? @bitosky

[bitosky]

were you able to get this to work? @bitosky

@aswanthkrishna I didn't solve it directly. I wrote my own scheduler and analyzer to deal with multi-objective optimization problems. It was not easy for me, And I spent quite a while to do that.

[sgbaird]

@bitosky do you mind sharing your custom scheduler and analyzer example?

Also came across Optuna Multi-objective Example in the Ray docs. It mentions that schedulers may not work correctly with multi-objective optimization.

[sidharrth2002]

Hi, @bitosky just wanted to follow up on this! An example of your custom scheduler would be super-helpful!

[sgbaird]

Maybe worth mentioning https://honegumi.readthedocs.io/ with the multi-objective selection set to True (without early stopping)