[tune] Add Initial Parameter Suggestion for HyperOpt

[markgoodhead]

What do these changes do?

Allows users of the HyperOptSearch suggestion algorithm to specify initial experiment values to run (typically already known good baseline parameters within the domain specified)

Related issue number

Discussed in the ray dev forums

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11531/
Test PASSed.

[richardliaw]

Can you amend the tune/examples/hyperopt_example.py with this feature? That would ensure it runs in the CI.

Right now, I did a quick test of this and I got an error:

Traceback (most recent call last):
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/hyperopt/pyll/base.py", line 868, in rec_eval
    int(switch_i)
ValueError: invalid literal for int() with base 10: 'relu'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "hyperopt_example.py", line 65, in <module>
    run_experiments(config, search_alg=algo, scheduler=scheduler)
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/tune.py", line 171, in run_experiments
    runner.step()
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 224, in step
    next_trial = self._get_next_trial()
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 371, in _get_next_trial
    self._update_trial_queue(blocking=wait_for_trial)
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 483, in _update_trial_queue
    trials = self._search_alg.next_trials()
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/suggest/suggestion.py", line 65, in next_trials
    for trial in self._trial_generator:
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/suggest/suggestion.py", line 86, in _generate_trials
    suggested_config = self._suggest(trial_id)
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/ray/tune/suggest/hyperopt.py", line 123, in _suggest
    print_node_on_error=self.domain.rec_eval_print_node_on_error)
  File "/data/rliaw/miniconda3/envs/ray/lib/python3.6/site-packages/hyperopt/pyll/base.py", line 870, in rec_eval
    raise TypeError('switch argument was', switch_i)
TypeError: ('switch argument was', 'relu')

Code I ran:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import ray
from ray.tune import run_experiments, register_trainable
from ray.tune.schedulers import AsyncHyperBandScheduler
from ray.tune.suggest import HyperOptSearch


def easy_objective(config, reporter):
    import time
    time.sleep(0.2)
    assert type(config["activation"]) == str, \
        "Config is incorrect: {}".format(type(config["activation"]))
    for i in range(config["iterations"]):
        reporter(
            timesteps_total=i,
            neg_mean_loss=-(config["height"] - 14)**2 +
            abs(config["width"] - 3))
        time.sleep(0.02)


if __name__ == '__main__':
    import argparse
    from hyperopt import hp

    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--smoke-test", action="store_true", help="Finish quickly for testing")
    args, _ = parser.parse_known_args()
    ray.init(redirect_output=True)

    register_trainable("exp", easy_objective)

    space = {
        'width': hp.uniform('width', 0, 20),
        'height': hp.uniform('height', -100, 100),
        'activation': hp.choice("activation", ["relu", "tanh"])
    }

    current_best_params = [
        {"width": 1, "height": 2, "activation": "relu"},
        {"width": 4, "height": 2, "activation": "relu"}
    ]

    config = {
        "my_exp": {
            "run": "exp",
            "num_samples": 10 if args.smoke_test else 1000,
            "config": {
                "iterations": 100,
            },
            "stop": {
                "timesteps_total": 100
            },
        }
    }
    algo = HyperOptSearch(space, max_concurrent=4, reward_attr="neg_mean_loss", points_to_evaluate=current_best_params)
    scheduler = AsyncHyperBandScheduler(reward_attr="neg_mean_loss")
    run_experiments(config, search_alg=algo, scheduler=scheduler)

Is there something I did wrong?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11533/
Test PASSed.

[markgoodhead]

Ah almost certainly a bug on my end, I only tested it with non-choice params. Let me investigate and I'll fix it up and provide the example you suggest

[markgoodhead]

@richardliaw The issue was that generate_trials_to_evaluate wants the index of the choice rather than the choice value itself. I considered trying to keep the same interface and to work it out for the user but it'd be a bit messy, so just opted for explaining this in the example / documentation as it's not that complex (and will also be more of a 'drop in' for users of hyperopt who want to run their code on Tune).

I've fixed up the example and it runs fine for me now. Let me know your thoughts and if you've any further changes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11611/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11640/
Test FAILed.

[markgoodhead]

jenkins retest this please

[markgoodhead]

^ is that how you're meant to kick off new tests if the previous ones failed due to unrelated flakiness?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11649/
Test PASSed.

[richardliaw]

yep - looks good, thanks a bunch for this contribution!