[tune] Initial Commit for Tune CLI

[richardliaw]

This introduces a light CLI for Tune.

TODOs:

• Make sure the columns are scoped to the terminal
• Add documentation onto doc/source/tune-usage.rst
• Verify that install enables tune ls

cc: @andrewztan

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11667/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11715/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12225/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12301/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12657/
Test FAILed.

[hartikainen]

$ tune list-trials ~/ray_results/gym/GoalHalfCheetah/v0/2019-03-03T19-19-32-goal-conditioned-sac-test-1/
Traceback (most recent call last):
  File "/home/kristian/conda/envs/softlearning/bin/tune", line 11, in <module>
    load_entry_point('ray', 'console_scripts', 'tune')()
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/scripts.py", line 20, in list_trials
    commands.list_trials(experiment_path, sort)
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/commands.py", line 145, in list_trials
    lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/pandas/core/series.py", line 3194, in apply
    mapped = lib.map_infer(values, f, convert=convert_dtype)
  File "pandas/_libs/src/inference.pyx", line 1472, in pandas._libs.lib.map_infer
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/commands.py", line 145, in <lambda>
    lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
OverflowError: timestamp out of range for platform time_t

ipdb> l
    142     if "last_update_time" in checkpoints_df:
    143         from pprint import pprint; import ipdb; ipdb.set_trace(context=30)
    144
    145         datetime_series = checkpoints_df["last_update_time"].dropna()
    146         datetime_series = datetime_series.apply(
--> 147             lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
    148         checkpoints_df["last_update_time"] = datetime_series
    149
    150     if "logdir" in checkpoints_df:
    151         # logdir often too verbose to view in table, so drop experiment_path
    152         checkpoints_df["logdir"] = checkpoints_df["logdir"].str.replace(

ipdb> checkpoints_df
     trainable_name       ...        last_update_time
0  ExperimentRunner       ...                    -inf
1  ExperimentRunner       ...                    -inf
2  ExperimentRunner       ...                    -inf

[3 rows x 5 columns]

[hartikainen]

I did some testing with this. Overall it feels very neat, but there were some problems with the tune list-trials timestamp parsing (see the error above).

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12668/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12680/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12681/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12688/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12699/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12700/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12704/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12710/
Test FAILed.

[richardliaw]

Jenkins tests passed before last commit, which was only a test change; relevant travis tests passed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12714/
Test PASSed.

[andrewztan]

should this be assert sum("1" in line for line in lines) >= num_experiments instead of 3