hyperband fixes

[donglinjy]

This PR include below fixes for Hyerband:

• Compute budgets does not output the same table as the original paper
  Fix: Besides the fix, also add a debug log to show the budget table like this:

• Duplicate points in a single hyperband execution will cause some points been tried but won't be able to register into the correct budget since we choose bracket one time based on fidelity.
  Fix: Won't allow repeated points in a single hyperband execution. And if the search space is too small to fill the first rungs of all brackets, hyperband won't be able to promote and experiment only run the points of the first rungs and exit.

• Support to run duplicate points in different hyperband execution only if the later one get bigger fidelity compared with previous execution.

• When multiple workers run in parallel, there are 2 issues:

  • Some workers won't be able to exit normally when some trials is running and naive algorithm is done but source algorithm is not.
    Fix: Sync the executed number of times from naive algorithm to source.
  • When hyberband.observe with points belong to different hyperband executions, points could be registered to the bracket of a wrong hyperband executions.
    Fix: Try to refresh brackets of hyperband after observe of each point.

• Add documentation for Hyperband

@bouthilx

[codecov-io]

Codecov Report

Merging #363 into develop will decrease coverage by 0.10%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop     #363      +/-   ##
===========================================
- Coverage    45.55%   45.45%   -0.11%     
===========================================
  Files           67       67              
  Lines        12270    12245      -25     
  Branches       318      306      -12     
===========================================
- Hits          5590     5566      -24     
+ Misses        6655     6654       -1     
  Partials        25       25              

Impacted Files	Coverage Δ
tests/unittests/algo/test_hyperband.py	100.00% <100.00%> (+0.20%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 40cc85d...f18c972. Read the comment docs.

[bouthilx]

Great! 😃 I will look at the PR in details tonight or tomorrow.

[bouthilx]

Fixes for Hyperband

[bouthilx]

Suggested change

	============================================================================================
	===========================================================================================

[bouthilx]

LGTM beside my few minor comments. Thanks! :)

[donglinjy]

Seems only osx test failed, maybe can try restart travis build @bouthilx

[donglinjy]

trvais failed again. There seems a potential bug of space transformer when the lower/upper bound value suggested from the original space.

[bouthilx]

Indeed. I saw this issue in the stress test I am implementing. I should fix it this week.