Use cases that could be simplified

[nikohansen]

This issue is meant to collect use cases which seem to be too complicated to achieve in some way or another, or even "non-existing" use cases (AKA missing features, meaning they are way too complex to do on the fly). It may serve as a guide for a reimplementation of cocopp.

[nikohansen]

This is how we can display the (simulated) run length distribution over all problems in 10-D from the experiment data exdata/fmin_slsqp_on_bbob_budget0100xD in a Jupyter IPython notebook:

from IPython.display import SVG, display
import cocopp
dsl = cocopp.load("exdata/fmin_slsqp_on_bbob_budget0100xD")
dsl10 = dsl.dictByDim()[10]
cocopp.compall.pprldmany.main(dsl10.dictByAlg())
SVG('pprldmany_default.svg')

This is not overly long, but it could still be shorter and, more importantly, without precisely to know what to do it is virtually impossible to figure out what to do in reasonable time.

Specifically, dictByAlg is superfluous and only necessary, because main bails out otherwise (which should be easily fixable), compall.pprldmany isn't a particularly user-friendly identifier,...

Something like

data_folder = "exdata/fmin_slsqp_on_bbob_budget0100xD"
cocopp.ecdf.plot(cocopp.load(data_folder).by('dim')[10])

looks nicer. It remains to be seen how the resulting figure can be communicated though.

[brockho]

Setting the minimal/hardest target value within COCO's output (or more general the overall set of targets to be displayed in the figures and/or tables) is another quite common use case.

This seems, though possible, also too complicated at the moment, e.g. to change the minimal/hardest target to 1e-6 for the standard bbob test suite and the ECDF plots for 2+ algorithms one has to type:

In [1]: import numpy as np
In [2]: import cocopp
# setting 41 targets equidistant in log-scale between `100` and `1e-6`:
In [3]: cocopp.testbedsettings.GECCOBBOBTestbed.settings["pprldmany_target_values"] = 
   ...: 10 ** np.arange(2, -6.2, -0.2) 
In [4]: cocopp.main(['BFGS_ros_noiseless.tgz', 'BIPOP-CMA-ES_hansen_noiseless.tgz'])
# setting things back to default:
In [5]: cocopp.testbedsettings.GECCOBBOBTestbed.settings["pprldmany_target_values"] = 
   ...: 10 ** np.arang e(2, -8.2, -0.2)
In [6]: cocopp.main(['BFGS_ros_noiseless.tgz', 'BIPOP-CMA-ES_hansen_noiseless.tgz'])

In the bbob-constrained and bbob-biobj cases, we actually do change the displayed targets for all plots and tables in a (relatively) generic way, but which needs quite some ugly setup:

min_target = 1e-6
min_target_latex = '-6'
min_target_exponent = -6.2
min_target_scatter = -6