[caption] budget crosses in runtime distributions

[nikohansen]

The crosses are now in better alignment with the new experimental restarts setup.

Caption

Two big (thin) crosses appear on the graphs when at least one target was reached and at least one trial did not reach the final target (was "unsuccessful"). The crosses depict the median over the considered functions (above we see only single functions) of +: the larger of the two 90%tiles of runtimes from all successful/unsuccessful single trials (from all instances) and of x: the 90%tile of the sum(runtimes) / max(1, #successes) of each instance which equals the instance-wise ERT when we have at least one success. Usually, + is to the left of x and the latter cross might be missing. A small dot indicates the last step of the step function. Roughly speaking, runtimes (i.e. steps) beyond + are generated by bootstrapping of data. Data should be interpreted with great care (or not at all) beyond x and also beyond + if the termination of trials was not induced by the algorithm but imposed by the user (e.g., by an experimentation budget).