Considerations for a noisy biobjective test suite

[brockho]

When discussing possible next steps towards noise and COCO during our COCO sprint, we came up with the idea of a noisy version of the bbob-biobj(-ext) suite.

Here some notes to ourselves to consider for the implementation:

• wrap noise around the bbob-biobj-ext functions
• only one noise type: uniform noise (Gaussian noise does not make sense to be used if we are anyway not close to the single-objective optima)
• we have to be careful with the seeds of the objective-wise noise-wrappers: they must have different seeds because otherwise, we will see correlated noise. A simple way is to use the single objective functions' instance ids as seeds.
• if we continue to use the usual performance assessment with the noise-free objective function values (similar to the single-objective case), we can reuse the non-dominated archives to get the hypervolume reference values (they will need to updated in case we restrict the search to $[-5, 5]^n$, see below)

Open questions:

• restrict the suite to $[-5, 5]^n$ or not?
• naming: do we want to call it bbob-biobj-ext-noisy or bbob-biobj-noisy or simply biobj-noisy even if we have 92 functions? [we also call the noisy single-objective suite bbob-noisy although the number of functions changes from 24 to 30]