Practical hints

CMA-ES requires two components from the user:

• the initial start point x0;
• the initial value for standard deviation sigma, the so-called step-size or error guess.

See http://cma.gforge.inria.fr/cmaes_sourcecode_page.html#practical for another set of detailed useful advices by Nikolaus Hansen on how to best use CMA-ES.

Choosing the initial step-size sigma and initial starting point x0

the optimum that is looked after should better not be far away from the interval [x0 - sigma0, x0 + sigma0] in each dimension, where distance is defined by sigma0.

How to best choose among algorithms and restart strategies (IPOP / BIPOP)

Active CMA-ES as 'aCMAES' as in

CMAParameters<> cmaparams(...);
cmaparams.set_algo(aCMAES);

is probably the best algorithm in most cases.

When optimization fails or fails finding what appears to be a good minimum, it is likely that increasing the number of offsprings per generation 'lambda' might lead to better results. Note that the initial value of sigma and x0 are also very important.

When increasing lambda seems to improve the minimum, it is a good bet to use active BIPOP-CMA-ES (abipop), that applies a successive scaling of lambda in order to explore both locally and globally.

However, bear in mind that both IPOP and BIPOP can greatly increase the number of calls to the objective function. So when calls to the objective function are expensive, this may not be a good strategy.

Note that the maximum number of restarts for IPOP and BIPOP can be controlled through the CMAParameters.set_restart function.

Also, BIPOP can yield best results when allowed to restart from an initial solution candidate within a user-specified domain. This allows a better exploration of solutions and in practice yields slightly better results. To specify a domain for the initial candidate, see the example below, assuming your CMAParameters object is 'cmaparams':

cmaparams.set_x0(-3.0,3.0); // set x0 domain as [-3,3]^d where d is the dimension of the problem

For more granularity, see the various set_x0 functions.

Dealing with large parameter spaces

CMA-ES complexity is quadratic the parameter space dimension. When this number becomes prohibitive, each iteration will become very costly or will exhaust memory.

When facing such a problem, the alternative is to use separable (active) CMA-ES (sepacmaes), a variant that limits the covariance error update to the diagonal elements. The lack in precision then comes at the benefit of a complexity that is linear the parameter space dimension.

Fixing one or more parameter values

Note: use with caution

The library supports fixing one or more parameters' values before optimization takes place or, when more control is required, at each iteration as needed.

This is achieved by calling CMAParameters.set_fixed_p function.

The fixed parameters then retain their values until CMAParameters.unset_fixed_p is called.