Meta-optimization

Optimization methods such as genetic algorithm and differential evolution have several parameters that govern their behaviour and efficiency in optimizing a given problem and these parameters must be chosen by the practitioner to achieve satisfactory results. Selecting the behavioural parameters by hand is a laborious task that is susceptible to human misconceptions of what makes the optimizer perform well.

The behavioural parameters of an optimizer can be varied and the optimization performance plotted as a landscape. This is computationally feasible for optimizers with few behavioural parameters and optimization problems that are fast to compute, but when the number of behavioural parameters increases the time usage for computing such a performance landscape increases exponentially. This is the curse of dimensionality for the search-space consisting of an optimizer's behavioural parameters. An efficient method is therefore needed to search the space of behavioural parameters.

A simple way of finding good behavioural parameters for an optimizer is to employ another overlaying optimizer, called the meta-optimizer. There are different ways of doing this depending on whether the behavioural parameters to be tuned are real-valued or discrete-valued, and depending on what performance measure is being used, etc.

Meta-optimizing the parameters of a genetic algorithm was done by Grefenstette and Keane, amongst others, and experiments with meta-optimizing both the parameters and the genetic operators were reported by Bäck. Meta-optimization of the COMPLEX-RF algorithm was done by Krus and Andersson, and, where performance index of optimization based on information theory was introduced and further developed. Meta-optimization of particle swarm optimization was done by Meissner et al., Pedersen and Chipperfield, and Mason et al. Pedersen and Chipperfield applied meta-optimization to differential evolution. Birattari et al. meta-optimized ant colony optimization. Statistical models have also been used to reveal more about the relationship between choices of behavioural parameters and optimization performance, see for example Francois and Lavergne, and Nannen and Eiben. A comparison of various meta-optimization techniques was done by Smit and Eiben.