**4. Experiments**

10 Will-be-set-by-IN-TECH

*r*<sup>2</sup> = *r*<sup>5</sup> + *c*<sup>3</sup>

The mapping process continues with the four next values, until we are left with the following

*r*<sup>2</sup> = *r*<sup>5</sup> + *c*<sup>3</sup> *r*<sup>0</sup> = *r*<sup>1</sup> × *r*<sup>1</sup>

To finalize the LDEP and CMA-LEP algorithms, the basic idea is to simply plug the float vector to program translation and the virtual machine program evaluation into the DE and CMA-ES schemes. However some technical points need to be taken into account to allow this

We have to decide about the length of the individuals (float vectors) since we usually cannot extract this feature from the problem. This length will determine the maximum number of

Moreover we need to fix a range of possible initial values to randomly generate the

Constant registers are initialized at the beginning of the run, and then are only accessed in read-only mode. This means that our set of constants remains fixed and does not evolve during the run. The number and value range of constant registers are user defined, and the additional parameter *PC* must be set to determine the probability of using a constant register

For LDEP, we tried two variants of the iteration loop described in Section 2.2: either generational replacement of individuals as in the original Storn and Price paper [11], or steady state replacement, which seems to be used in [17]. In the generational case, newly created individuals are stored in a temporary set, and once the generation is completed, they replace their respective parent if their fitness is better. In the steady state scheme, each new individual is immediately compared with its parent and replaces it if its fitness is better, and thus it can be used in remaining crossovers for the current generation. Using the steady state variant seems

During the iteration loop of either LDEP or CMA-LEP, the vector solutions are decoded using equations 6, 7 and 8. The resulting linear programs are then evaluated on a set of fitness cases (training examples). The fitness value is then returned to the evolution engine that continues

So the 4 first values of the genotype are translated as:

program:

**3.3. Algorithm**

*Initialization*

integration and they are detailed below.

instructions allowed in the evolved programs.

in an expression, as explained above in Eq. 8.

to accelerate convergence, see Section 4.

*Main algorithm iteration*

the evolution process.

components of the initial population {*Xi*}1≤*i*≤*N*, as typical in DE.

We use the same benchmark problems as in [17] (4 symbolic regressions and the Santa Fe artificial ant), and we also add two regression problems that include float constants.

Before listing our experimental parameters in Table 1, we explain some of our implementation choices:

	- For regression, 1500 iterations on a population of 20 vectors were allowed, and runs were done for every tree depth in the range {1, . . . , 10}. It thus amounts to a total of 300, 000 evaluations. Among these runs, reference [17] reported only those associated to the tree depth that obtained the best result (which may well imply a favorable bias, in our opinion). As we could not apply this notion of best tree depth in our heuristic, we decided as a trade-off to allow 50, 000 evaluations for regression with both LDEP, CMA-LEP and GP.
	- For the Santa Fe Trail artificial ant problem, the same calculation gives a total of 450, 000 evaluations for TreeDE. We decided for a trade-off of 200, 000 evaluations for LDEP, CMA-LEP and GP.

#### **4.1. Symbolic regression problems**

The aim of these 1-dimensional symbolic regression problems is to find some symbolic mathematical expression (or program) that best approximates a target function that is known only by a set of examples, or fitness cases, (*xk*, *f*(*xk*)). In our case, 20 values *xk* are chosen evenly distributed in the range [−1.0, +1.0]. The evaluation of programs (or *fitness*

<sup>1</sup> http://cs.gmu.edu/~eclab/projects/ecj/

<sup>2</sup> http://www.lri.fr/~hansen/cmaes\_inmatlab.html

computation) is done according to the classic Koza's book [1], that is computing the sum of deviations by looping over all fitness cases:

generational LDEP steady state LDEP TreeDE Problem Fit. % hits Eval. Fit. % hits Eval. Fit. % hits Eval. *f*<sup>1</sup> 0.0 100% 4297 0.0 100% 2632 0.0 100% 1040 *f*<sup>2</sup> 0.0 100% 12033 0.0 100% 7672 0.0 100% 3000 *f*<sup>3</sup> 0.28 72.5% 21268 0.08 85% 21826 0.027 98% 8440 *f*<sup>4</sup> 0.20 62.5% 33233 0.13 75% 26998 0.165 68% 14600

Continuous Schemes for Program Evolution 39

standard GP

Problem Fit. % hits Eval. *f*<sup>1</sup> 0.0 100% 1815 *f*<sup>2</sup> 0.0 100% 2865 *f*<sup>3</sup> 0.03 97% 6390 *f*<sup>4</sup> 0.01 80% 10845 For each heuristic, over 40 independent runs, the column Fit. gives the average of the best fitness (taken from [17] for TreeDE), then we have the percentage of run reaching a hit solution, then the average number of evaluations to produce the first hit solution (if

state variant versus generational, in a limited number of evaluations. This steady state faster

In the second set of experiments, presented in Table 3, heuristics are allowed to evolve programs with constants, thus ruling out TreeDE from the comparison. All problems from *f*<sup>1</sup> to *f*<sup>6</sup> are tested, which means that heuristics manage float constants even on the first 4 problems when they are not needed. This simulates the frequent absence of background

• For LDEP and CMA-LEP, we add 50 constant registers, with a probability of occurrence

• For GP, we define 4 redundant input terminals reading the same input value *xk* for each fitness case (*xk*, *yk*), against only one ephemeral random constant (ERC) terminal, that draws new random value instances when needed, in the range [−1.0, +1.0]. Thus the probability to generate a constant, e.g. during program initialization or in a subtree mutation, is much lower than the usual 50% when having only one *x* terminal. This is

In Table 3, we again observe that the steady state variant of LDEP is better than the generational. For its best version LDEP is comparable to GP, with a slightly higher hit ratio and better average fitness (except on *f*6), with more evaluations on average. For CMA-LEP, two values for *σ* ∈ {1, 10} and two values for *λ* ∈ {10, 100} were tried with no significant

knowledge on a new problem and this also tests the robustness of heuristics.

closer to the LDEP setting and it significantly improves the GP results.

ever produced).

*Runs with constants*

convergence may also benefit to TreeDE.

**Table 2.** Results for symbolic regression problems without constants.

*PC* = 0.05, and initial values in the range [−1.0, +1.0].

$$fitness = \sum\_{1 \le k \le N} |f(\mathbf{x}\_k) - P(\mathbf{x}\_k)|$$

where *P*(*xk*) is the value computed by the evolved program *P* on input *xk*, *f* is the benchmark function and *N* = 20 is the number of (input, output) fitness cases. A *hit solution* means that the deviation is less than 10−<sup>4</sup> on each fitness case.

The first 4 test functions are from [17]:

$$\begin{aligned} f\_1(\mathbf{x}) &= \mathbf{x}^3 + \mathbf{x}^2 + \mathbf{x} \\ f\_2(\mathbf{x}) &= \mathbf{x}^4 + \mathbf{x}^3 + \mathbf{x}^2 + \mathbf{x} \\ f\_3(\mathbf{x}) &= \mathbf{x}^5 + \mathbf{x}^4 + \mathbf{x}^3 + \mathbf{x}^2 + \mathbf{x} \\ f\_4(\mathbf{x}) &= \mathbf{x}^5 - 2\mathbf{x}^3 + \mathbf{x} \end{aligned}$$

While TreeDE benchmarks were run without constants in [17], we strongly believe that it is interesting to use benchmark problems that are expressed as functions both with and without float constants, in order to assess the impact of constant management by the heuristics. Moreover in the general case, especially on real world problems, one cannot know in advance whether or not float constants may be useful. For this reason we add two benchmarks:

$$\begin{array}{l} f\_5(\mathbf{x}) = \pi \quad \text{(a constant function)}\\ f\_6(\mathbf{x}) = \frac{\mathbf{x}}{\pi} + \frac{\mathbf{x}^2}{\pi^2} + 2\mathbf{x}\pi \end{array}$$

The set of operators is {+, −, ×, ÷} with ÷ being the protected division (*i.e. a* ÷ *b* = *a*/*b* if *b* �= 0 else *a* ÷ *b* = 0 if *b* = 0).

For LDEP and CMA-LEP, 6 read-write registers are used for calculation (from *r*<sup>0</sup> to *r*5), with *r*<sup>0</sup> being the output register. For each fitness case (*xk*, *f*(*xk*)) that is submitted to the evolved program inside the evaluation loop, all 6 calculation registers are initialized with the same input value *xk*. This standard LGP practice provides redundancy of the input value and thus more robustness to the run.

#### *Runs without constants*

In the first set of experiments, programs are evolved without constants. This unrealistic setting is proposed here only to allow a comparison of DE-based scheme, confronting LDEP versus Veenhuis's TreeDE, and excluding CMA-LEP. Results are reported in table 2, all three heuristics exhibit close results on the *f*1, *f*2, *f*3, *f*<sup>4</sup> problems, with GP providing the overall most precise approximation, and LDEP needing the largest number of evaluations (notwithstanding the possible bias in the TreeDE figures, as mentioned at the beginning of Section 4). Note that the steady state variant of LDEP converges faster than the generational, as shown by the average number of evaluations for perfect solutions. It seems safe to conclude that this increased speed of convergence is the explanation for the better result of the steady

#### 38 Genetic Programming – New Approaches and Successful Applications Continuous Schemes for Program Evolution <sup>13</sup> Continuous Schemes for Program Evolution 39



For each heuristic, over 40 independent runs, the column Fit. gives the average of the best fitness (taken from [17] for TreeDE), then we have the percentage of run reaching a hit solution, then the average number of evaluations to produce the first hit solution (if ever produced).

**Table 2.** Results for symbolic regression problems without constants.

state variant versus generational, in a limited number of evaluations. This steady state faster convergence may also benefit to TreeDE.

#### *Runs with constants*

12 Will-be-set-by-IN-TECH

computation) is done according to the classic Koza's book [1], that is computing the sum of

where *P*(*xk*) is the value computed by the evolved program *P* on input *xk*, *f* is the benchmark function and *N* = 20 is the number of (input, output) fitness cases. A *hit solution* means that


1≤*k*≤*N*

*f*1(*x*) = *x*<sup>3</sup> + *x*<sup>2</sup> + *x*

*<sup>f</sup>*4(*x*) = *<sup>x</sup>*<sup>5</sup> <sup>−</sup> <sup>2</sup>*x*<sup>3</sup> <sup>+</sup> *<sup>x</sup>*

*<sup>π</sup>* <sup>+</sup> *<sup>x</sup>*<sup>2</sup>

*f*6(*x*) = *<sup>x</sup>*

*f*2(*x*) = *x*<sup>4</sup> + *x*<sup>3</sup> + *x*<sup>2</sup> + *x*

*f*3(*x*) = *x*<sup>5</sup> + *x*<sup>4</sup> + *x*<sup>3</sup> + *x*<sup>2</sup> + *x*

While TreeDE benchmarks were run without constants in [17], we strongly believe that it is interesting to use benchmark problems that are expressed as functions both with and without float constants, in order to assess the impact of constant management by the heuristics. Moreover in the general case, especially on real world problems, one cannot know in advance whether or not float constants may be useful. For this reason we add two benchmarks:

*f*5(*x*) = *π* (a constant function)

The set of operators is {+, −, ×, ÷} with ÷ being the protected division (*i.e. a* ÷ *b* = *a*/*b* if

For LDEP and CMA-LEP, 6 read-write registers are used for calculation (from *r*<sup>0</sup> to *r*5), with *r*<sup>0</sup> being the output register. For each fitness case (*xk*, *f*(*xk*)) that is submitted to the evolved program inside the evaluation loop, all 6 calculation registers are initialized with the same input value *xk*. This standard LGP practice provides redundancy of the input value and thus

In the first set of experiments, programs are evolved without constants. This unrealistic setting is proposed here only to allow a comparison of DE-based scheme, confronting LDEP versus Veenhuis's TreeDE, and excluding CMA-LEP. Results are reported in table 2, all three heuristics exhibit close results on the *f*1, *f*2, *f*3, *f*<sup>4</sup> problems, with GP providing the overall most precise approximation, and LDEP needing the largest number of evaluations (notwithstanding the possible bias in the TreeDE figures, as mentioned at the beginning of Section 4). Note that the steady state variant of LDEP converges faster than the generational, as shown by the average number of evaluations for perfect solutions. It seems safe to conclude that this increased speed of convergence is the explanation for the better result of the steady

*<sup>π</sup>*<sup>2</sup> + 2*xπ*

*fitness* = ∑

deviations by looping over all fitness cases:

the deviation is less than 10−<sup>4</sup> on each fitness case.

The first 4 test functions are from [17]:

*b* �= 0 else *a* ÷ *b* = 0 if *b* = 0).

more robustness to the run.

*Runs without constants*

In the second set of experiments, presented in Table 3, heuristics are allowed to evolve programs with constants, thus ruling out TreeDE from the comparison. All problems from *f*<sup>1</sup> to *f*<sup>6</sup> are tested, which means that heuristics manage float constants even on the first 4 problems when they are not needed. This simulates the frequent absence of background knowledge on a new problem and this also tests the robustness of heuristics.


In Table 3, we again observe that the steady state variant of LDEP is better than the generational. For its best version LDEP is comparable to GP, with a slightly higher hit ratio and better average fitness (except on *f*6), with more evaluations on average. For CMA-LEP, two values for *σ* ∈ {1, 10} and two values for *λ* ∈ {10, 100} were tried with no significant

#### 14 Will-be-set-by-IN-TECH 40 Genetic Programming – New Approaches and Successful Applications Continuous Schemes for Program Evolution <sup>15</sup>

differences. In contrast with the other methods, CMA-LEP results are an order of magnitude worse. Tuning the CMA-ES engine to tackle the problem as separable did not improve the results. We think this behavior may result from the high dimensionality of the problem (N=128), that certainly disrupts the process of modeling an ideal mean solution from a comparatively tiny set of search points. This is combined to the lack of elitism, inherent to the CMA-ES method, thus when it comes to generate new test points, the heuristic is left solely with a probably imperfect model.


For each heuristic, over 40 independent runs, the column Fit. gives the average of the best fitness, then we have the percentage of run reaching a hit solution, then the average number of evaluations to produce the first hit solution (if ever produced or else NA if no run produced a hit solution).

**Figure 1.** Illustration of the Santa Fe Trail (the ant starts in the upper left corner, heading to the east,

it contains a piece of food or not. Turns and moves cost one time step, and a maximum time steps threshold is set at start (typical values are either 400 or 600 time steps). If the program finishes before the exhaustion of the time steps, it is restarted (which amounts to iterating the

Continuous Schemes for Program Evolution 41

We do not need mathematical operators nor registers, only the following instructions are

• MOVE: moves the ant forward one step (grid cell) in the direction the ant is facing, retrieving

• IF-FOOD-AHEAD: conditional statement that executes the next instruction or group of instructions if a food pellet is located on the neighboring cell in front of the ant, else the

• PROGN2: groups the two instructions that follow in the program vector, notably allowing IF-FOOD-AHEAD to perform several instructions if the condition is true (the PROGN2

• PROGN3: same as the previous operator, but groups the three following instructions.

large dots are food pellets, and small dots are empty cells on the ideal path).

operator does not affect *per se* the ant position and direction);

• Each MOVE, RIGHT and LEFT instruction requires one time step.

an eventual food pellet in the cell of arrival; • LEFT: turns on place 45 degrees anti-clockwise; • RIGHT: turns on place 45 degrees clockwise;

next instruction or group is skipped;

whole program).

available:

**Table 3.** Results for symbolic regression problems with constants.

Overall, these results confirm that DE is an interesting heuristic, even when the continuous representation hides a combinatorial type problem, and thus the heuristic is used outside its original field. The LDEP mix of linear programs and constant management appears competitive with the standard GP approach.
