**3.1. Quantitative genetics basics**

Quantitative Genetics theory [9, 31]is concerned with tracking quantitative phenotypic traits within an evolving population in order to analyze the evolutionary process. One group that commonly uses the approach are animal breeders for the purpose of estimating what would be involved in accentuating certain traits (such as size, milk production or pelt color) within their populations.

A quantitative trait is essentially any aspect of an individual that can be measured. Since much of the theory was developed before the structure of DNA was known, traits have tended to measure phenotypic qualities like the ones listed in the paragraph above. Traits can measure real values, integer or boolean (threshold) properties, although real valued properties are generally preferred [9].

This approach offers a potential advantage to EC practitioners. Most EC theory is defined in terms of the underlying representation. As a consequence, it becomes difficult to adapt these theories to new types of problems and representations when they are developed. This generally means that the practitioner must modify or re-derive the theoretical equations before they can apply these theories to a new EA that has been customized for a new problem. For the few theories where this is not the case, a detailed understanding of the problem landscape is typically needed instead. Again this presents problems for the practitioner. After all, if they knew this much about their problem, they would not need an EA to solve it in the first place. Quantitative genetics is one of the few theories that does not suffer from these problems.

Populations are modeled as probability distributions of traits by using simple statistical measures like mean, variance and covariance. A set of equations then describe how the distributions change from one generation to the next as a result of certain evolutionary forces like selection and heritability.

An extended version of the theory called multivariate quantitative genetics [13] aims to model the behaviors and interactions of multiple traits within the population simultaneously. This approach represents multiple traits as a vector. As a result, means are also represented as a vector, and variance calculations produce covariance matrices, as do cross-covariance calculations. In other words, a vector and a covariance matrix are needed to describe a joint probability distribution. Other than this change, the equations remain largely the same.

It is difficult to do any long term prediction with this theory [11]. Instead, its value lies in its ability to perform analysis after the fact [11]. In other words, for our purposes the theory is most useful for understanding the forces at work inside an existing algorithm during or after it has been run, rather than predicting how an proposed algorithm might work.

**P**, **O**, **D** are all covariance matrices of the traits defined as **P** = Var(*φλ*), **O** = Var(*φ*�

*<sup>λ</sup>*� − *φλ*). **G**� is a cross-covariance matrix defined as **G**� = Cov(*φ*�

**O** = **P**[2**G**�

low, that indicates that there is an unexpected bias in the search.

it is measuring the larger scale behavior of the EA.

*3.1.1. Scalar metric for matrices and vector operations*

generation to the next using the following equation:

Given these matrices, we can now describe how the population distributions change from one

where **I** is the identity matrix. In this case, we can view everything within the brackets as defining a transformation matrix that describes how the trait distribution of the selected parents (**P**) is transformed by the operators into the distribution of the offspring population

genetics notion of narrow-sense heritability (commonly just called heritability). It describes the average similarity between an offspring and *one* of it's parents. The term **DP**−1, which we refer to as perturbation, describes the amount of new phenotypic variation that the operators are introducing into the population relative to what already exists. Perturbation can be thought of as measuring an operator's capacity for exploration, while heritability provides an indication of it's ability to exploit existing information in the population. If heritability is

Another relationship that can be drawn from equation 2 is **OP**−1. This does not have a corresponding concept in biology, although it is similar in some ways to broad-sense heritability and repeatability. This term describes the similarity of the parent and offspring *populations*, and so we refer to it as population heritability. This is another measure of exploitation, in addition to narrow-sense heritability. We think it is the better choice because

Biologists consider the multivariate notion of heritability as the degree of similarity between the two probability distributions that **P** and **G** describe. These comparisons are often performed using statistical techniques like Common Principle Component Analysis [10, 12]. For simplicity and ease of understanding, it would be ideal to find a metric that expresses terms like heritability and perturbation as a single scalar value. We have chosen to use the

, *P*) = tr(**G**�

where m is the metric function, and **G**� and **P** are *M* by *M* covariance matrices as described in

The result of equation 3 is, of course, our scalar version of heritability from a single parent. Similarly, tr(**D**)/tr(**P**) would measure perturbation, and tr(**O**)/tr(**P**) gives us a measure of the overall similarity between the selected parent population and the resulting offspring

We chose to use trace because they have an intuitive geometric interpretation. The trace functions is equal to the sum of the diagonal elements in the matrix. It's also equal to the

m(**G**�

**P**−<sup>1</sup> is a regression coefficient matrix, and it is very similar to the quantitative

**D** = Var(*φ*�

traits (**O**). The factor **G**�

following metric,

the previous section.

population.

which can be rewritten as

), and

*<sup>λ</sup>*� , *φλ*).

**O** = 2**G**� + **D** − **P**, (1)

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 7

**<sup>P</sup>**−<sup>1</sup> <sup>+</sup> **DP**−<sup>1</sup> <sup>−</sup> **<sup>I</sup>**], (2)

)/tr(**P**) (3)

**Figure 1.** A sample generation showing offspring and parents [3]

In previous work [3], we adapted multivariate quantitative genetics for use with evolutionary algorithms. The goal of that work was to demonstrate how these theories can be used to aid in customizing EA operators for new and unusual problems. Here we will review some important aspects of that model.

To describe the equations, we will refer to Figure 1, which shows a directed graph illustrating two successive generations during an EA run. A subset of parents (left) are selected and produce offspring, either through crossover, mutation or cloning. Directed edges are draw from each selected parent to all the offspring (right) that it produces. Because the quantitative genetics models are built on the idea of a generational evolutionary process, they are most easily applied to to generational EAs like GAs and GP.

It is important that each directed edge represent the same amount of "influence" that a parent has on its offspring. In the figure, each edge represents an influence of 1/2. That is why two edges are drawn between parent and offspring in instances where only cloning or mutation are performed. A vector of quantitative traits *φ<sup>i</sup>* is associated with each parent *i* and another vector of traits *φ <sup>j</sup>* is associated with each offspring *j*. The two functions *λ*(*k*) and *λ* (*k*) are defined such that they return the index of corresponding parent and offspring, for a given edge *k*.

We also use the abbreviations *φ* and *φ* to describe all the traits in the different populations. The symbol *φ* describes all the parent traits, while *φ* describes all offspring traits. Similarly *φλ* refers to all traits of the *selected* parents, and *φ <sup>λ</sup>* again refers to all the traits of the offspring, although in the case of figure 1 there are two copies of each child.

Several covariance matrices are defined to describe the populations distributions and the forces that cause them to change. **P** and **O** are covariance matrices that describe the distributions of the *selected* parent and offspring populations respectively. **D** describes the amount of trait variation that the operators are adding to the offspring, and **G** can be thought of as quantifying the amount of variation from **P** that is retained in **O**.

**P**, **O**, **D** are all covariance matrices of the traits defined as **P** = Var(*φλ*), **O** = Var(*φ*� ), and **D** = Var(*φ*� *<sup>λ</sup>*� − *φλ*). **G**� is a cross-covariance matrix defined as **G**� = Cov(*φ*� *<sup>λ</sup>*� , *φλ*).

Given these matrices, we can now describe how the population distributions change from one generation to the next using the following equation:

$$\mathbf{O} = 2\mathbf{G}' + \mathbf{D} - \mathbf{P},\tag{1}$$

which can be rewritten as

4 Will-be-set-by-IN-TECH

 


In previous work [3], we adapted multivariate quantitative genetics for use with evolutionary algorithms. The goal of that work was to demonstrate how these theories can be used to aid in customizing EA operators for new and unusual problems. Here we will review some

To describe the equations, we will refer to Figure 1, which shows a directed graph illustrating two successive generations during an EA run. A subset of parents (left) are selected and produce offspring, either through crossover, mutation or cloning. Directed edges are draw from each selected parent to all the offspring (right) that it produces. Because the quantitative genetics models are built on the idea of a generational evolutionary process, they are most

It is important that each directed edge represent the same amount of "influence" that a parent has on its offspring. In the figure, each edge represents an influence of 1/2. That is why two edges are drawn between parent and offspring in instances where only cloning or mutation are performed. A vector of quantitative traits *φ<sup>i</sup>* is associated with each parent *i* and another

defined such that they return the index of corresponding parent and offspring, for a given

We also use the abbreviations *φ* and *φ* to describe all the traits in the different populations. The symbol *φ* describes all the parent traits, while *φ* describes all offspring traits. Similarly

Several covariance matrices are defined to describe the populations distributions and the forces that cause them to change. **P** and **O** are covariance matrices that describe the distributions of the *selected* parent and offspring populations respectively. **D** describes the amount of trait variation that the operators are adding to the offspring, and **G** can be thought

*<sup>j</sup>* is associated with each offspring *j*. The two functions *λ*(*k*) and *λ*

*<sup>λ</sup>* again refers to all the traits of the offspring,

(*k*) are



**Figure 1.** A sample generation showing offspring and parents [3]

easily applied to to generational EAs like GAs and GP.

*φλ* refers to all traits of the *selected* parents, and *φ*

although in the case of figure 1 there are two copies of each child.

of as quantifying the amount of variation from **P** that is retained in **O**.

important aspects of that model.

vector of traits *φ*

edge *k*.

$$\mathbf{O} = \mathbf{P}[2\mathbf{G}'\mathbf{P}^{-1} + \mathbf{D}\mathbf{P}^{-1} - \mathbf{I}],\tag{2}$$

where **I** is the identity matrix. In this case, we can view everything within the brackets as defining a transformation matrix that describes how the trait distribution of the selected parents (**P**) is transformed by the operators into the distribution of the offspring population traits (**O**).

The factor **G**� **P**−<sup>1</sup> is a regression coefficient matrix, and it is very similar to the quantitative genetics notion of narrow-sense heritability (commonly just called heritability). It describes the average similarity between an offspring and *one* of it's parents. The term **DP**−1, which we refer to as perturbation, describes the amount of new phenotypic variation that the operators are introducing into the population relative to what already exists. Perturbation can be thought of as measuring an operator's capacity for exploration, while heritability provides an indication of it's ability to exploit existing information in the population. If heritability is low, that indicates that there is an unexpected bias in the search.

Another relationship that can be drawn from equation 2 is **OP**−1. This does not have a corresponding concept in biology, although it is similar in some ways to broad-sense heritability and repeatability. This term describes the similarity of the parent and offspring *populations*, and so we refer to it as population heritability. This is another measure of exploitation, in addition to narrow-sense heritability. We think it is the better choice because it is measuring the larger scale behavior of the EA.

#### *3.1.1. Scalar metric for matrices and vector operations*

Biologists consider the multivariate notion of heritability as the degree of similarity between the two probability distributions that **P** and **G** describe. These comparisons are often performed using statistical techniques like Common Principle Component Analysis [10, 12].

For simplicity and ease of understanding, it would be ideal to find a metric that expresses terms like heritability and perturbation as a single scalar value. We have chosen to use the following metric,

$$\operatorname{tr}(\mathbf{G}', P) = \operatorname{tr}(\mathbf{G}') / \operatorname{tr}(\mathbf{P}) \tag{3}$$

where m is the metric function, and **G**� and **P** are *M* by *M* covariance matrices as described in the previous section.

The result of equation 3 is, of course, our scalar version of heritability from a single parent. Similarly, tr(**D**)/tr(**P**) would measure perturbation, and tr(**O**)/tr(**P**) gives us a measure of the overall similarity between the selected parent population and the resulting offspring population.

We chose to use trace because they have an intuitive geometric interpretation. The trace functions is equal to the sum of the diagonal elements in the matrix. It's also equal to the

6 Will-be-set-by-IN-TECH 8 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>7</sup>

sum of the eigenvalues of the matrix. In geometric representation it shows the sum total of all the variation in the distribution. Determinants are normally used as single measure for matrix operation. It was observed that determinants couldn't be computed for representation like GP, due to generation of individuals that can lead to non-positive semidefinite matrices.

• Competitive and Co-Competitive Individuals

• Cooperative Individuals

success or failures, etc.

• Design based Individuals

• Regression based Individuals

phenotypic search behaviors.

2. Aggregated Fitness Functions

traits.

*3.2.1. Issues*

are discussed in this section.

• Coverage/Completeness

Many agent-based systems are competitive in nature, like the predator and prey class of the problems. Effective traits that determine metrics leading to success and failure of competing individuals may be more useful than agent-based traits. For instance, in a predator-prey based agents the fitness is basically how well you are doing against the other. If lower level details like "closeness" to the others, number of moves till attacked, number of changes in directions while moving, etc.

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 9

Another subclass of the agent based problems is the cooperative based agents. These individuals have to be in some kind of team to accomplish the goal. The individual behaviors can be specific decomposable ones or can be evolved during the execution. The performance evaluation of most fitness functions in these domains is measured by weighting individual and team performances. Various cooperative metrics can be measured again at different levels like attempts of cooperative moves, success and failures in the moves, ratio of total attempts to the

Many GP applications are used mostly in design sub class of problems like circuit design, layout and network design and plan generation problems. Each of these use very high level measures combined in weights like the cost saved, components used, power distribution, etc. Again, using individualized measures and adding as many metrics that are circuit or layout specifics may give more clarity to the search behavior.

Many GP applications are used in curve fitting- finding equations hidden in the data as a category of problems. Various mathematical values ranging from values at different interesting points on the landscape, distances from each point projected to that on the curve, relative errors, etc can form good traits for such individuals to show the

In general there is a certain class of problems where you can use a general notion of decomposing the aggregated fitness function to individualized metrics as traits. In bioinformatics, GP is used in wide range of protein conformation, motif search, feature generations, etc. Most fitness functions are complex aggregated values combining many search metrics. For example, in sequence /structure classification programs many aspects of classification into one value, like true positives, false positives, true negatives, weighted distance and angles etc are combined to give a single score to the individuals. Instead of having such a single aggregated function value, we can use each of them as phenotypic

After discussing some design principles and methodology, issues related to choice of the traits

Ideally we would like to develop as complete a set of traits as possible. By "complete"

can provide interesting metrics that can be used as traits in these domains.

## **3.2. Phenotypic trait design**

Understanding the problem phenotypic landscape along with the search characteristics of the individual (GP program) will be an important step in designing the quantitative phenotypic traits. The key element is that the trait measure defines some search aspect of the individual in the phenotypic landscape. The design of phenotypic trait measures is similar to designing a fitness function for EA - they are problem-specific, and it is more an art and an iterative process to come up with one or more functions that capture the behavior. We have given some broad high level ideas below that can help the designer in more concrete way in coming up with the phenotypic traits for a given problem. Broadly speaking, we can devise the traits thus:


#### 1. Application domain specific traits:

Since most GP programs are used in agent based environments, we will generalize application domain traits to be more for agent based individuals.

• Agent Based Individuals

Agent based individuals, can be considered to have some sensors and to execute series of tasks in an environment. One may use several interesting properties as traits such as recognizing the sensors available for the agents , constraints in motion, number of tasks allowed, traps in the environment and way to avoid the traps etc can be interesting set of properties that user might want to use as traits. These properties will vary amongst the individuals and using them as phenotypic traits can give interesting multivariate analyses like the correlation between properties, correlation of these properties with fitness, etc. We can come up with more traits based on exact nature of the agents and tasks they are performing. Some of these may be orthogonal while some may have an overlap with each other. Having an overlap should be avoided as correlated traits can lead to problems likenon-positive semi-definite matrices.

• Task Oriented Individuals

In many GP applications, the agent is meant to be working on various sub-tasks. These tasks can be considered decomposable into smaller units. Normally the fitness measures only the end goal or just the higher level tasks performed, sometimes for example the amount of food eaten by the ant agent as the fitness in the ant trail problem. Various behaviors that lead to (or do not lead to) the tasks when quantified, might give good phenotypic behavior of the individuals. Some of the tasks or units can be very important and can be weighted higher as compared to others.

#### • Competitive and Co-Competitive Individuals

Many agent-based systems are competitive in nature, like the predator and prey class of the problems. Effective traits that determine metrics leading to success and failure of competing individuals may be more useful than agent-based traits. For instance, in a predator-prey based agents the fitness is basically how well you are doing against the other. If lower level details like "closeness" to the others, number of moves till attacked, number of changes in directions while moving, etc. can provide interesting metrics that can be used as traits in these domains.

### • Cooperative Individuals

6 Will-be-set-by-IN-TECH

sum of the eigenvalues of the matrix. In geometric representation it shows the sum total of all the variation in the distribution. Determinants are normally used as single measure for matrix operation. It was observed that determinants couldn't be computed for representation like GP, due to generation of individuals that can lead to non-positive semidefinite matrices.

Understanding the problem phenotypic landscape along with the search characteristics of the individual (GP program) will be an important step in designing the quantitative phenotypic traits. The key element is that the trait measure defines some search aspect of the individual in the phenotypic landscape. The design of phenotypic trait measures is similar to designing a fitness function for EA - they are problem-specific, and it is more an art and an iterative process to come up with one or more functions that capture the behavior. We have given some broad high level ideas below that can help the designer in more concrete way in coming up with the phenotypic traits for a given problem. Broadly speaking, we can devise the traits

1. At the application domain specific level to see the search behavior measured as

2. By decomposing an already aggregated fitness function into individual quantitative traits.

Since most GP programs are used in agent based environments, we will generalize

Agent based individuals, can be considered to have some sensors and to execute series of tasks in an environment. One may use several interesting properties as traits such as recognizing the sensors available for the agents , constraints in motion, number of tasks allowed, traps in the environment and way to avoid the traps etc can be interesting set of properties that user might want to use as traits. These properties will vary amongst the individuals and using them as phenotypic traits can give interesting multivariate analyses like the correlation between properties, correlation of these properties with fitness, etc. We can come up with more traits based on exact nature of the agents and tasks they are performing. Some of these may be orthogonal while some may have an overlap with each other. Having an overlap should be avoided as correlated traits can

In many GP applications, the agent is meant to be working on various sub-tasks. These tasks can be considered decomposable into smaller units. Normally the fitness measures only the end goal or just the higher level tasks performed, sometimes for example the amount of food eaten by the ant agent as the fitness in the ant trail problem. Various behaviors that lead to (or do not lead to) the tasks when quantified, might give good phenotypic behavior of the individuals. Some of the tasks or units can be very important and can be weighted higher as compared to

application domain traits to be more for agent based individuals.

lead to problems likenon-positive semi-definite matrices.

**3.2. Phenotypic trait design**

quantitative traits.

1. Application domain specific traits:

• Agent Based Individuals

• Task Oriented Individuals

others.

thus:

Another subclass of the agent based problems is the cooperative based agents. These individuals have to be in some kind of team to accomplish the goal. The individual behaviors can be specific decomposable ones or can be evolved during the execution. The performance evaluation of most fitness functions in these domains is measured by weighting individual and team performances. Various cooperative metrics can be measured again at different levels like attempts of cooperative moves, success and failures in the moves, ratio of total attempts to the success or failures, etc.

• Design based Individuals

Many GP applications are used mostly in design sub class of problems like circuit design, layout and network design and plan generation problems. Each of these use very high level measures combined in weights like the cost saved, components used, power distribution, etc. Again, using individualized measures and adding as many metrics that are circuit or layout specifics may give more clarity to the search behavior.

• Regression based Individuals

Many GP applications are used in curve fitting- finding equations hidden in the data as a category of problems. Various mathematical values ranging from values at different interesting points on the landscape, distances from each point projected to that on the curve, relative errors, etc can form good traits for such individuals to show the phenotypic search behaviors.

2. Aggregated Fitness Functions

In general there is a certain class of problems where you can use a general notion of decomposing the aggregated fitness function to individualized metrics as traits. In bioinformatics, GP is used in wide range of protein conformation, motif search, feature generations, etc. Most fitness functions are complex aggregated values combining many search metrics. For example, in sequence /structure classification programs many aspects of classification into one value, like true positives, false positives, true negatives, weighted distance and angles etc are combined to give a single score to the individuals. Instead of having such a single aggregated function value, we can use each of them as phenotypic traits.

#### *3.2.1. Issues*

After discussing some design principles and methodology, issues related to choice of the traits are discussed in this section.

• Coverage/Completeness

Ideally we would like to develop as complete a set of traits as possible. By "complete"

#### 8 Will-be-set-by-IN-TECH 10 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>9</sup>

we mean that we would like to have a set of traits that describe the whole phenotypic search space as well as possible. Another way of viewing this is to ask "Do the traits that we have uniquely define an individual?" As we mentioned earlier, previous applications of quantitative genetics to EA have used fitness alone, this provided a very limited and incomplete view of the nature of the fitness landscape, especially individuals that are very different can have the same fitness. Similarly an incomplete set of traits can fail to illuminate certain important aspects of a problem.

At the other end of the scale, a low-level phenotype (conceptually, at least) might be something as simple as the input-output map that exactly describes the actions the agent would take for any given set of inputs. Here we have a much better chance of relating such information to the representational structure, and the effects of the reproductive operators. Unfortunately, it becomes much more difficult to define a complete set of traits. Such a set would have to describe the entire map, and this might mean thousands of traits. The only viable option is to create sample sets of inputs, where each sample would define a single trait. If one can define enough traits to get a reasonable sampling of the input space, or identify important samples that yield particularly valuable information, then this

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 11

Exactly how to solve this trade-off remains an open issue. Some possible solutions include combining low-level and high-level traits, using different kinds of traits depending on ones goals, or trying to find traits that achieve a middle ground between these two extremes.

To correlate some important evolutionary behaviors we need to measure genotypic diversity changes in the populations. There are many ways to measure genotypic diversity measurements like tree-edit distances, genetic lineages, entropy etc for understanding the genotypic behavior and correlating it with phenotypic behaviors [7]. Genetic Lineage is the metric more commonly used as it shows significant correlation to fitness [8]. In context of GP, with individuals as trees, when an operator like crossover breeds and produces an offspring, the offspring that has the root node of parent has the lineage of that parent. This provides a way to measure distribution of lineage over generations and also the count of unique lineages

Another important factor that we use to correlate the evolutionary behavior changes is with bloat. Bloat, has been described in various researches but very few of them have defined it quantitatively. In our study since we have to measure bloat quantitatively we use the metrics

*bloat*(*g*) = (*δ*(*g*) <sup>−</sup> *<sup>δ</sup>*(0))/*δ*(0)

where *δ*(*g*) is the average number of nodes per individual in the population at generation *g*,

In next subsections we will walk through three different GP problems, to discuss the methodology of defining traits, performing experiments with different evolutionary operators and understanding the evolutionary behaviors in context of the given problem. We start with the ant trail problem and perform various experiments by changing the operators, selection mechanisms and pressure to investigate the evolutionary behavior with respect to quantitative genetics metrics. We then move to another agent oriented problem, lawn mower problem showing few experiments involving breeding operators and different selection mechanisms.

and *f*(*g*) is the average fitness of individuals in the population at generation *g*.

*<sup>f</sup>*(0) <sup>−</sup> *<sup>f</sup>*(*g*))/ *<sup>f</sup>*(0) (4)

approach could still be useful.

**3.3. Genetic diversity using lineage**

in the population over generations.

as defined in the recent research [32].

**4. GP benchmark problems and analyses**

**3.4. Bloat measure**

Domains involving executable objects (like GP), and most machine learning in general, are particularly susceptible to this problem. This is because generalization is a critical part of the learning process. We expect our systems to be able to handle new situations that they never faced during training. One way of addressing this issue is to create traits that are, in a sense, general too. Traits that measure a set of behaviors that all fall into a broad category will be able to achieve the best coverage of the search space.

It is difficult to offer advice as to how one can recognize when they face this situation. Asking the question about uniqueness seems to offer the best general approach. It may be wise to ask oneself this question throughout the design and implementation process. One advantage that quantitative genetics offer though, is that it degrades gracefully in the sense that all the equations are still completely accurate, even with an incomplete set of traits. Ultimately one may only need a subset of traits in order to observe the important behaviors their algorithms, just so long as they are the right subset.

• Unnecessary traits

Unnecessary traits are either those that are always constant, or those that are so closely correlated with another trait that they essentially are measuring the same thing. These can be more problematic because they can result in matrices that are non-positive definite. In this particular case it would mean that the matrices have one or more eigenvalues that are zero. While this is not actually wrong, with just a small amount of measurement error, or round-off error, the matrices could have negative eigenvalues, which is more problematic. We have devised the metric equation (equation 3) to minimize computational problems related to this situation, but one should try to avoid it if possible.

• Phenotype to Genotype Linking

If one's goal in using these tools is to identify and fix problem in an algorithm, then one will need to make a connection between the traits, and any aspects of the representation or reproductive operators that are affecting those traits. The more abstract the traits are, the more difficult this becomes, and so very low-level descriptions of behaviors may be more appropriate to achieve this.

Unfortunately, this can creates a conflict with the issue of trait completeness described above. There we suggested that higher-level traits may be better for getting the best landscape description possible. For example, consider a problem where we are trying to teach an agent to track another agent without being detected. A high-level set of traits might measure thing like: how much distance an agent keeps between itself and the target, the length of time that it is able to maintain surveillance, and the number of times it is detected. These traits may be ideal for covering all the skills that may be necessary for describing the fitness landscape, but they may not be very helpful in identifying what aspect of a representation or reproductive operators are problematic for learn well in this domain. Such connections would be tenuous at best.

At the other end of the scale, a low-level phenotype (conceptually, at least) might be something as simple as the input-output map that exactly describes the actions the agent would take for any given set of inputs. Here we have a much better chance of relating such information to the representational structure, and the effects of the reproductive operators. Unfortunately, it becomes much more difficult to define a complete set of traits. Such a set would have to describe the entire map, and this might mean thousands of traits. The only viable option is to create sample sets of inputs, where each sample would define a single trait. If one can define enough traits to get a reasonable sampling of the input space, or identify important samples that yield particularly valuable information, then this approach could still be useful.

Exactly how to solve this trade-off remains an open issue. Some possible solutions include combining low-level and high-level traits, using different kinds of traits depending on ones goals, or trying to find traits that achieve a middle ground between these two extremes.

#### **3.3. Genetic diversity using lineage**

To correlate some important evolutionary behaviors we need to measure genotypic diversity changes in the populations. There are many ways to measure genotypic diversity measurements like tree-edit distances, genetic lineages, entropy etc for understanding the genotypic behavior and correlating it with phenotypic behaviors [7]. Genetic Lineage is the metric more commonly used as it shows significant correlation to fitness [8]. In context of GP, with individuals as trees, when an operator like crossover breeds and produces an offspring, the offspring that has the root node of parent has the lineage of that parent. This provides a way to measure distribution of lineage over generations and also the count of unique lineages in the population over generations.

#### **3.4. Bloat measure**

8 Will-be-set-by-IN-TECH

illuminate certain important aspects of a problem.

• Unnecessary traits

• Phenotype to Genotype Linking

appropriate to achieve this.

will be able to achieve the best coverage of the search space.

behaviors their algorithms, just so long as they are the right subset.

related to this situation, but one should try to avoid it if possible.

domain. Such connections would be tenuous at best.

we mean that we would like to have a set of traits that describe the whole phenotypic search space as well as possible. Another way of viewing this is to ask "Do the traits that we have uniquely define an individual?" As we mentioned earlier, previous applications of quantitative genetics to EA have used fitness alone, this provided a very limited and incomplete view of the nature of the fitness landscape, especially individuals that are very different can have the same fitness. Similarly an incomplete set of traits can fail to

Domains involving executable objects (like GP), and most machine learning in general, are particularly susceptible to this problem. This is because generalization is a critical part of the learning process. We expect our systems to be able to handle new situations that they never faced during training. One way of addressing this issue is to create traits that are, in a sense, general too. Traits that measure a set of behaviors that all fall into a broad category

It is difficult to offer advice as to how one can recognize when they face this situation. Asking the question about uniqueness seems to offer the best general approach. It may be wise to ask oneself this question throughout the design and implementation process. One advantage that quantitative genetics offer though, is that it degrades gracefully in the sense that all the equations are still completely accurate, even with an incomplete set of traits. Ultimately one may only need a subset of traits in order to observe the important

Unnecessary traits are either those that are always constant, or those that are so closely correlated with another trait that they essentially are measuring the same thing. These can be more problematic because they can result in matrices that are non-positive definite. In this particular case it would mean that the matrices have one or more eigenvalues that are zero. While this is not actually wrong, with just a small amount of measurement error, or round-off error, the matrices could have negative eigenvalues, which is more problematic. We have devised the metric equation (equation 3) to minimize computational problems

If one's goal in using these tools is to identify and fix problem in an algorithm, then one will need to make a connection between the traits, and any aspects of the representation or reproductive operators that are affecting those traits. The more abstract the traits are, the more difficult this becomes, and so very low-level descriptions of behaviors may be more

Unfortunately, this can creates a conflict with the issue of trait completeness described above. There we suggested that higher-level traits may be better for getting the best landscape description possible. For example, consider a problem where we are trying to teach an agent to track another agent without being detected. A high-level set of traits might measure thing like: how much distance an agent keeps between itself and the target, the length of time that it is able to maintain surveillance, and the number of times it is detected. These traits may be ideal for covering all the skills that may be necessary for describing the fitness landscape, but they may not be very helpful in identifying what aspect of a representation or reproductive operators are problematic for learn well in this Another important factor that we use to correlate the evolutionary behavior changes is with bloat. Bloat, has been described in various researches but very few of them have defined it quantitatively. In our study since we have to measure bloat quantitatively we use the metrics as defined in the recent research [32].

$$
abla t(g) = \frac{(\overline{\delta}(g) - \overline{\delta}(0)) / \overline{\delta}(0)}{\overline{f}(0) - \overline{f}(g)) / \overline{f}(0)}\tag{4}$$

where *δ*(*g*) is the average number of nodes per individual in the population at generation *g*, and *f*(*g*) is the average fitness of individuals in the population at generation *g*.

### **4. GP benchmark problems and analyses**

In next subsections we will walk through three different GP problems, to discuss the methodology of defining traits, performing experiments with different evolutionary operators and understanding the evolutionary behaviors in context of the given problem. We start with the ant trail problem and perform various experiments by changing the operators, selection mechanisms and pressure to investigate the evolutionary behavior with respect to quantitative genetics metrics. We then move to another agent oriented problem, lawn mower problem showing few experiments involving breeding operators and different selection mechanisms.

10 Will-be-set-by-IN-TECH 12 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>11</sup>

Finally we use the symbolic regression problem to describe how traits can be defined and the observations showing generality of our methodology.

• **Sum of Distances from Last Point**:This is the manhattan distance computed for all the moves from where it is to point last point. This trait measures the "geometric

• **Count of Null Movements**:This is the count of zero movements, i.e. no change in displacement for the agent over all its moves. This trait measures the effect of changing

Most of these quantitative traits show exponential distribution and hence they are transformed to the new set of derived traits by taking the *log* of the originals as insisted

To understand the effects of the operator and selection, we will be performing one operator

Since most GP problems use subtree crossover as the main breeding operator and normally higher selection pressure with tournament size 7 are employed, we use these to plot different metrics explained in the quantitative genetics section as shown in

We change the tournament selection to have lower pressure by changing the

Fitness Proportionate Selection generally has lower selection pressure as compared to tournament selection, and by changing the selection mechanism the metrics are shown

Homologous Crossover was designed and successfully employed to control bloat and improve fitness in many GP problems [15]. The impact of using homologous crossover

• Tournament size 2 gives a weaker selection pressure than tournament size 7. It can be seen that with selection 7 as compared to selection 2, there is rapid convergence in genotypic diversity. This correlates to rapid convergence in the phenotypic trait measurements of O and P. It can be observed that when the genotypic diversity and corresponding phenotypic traits converge, there is rise in the perturbation tr(**D**)/tr(**P**) curve. The point at which this happens and magnitude of change shifts in generations with selection pressure, i.e with tournament selection size 2 it happens later around generation 50 as compared to around generation 20 with selection 7. Also the increase is

tournament size to 2, and observing all the metrics are shown in Figure 3.

on tournament selection size 2 using the metrics is shown in Figure 5.

at a time with the selection mechanism mentioned to see the impact.

• **Subtree Crossover and Tournament Selection size 7**

• **Subtree Crossover and tournament Selection size 2**

• **Subtree Crossover and Fitness Proportionate Selection**

• **Homologous Crossover and Tournament Selection size 7**

�*δi*,*<sup>d</sup>* − *<sup>δ</sup>i*−1,*d*� (7)

∀*d*, {*i f*(*δi*,*<sup>d</sup>* − *<sup>δ</sup>i*−1,*d*) = 0, *count* = *count* + <sup>1</sup>} (8)

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 13

*m* ∑ *i*=1

*d* ∑ *j*=1

displacement effect" irrespective of trail for the agent.

code not altering the behavior of the agent. *m* ∑ *i*=1

by various biologist [9]. 2. **Santa-Fe Ant trail GP Experiments**

Figure 2.

in the Figure 4.

3. **Santa-Fe Ant trail Observations**

All the experiments are performed using ECJ [17] with various standard default parameters like population size of 1024, a crossover depth limit of 17,and the ramped half and half method of generating tree (min/max of 2 and 6) for creating individuals. We will plot average tr(**D**)/tr(**P**), tr(**G**)/tr(**P**) and tr(**O**)/tr(**P**) as quantitative genetics metrics for each generations. We will also plot the average unique ancestors as our genetic lineage diversity measure and bloat metrics from above for some correlations.

## **4.1. Experiment 1: Santa-Fe Ant trail**

Artificial Ant is representative of an agent search problem and also it is considered to be highly deceptive and difficult for genetic programming [16]. The Santa-fe ant problem has a difficult trail and the objective is to devise a program which can successfully navigate an artificial ant to find all pieces of food located on a grid. The total amount of food consumed is used as single point measure of the fitness of the program. The program has three terminal operations forward, left and right for navigation. It has three basic actions like IfFoodAhead, progn2 and progn3 for performing single action and parameter based execution in the sequence. It has three basic actions like IfFoodAhead, progn2 and progn3 for performing single action and parameter based execution in the sequence. IfFoodAhead is a non-terminal function that takes two parameters and executes the first if there is food one step ahead and the second otherwise. Progn2 takes 2 parameters while progn3 takes 3 parameters and executes them in a sequence.

#### 1. **Quantitative Traits for Santa-Fe Ant trail**

As per our discussions in the phenotypic traits section, various search properties are devised to measure quantitatively behavior of an agent like ant and used for phenotypic traits in the calculations for equation above.

For all the formulas

*m= moves, d= dimension, trail= point on trail,closest-trail= closest point on trail δ* = *distance*

• **Sum of Distances from Last Trail**: This is the manhattan distance computed for all the moves from where it is to where it was last on the trail. This trait measures the "moving away effect" of the agent to the trail.

$$\sum\_{i=1}^{m} \sum\_{j=1}^{d} ||\delta\_{i,d} - \delta\_{trail,d}||\tag{5}$$

• **Sum of Distances to Closest Point on Trail**:This is the manhattan distance computed for all the moves from where it is to point closest on the trail. This trait measures the "closeness" of the agent to the trail.

$$\sum\_{i=1}^{m} \sum\_{j=1}^{d} ||\delta\_{i,d} - \delta\_{\text{closest}-trail,d}|| \tag{6}$$

• **Sum of Distances from Last Point**:This is the manhattan distance computed for all the moves from where it is to point last point. This trait measures the "geometric displacement effect" irrespective of trail for the agent.

$$\sum\_{i=1}^{m} \sum\_{j=1}^{d} ||\delta\_{i,d} - \delta\_{i-1,d}||\tag{7}$$

• **Count of Null Movements**:This is the count of zero movements, i.e. no change in displacement for the agent over all its moves. This trait measures the effect of changing code not altering the behavior of the agent.

$$\sum\_{i=1}^{m} \forall d, \left\{ if(\delta\_{i,d} - \delta\_{i-1,d}) = 0, count = count + 1 \right\} \tag{8}$$

Most of these quantitative traits show exponential distribution and hence they are transformed to the new set of derived traits by taking the *log* of the originals as insisted by various biologist [9].

#### 2. **Santa-Fe Ant trail GP Experiments**

10 Will-be-set-by-IN-TECH

Finally we use the symbolic regression problem to describe how traits can be defined and the

All the experiments are performed using ECJ [17] with various standard default parameters like population size of 1024, a crossover depth limit of 17,and the ramped half and half method of generating tree (min/max of 2 and 6) for creating individuals. We will plot average tr(**D**)/tr(**P**), tr(**G**)/tr(**P**) and tr(**O**)/tr(**P**) as quantitative genetics metrics for each generations. We will also plot the average unique ancestors as our genetic lineage diversity

Artificial Ant is representative of an agent search problem and also it is considered to be highly deceptive and difficult for genetic programming [16]. The Santa-fe ant problem has a difficult trail and the objective is to devise a program which can successfully navigate an artificial ant to find all pieces of food located on a grid. The total amount of food consumed is used as single point measure of the fitness of the program. The program has three terminal operations forward, left and right for navigation. It has three basic actions like IfFoodAhead, progn2 and progn3 for performing single action and parameter based execution in the sequence. It has three basic actions like IfFoodAhead, progn2 and progn3 for performing single action and parameter based execution in the sequence. IfFoodAhead is a non-terminal function that takes two parameters and executes the first if there is food one step ahead and the second otherwise. Progn2 takes 2 parameters while progn3 takes 3 parameters and executes them in a sequence.

As per our discussions in the phenotypic traits section, various search properties are devised to measure quantitatively behavior of an agent like ant and used for phenotypic

• **Sum of Distances from Last Trail**: This is the manhattan distance computed for all the moves from where it is to where it was last on the trail. This trait measures the "moving

• **Sum of Distances to Closest Point on Trail**:This is the manhattan distance computed for all the moves from where it is to point closest on the trail. This trait measures the

�*δi*,*<sup>d</sup>* − *δtrail*,*d*� (5)

�*δi*,*<sup>d</sup>* − *<sup>δ</sup>closest*−*trail*,*d*� (6)

*m= moves, d= dimension, trail= point on trail,closest-trail= closest point on trail*

*m* ∑ *i*=1

*m* ∑ *i*=1

*d* ∑ *j*=1

*d* ∑ *j*=1

observations showing generality of our methodology.

measure and bloat metrics from above for some correlations.

**4.1. Experiment 1: Santa-Fe Ant trail**

1. **Quantitative Traits for Santa-Fe Ant trail**

For all the formulas

*δ* = *distance*

traits in the calculations for equation above.

away effect" of the agent to the trail.

"closeness" of the agent to the trail.

To understand the effects of the operator and selection, we will be performing one operator at a time with the selection mechanism mentioned to see the impact.

#### • **Subtree Crossover and Tournament Selection size 7**

Since most GP problems use subtree crossover as the main breeding operator and normally higher selection pressure with tournament size 7 are employed, we use these to plot different metrics explained in the quantitative genetics section as shown in Figure 2.

#### • **Subtree Crossover and tournament Selection size 2**

We change the tournament selection to have lower pressure by changing the tournament size to 2, and observing all the metrics are shown in Figure 3.

#### • **Subtree Crossover and Fitness Proportionate Selection**

Fitness Proportionate Selection generally has lower selection pressure as compared to tournament selection, and by changing the selection mechanism the metrics are shown in the Figure 4.

#### • **Homologous Crossover and Tournament Selection size 7**

Homologous Crossover was designed and successfully employed to control bloat and improve fitness in many GP problems [15]. The impact of using homologous crossover on tournament selection size 2 using the metrics is shown in Figure 5.

#### 3. **Santa-Fe Ant trail Observations**

• Tournament size 2 gives a weaker selection pressure than tournament size 7. It can be seen that with selection 7 as compared to selection 2, there is rapid convergence in genotypic diversity. This correlates to rapid convergence in the phenotypic trait measurements of O and P. It can be observed that when the genotypic diversity and corresponding phenotypic traits converge, there is rise in the perturbation tr(**D**)/tr(**P**) curve. The point at which this happens and magnitude of change shifts in generations with selection pressure, i.e with tournament selection size 2 it happens later around generation 50 as compared to around generation 20 with selection 7. Also the increase is

0 20 40 60 80 100

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P)**

Subtree Crossover and Tournament 2

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 15

Subtree Crossover and Tournament 2

0.0 0.2 0.4 0.6 0.8 1.0

0

0

 5

 10

 15

 20

 2

 4

 6

Trace

 8 10

Trace

Generations

**Trace(O)/Trace(P)**

Subtree Crossover and Tournament 2

0 20 40 60 80 100

Generations

**Genetic Lineage Plot**

Subtree Crossover wit

0 20 40 60 80 100

Generations

**Figure 3.** Ant, Subtree crossover, tournament size 2, depth limit 17

**Trace(O) and Trace(P)**

(tr(P)) (tr(O))

0

0

Average Ancestors

 2

 4

 6

Trace

 8 10

 2

 4

 6

Trace

**Figure 2.** Ant, Subtree crossover, tournament size 7, depth limit 17

**Figure 3.** Ant, Subtree crossover, tournament size 2, depth limit 17

12 Will-be-set-by-IN-TECH

0.0 0.2 0.4 0.6 0.8 1.0

0

 5

 10

 15

 20

Trace

Trace

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P)**

Subtree Crossover and Tournament 7

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Subtree Crossover and Tournament 7

0 20 40 60 80 100

Generations

**Trace(O)/Trace(P)**

Subtree Crossover and Tournament 7

0 20 40 60 80 100

Generations

**Genetic Lineage Plot**

Subtree Crossover with Tournament 7

0 20 40 60 80 100

Generations

**Figure 2.** Ant, Subtree crossover, tournament size 7, depth limit 17

**Trace(O) and Trace(P)**

(tr(P)) (tr(O))

0

Average Ancestors

Trace

 2

 4

 6

Trace

0 20 40 60 80 100

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P): TournamentSelection7**

Homologous and Tournament 7

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 17

Homologous and Tournament 7

0.0 0.2 0.4 0.6 0.8 1.0

0

 5

 10

 15

 20

Trace

Trace

Generations

**Trace(O)/Trace(P)**

Homologous and Tournament 7

0 20 40 60 80 100

Generations

**Genetic Lineage Plot**

Homologous with Tournament 7

0 20 40 60 80 100

Generations

**Figure 5.** Ant, Homologous crossover, tournament size 7, depth limit 17

**Trace(O) and Trace(P)**

(tr(P)) (tr(O))

0

Average Ancestors

Trace

 2

 4

 6

Trace

**Figure 4.** Ant, Subtree Crossover, Fitness Proportionate Selection (FPS), depth limit 17

**Figure 5.** Ant, Homologous crossover, tournament size 7, depth limit 17

14 Will-be-set-by-IN-TECH

0.0 0.2 0.4 0.6 0.8 1.0

012345

0

 50

 100

 150

 200

Trace

Trace

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P)**

Subtree Crossover and FPS

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Subtree Crossover and FPS

0 20 40 60 80 100

Generations

**Trace(O)/Trace(P)**

Subtree Crossover and FPS

0 20 40 60 80 100

Generations

**Genetic Lineage Plot**

Subtree Crossover wit

**Figure 4.** Ant, Subtree Crossover, Fitness Proportionate Selection (FPS), depth limit 17

0 20 40 60 80 100

Generations

**Trace(O) and Trace(P)**

(tr(P)) (tr(O))

0

012345

Average Ancestors

Trace

 2

 4

 6

Trace

16 Will-be-set-by-IN-TECH 18 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>17</sup>

**4.2. Experiment 2: Lawn mower**

moving forward and jumping to specified squares.

execution, which we will refer as m.

*m* ∑ *i*=1

> *m* ∑ *i*=1

> > *m* ∑ *i*=1

differences and similarity in the evolutionary behaviors.

*m= moves, d= dimension, δ* = *distance and* Ω = *orientation*

changing code not altering the behavior of the agent.

measures the "geometric displacement effect" in the movement.

*m* ∑ *i*=1

*d* ∑ *j*=1

1. **Quantitative Traits for Lawn Mower**

the movements.

For all the formulas below

lawn mower is changed.

2. **Lawn Mower GP Experiments**

The essence of this problem is to find a solution for controlling the movement of a robotic lawn mower so that the lawn mower visits each of the squares on two-dimensional n x m toroidal grid. The toroidal nature of the grid allows the lawnmower to wrap around to the opposite side of the grid when it moves off one of the edges. The lawnmower has state consisting of the squares on which the lawnmower is currently residing and the direction (up,down,left and right) which is facing. The lawnmower has 3 actions that change its state: turning left,

Similar to ant problem, we came up with some quantitative traits to measure the lawn mower behavior in the phenotypic landscape using the design principles. We keep a memory of visited location and have a function *visited(d)* for validating the revisit. We also keep memory of last orientation using omega in for measuring change in orientations in

• **Number of Moves**:This measures total number of moves performed by the agent in the

• **Count of Null Movements**:This is the count of zero movements, i.e. no change in displacement for the lawn mower over all its moves. This trait measures the effect of

• **Sum of Distances**:This is the manhattan distance computed for all the moves. This trait

• **Number of Orientation changes**:This measures number of times the orientation of the

• **Count of Revisits**:This measures number of times the already visited spot is visited.

We performed subset of experiments from our ant problem on the lawn mower to see

∀*d*, {*i f*(*δi*,*<sup>d</sup>* − *<sup>δ</sup>i*−1,*d*) = 0, *count* = *count* + <sup>1</sup>} (9)

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 19

∀*d*, {*i f*(Ω*i*,*<sup>d</sup>* �= <sup>Ω</sup>*i*−1,*d*), *count* = *count* + <sup>1</sup>} (11)

∀*d*, {*i f*(*visited*(*d*)), *count* = *count* + 1} (12)

�*δi*,*<sup>d</sup>* − *<sup>δ</sup>i*−1,*d*� (10)

**Figure 6.** Average and BSF fitness for ant experiments

magnitude lesser (scale of DP, OP with selection 2 as compared to selection 7). Increased selection pressure which may result in lack of diversity may increase perturbation in the system. This increase may be useful in some difficult problems for finding area in the landscape that is not reachable otherwise and may not be effective when more of greedy local search is necessary to reach optimum. In ant trail problem, being a difficult landscape, increased perturbation is helpful to find solution faster as shown in the fitness curves in the Figure 6.


#### **4.2. Experiment 2: Lawn mower**

16 Will-be-set-by-IN-TECH

0

magnitude lesser (scale of DP, OP with selection 2 as compared to selection 7). Increased selection pressure which may result in lack of diversity may increase perturbation in the system. This increase may be useful in some difficult problems for finding area in the landscape that is not reachable otherwise and may not be effective when more of greedy local search is necessary to reach optimum. In ant trail problem, being a difficult landscape, increased perturbation is helpful to find solution faster as shown in

• It can be observed that the increase in perturbation with selection size 7, eventually tapers down and may be attributed to rise in the bloat. As bloat increases beyond a threshold, the effect of changes is reduced and that brings the perturbation down. • Another important thing to note is with higher selection pressure, when there is premature convergence, it results in statistically significant (95% confidence) difference between the phenotypic behavior of offsprings and parents, while lower selection

• FPS results in higher genotypic diversity amongst the individuals as observed in the Figure 4, and that results in lower convergence in the population phenotypically and

• Figure 5 shows that the perturbation increases with reduction in diversity exactly like in subtree crossover, but the perturbation continues to stay higher because of bloat control, however the max-value of perturbation is still lower than in normal crossover. Thus bloat which helped subtree crossover to reduce the impact of perturbation, when controlled by homologous crossover, showed constant value. This is consistent with

• Figure 6 show the comparative plots of average and best so far (bsf) with 95% confidence intervals as whiskers. It can be seen that tournament selection with 7 with subtree or homologous are similar. Homologous crossover with reduced perturbations

as a result the perturbation effect is constant across all the generations.

theory that the bloat is a defensive mechanism against crossover [1].

and bloat has real advantage over subtree crossover in this experiment.

 10 20

 30

Fitness

 40

 50

0 20 40 60 80 100

Generations

**Average Best−so−far**

Homo Cross w/ Tourn 7 Subtree w/ Tourn 2 Subtree w/ FPS Subtree w/ Tourn 7

0 20 40 60 80 100

Generations

**Figure 6.** Average and BSF fitness for ant experiments

the fitness curves in the Figure 6.

pressure reduces the difference.

**Average Fitness**

Subtree w/ FPS Subtree w/ Tourn 2 Homo Cross w/ Tourn 7 Subtree w/ Tourn 7

0

 20

 40

Fitness

 60

 80 The essence of this problem is to find a solution for controlling the movement of a robotic lawn mower so that the lawn mower visits each of the squares on two-dimensional n x m toroidal grid. The toroidal nature of the grid allows the lawnmower to wrap around to the opposite side of the grid when it moves off one of the edges. The lawnmower has state consisting of the squares on which the lawnmower is currently residing and the direction (up,down,left and right) which is facing. The lawnmower has 3 actions that change its state: turning left, moving forward and jumping to specified squares.

#### 1. **Quantitative Traits for Lawn Mower**

Similar to ant problem, we came up with some quantitative traits to measure the lawn mower behavior in the phenotypic landscape using the design principles. We keep a memory of visited location and have a function *visited(d)* for validating the revisit. We also keep memory of last orientation using omega in for measuring change in orientations in the movements.

For all the formulas below *m= moves, d= dimension, δ* = *distance and* Ω = *orientation*


$$\sum\_{i=1}^{m} \forall d, \left\{ if(\delta\_{i,d} - \delta\_{i-1,d}) = 0, count = count + 1 \right\} \tag{9}$$

• **Sum of Distances**:This is the manhattan distance computed for all the moves. This trait measures the "geometric displacement effect" in the movement.

$$\sum\_{i=1}^{m} \sum\_{j=1}^{d} ||\delta\_{i,d} - \delta\_{i-1,d}||\tag{10}$$

• **Number of Orientation changes**:This measures number of times the orientation of the lawn mower is changed.

$$\sum\_{i=1}^{m} \forall d, \{if(\Omega\_{i,d} \neq \Omega\_{i-1,d}), count = count + 1\} \tag{11}$$

• **Count of Revisits**:This measures number of times the already visited spot is visited.

$$\sum\_{i=1}^{m} \forall d, \{if(visited(d)), count = count + 1\} \tag{12}$$

#### 2. **Lawn Mower GP Experiments**

We performed subset of experiments from our ant problem on the lawn mower to see differences and similarity in the evolutionary behaviors.

18 Will-be-set-by-IN-TECH 20 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>19</sup>

0 20 40 60 80 100

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P): FPS**

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 21

Subtree Crossover and FPS

the difference between these two amounts will be higher relative to the amount of variation in the selection parent populations. As a result, perturbation and population heritability will appear higher, but this is only because they had further to go to get

• Homologous crossover shows fairly stable tr(**D**)/tr(**P**), tr(**O**)/tr(**P**) and tr(**G**)/tr(**P**) curves as shown in Figure 9, where the operator on this problem acts similar to the GA based crossover on a simple problem like sphere [3]. As the population converges in phenotype space, crossover is able to adapt and create offspring populations with similar distributions to those of the parent population (as can be seen by the fact that tr(**G**)/tr(**P**) stays close to 0.5, and even more importantly that O/P stays relatively close to 1). The fact that it is able to do this even at the end of the run is important. It allows the population to truly converge on a very small part of the search space until there is (almost) no variation left. This is often considered to be a weakness of crossover, but in some ways it is really a strength. Without this ability, the algorithm cannot fully

• Figure 10 shows again at the end of the generations there is no significant difference between subtree crossover and homologous crossover, while homologous crossover

with better perturbation and heritability may be at advantage.

Trace

Generations

**Trace(G)/Trace(P)**

Subtree Crossover and FPS

0 20 40 60 80 100

Generations

**Figure 8.** Lawn Mower Subtree crossover, Fitness Proportionate Selection, depth limit 17

back to the same place (i.e. the lower bound defined by the operators).

**Trace(O)/Trace(P)**

Subtree Crossover and FPS

0.0 0.2 0.4 0.6 0.8 1.0

exploit the information it gains.

Trace

Trace

**Figure 7.** Lawn Mower Subtree crossover, tournament size 2, depth limit 17


#### 3. **Lawn Mower Observations**

• An interesting observation about the perturbation tr(**D**)/tr(**P**) and tr(**O**)/tr(**P**) curves can be made from Figures 8 and 9. Both curves tend to increase to a higher level with binary tournament selection as compared to FPS. This is actually a result of the fact that the GP crossover operators have a lower bound on the amount of variation they add to the population [4]. Higher selection pressures will reduce the phenotypic variation in the population more that lower selection pressures. Reproductive operators then return the variation to the operators minimum levels. When selection pressures are higher, 20 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>19</sup> Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 21

18 Will-be-set-by-IN-TECH

0

0

We perform comparative subtree crossover with lower selection pressure on our lawn mower problem and show the quantitative genetics metrics plotted in the Figure 7.

We change the selection pressure totally by going for FPS instead of tournament

Impact of bloat control by using homologous crossover with tournament selection with

• An interesting observation about the perturbation tr(**D**)/tr(**P**) and tr(**O**)/tr(**P**) curves can be made from Figures 8 and 9. Both curves tend to increase to a higher level with binary tournament selection as compared to FPS. This is actually a result of the fact that the GP crossover operators have a lower bound on the amount of variation they add to the population [4]. Higher selection pressures will reduce the phenotypic variation in the population more that lower selection pressures. Reproductive operators then return the variation to the operators minimum levels. When selection pressures are higher,

 20

 40

 60

 80

Trace

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P): TournamentSelection−2**

Subtree Crossover and Tournament 7

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Subtree Crossover and Tournament 7

0 20 40 60 80 100

Generations

• **Subtree Crossover and Tournament Selection size 2**

• **Subtree Crossover and Fitness Proportionate Selection**

size 2 with various metrics are shown in the Figure 9.

selection and plot various metrics in the Figure 8. • **Homologous Crossover and Tournament Selection size 2**

**Figure 7.** Lawn Mower Subtree crossover, tournament size 2, depth limit 17

**Trace(O)/Trace(P)**

Subtree Crossover and Tournament 7

0

0.0 0.2 0.4 0.6 0.8 1.0

3. **Lawn Mower Observations**

Trace

Trace

**Figure 8.** Lawn Mower Subtree crossover, Fitness Proportionate Selection, depth limit 17

the difference between these two amounts will be higher relative to the amount of variation in the selection parent populations. As a result, perturbation and population heritability will appear higher, but this is only because they had further to go to get back to the same place (i.e. the lower bound defined by the operators).


**4.3. Experiment 3: Symbolic regression**

Symbolic Regression problem is about finding the equation closest to the given problem, by generating different curves and testing on the sample training points. The absolute error over the training points is used as the fitness function. Terminal would be the variable X and the non-terminals would be mathematical functions like log, sine, cosine, addition, multiplication

1. **Quantitative Traits for Symbolic Regression** Regression being a mathematical problem in an euclidean space rather than a behavior based agent, we used the values of 10 random points equally distributed on the curve as the trait measurements like [-0.9, -0.7,-0.5...0.5,0.7,0.9]. This is similar to fitness being evaluated over fixed training point, but the difference being here we get individual values rather than aggregated measure. These individual trait values can be important in identifying how the curve changes

0

 5

 10

 15

 20

Trace

*<sup>y</sup>* <sup>=</sup> *<sup>x</sup>*<sup>5</sup> <sup>−</sup> <sup>2</sup>*x*<sup>3</sup> <sup>+</sup> *<sup>x</sup>*, *<sup>x</sup>* =[-1,1] (13)

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 23

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P): TournamentSelection−2**

Homologous and Tournament 2

etc. We used the quintic function for our test. Quintic is given by equation

between parent and offspring during the evolutionary process.

**Trace(O)/Trace(P)**

Homologous and Tournament 2

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Homologous and Tournament 2

0 20 40 60 80 100

Generations

**Figure 11.** Symbolic Regression, Homologous crossover, tournament size 2, depth limit 17

0.0 0.2 0.4 0.6 0.8 1.0

Trace

Trace

**Figure 9.** Lawn Mower Homologous crossover, tournament size 2, depth limit 17

**Figure 10.** Average and BSF fitness for lawn mower experiments

#### **4.3. Experiment 3: Symbolic regression**

20 Will-be-set-by-IN-TECH

Fitness

Trace

0 20 40 60 80 100

Generations

Parent bloat metric Offspring bloat metric

0 20 40 60 80 100

Generations

**Average Best−so−far**

Subtree w/ Tourn 2 Subtree w/ FPS Homo Cross w/ Tourn 7

0 20 40 60 80 100

Generations

**Trace(D)/Trace(P): TournamentSelection−2**

Homologous and Tournament 2

0 20 40 60 80 100

Generations

**Trace(G)/Trace(P)**

Homologous and Tournament 2

0 20 40 60 80 100

Generations

**Average Fitness**

0 20 40 60 80 100

Generations

**Figure 10.** Average and BSF fitness for lawn mower experiments

**Figure 9.** Lawn Mower Homologous crossover, tournament size 2, depth limit 17

Subtree w/ FPS Subtree w/ Tourn 2 Homo Cross w/ Tourn 7

**Trace(O)/Trace(P)**

Homologous and Tournament 2

0.0 0.2 0.4 0.6 0.8 1.0

Fitness

Trace

Trace

Symbolic Regression problem is about finding the equation closest to the given problem, by generating different curves and testing on the sample training points. The absolute error over the training points is used as the fitness function. Terminal would be the variable X and the non-terminals would be mathematical functions like log, sine, cosine, addition, multiplication etc. We used the quintic function for our test. Quintic is given by equation

$$y = \mathbf{x}^5 - 2\mathbf{x}^3 + \mathbf{x}, \quad \mathbf{x} = \text{[-1,1]} \tag{13}$$

1. **Quantitative Traits for Symbolic Regression** Regression being a mathematical problem in an euclidean space rather than a behavior based agent, we used the values of 10 random points equally distributed on the curve as the trait measurements like [-0.9, -0.7,-0.5...0.5,0.7,0.9]. This is similar to fitness being evaluated over fixed training point, but the difference being here we get individual values rather than aggregated measure. These individual trait values can be important in identifying how the curve changes between parent and offspring during the evolutionary process.

**Figure 11.** Symbolic Regression, Homologous crossover, tournament size 2, depth limit 17

#### 22 Will-be-set-by-IN-TECH 24 Genetic Programming – New Approaches and Successful Applications Using Quantitative Genetics and Phenotypic Traits in Genetic Programming <sup>23</sup>

#### 2. **Regression GP Experiment**

We will analyze one experiment using Homologous crossover and tournament selection to see generic behavior of GP problems given similar operators and selection pressure.

[5] Bassett, J. K., Potter, M. A. & De Jong, K. A. [2004]. Looking under the EA hood with Price's equation, *in* K. Deb, R. Poli, W. Banzhaf, H.-G. Beyer, E. Burke, P. Darwen, D. Dasgupta, D. Floreano, J. Foster, M. Harman, O. Holland, P. L. Lanzi, L. Spector, A. Tettamanzi, D. Thierens & A. Tyrrell (eds), *Genetic and Evolutionary Computation – GECCO-2004, Part I*, Vol. 3102 of *Lecture Notes in Computer Science*, Springer-Verlag,

Using Quantitative Genetics and Phenotypic Traits in Genetic Programming 25

[6] Bassett, J. K., Potter, M. A. & De Jong, K. A. [2005]. Applying Price's equation to survival selection, *in* H.-G. Beyer, U.-M. O'Reilly, D. V. Arnold, W. Banzhaf, C. Blum, E. W. Bonabeau, E. Cantu-Paz, D. Dasgupta, K. Deb, J. A. Foster, E. D. de Jong, H. Lipson, X. Llora, S. Mancoridis, M. Pelikan, G. R. Raidl, T. Soule, A. M. Tyrrell, J.-P. Watson & E. Zitzler (eds), *GECCO 2005: Proceedings of the 2005 Conference on Genetic and Evolutionary*

[7] Burke, E., Gustafson, S. & Kendall, G. [2002]. A survey and analysis of diversity measures

[8] Burke, E. K., Gustafson, S., Kendall, G. & Krasnogor, N. [2003]. Is increased diversity in genetic programming beneficial? an analysis of lineage selection, *Congress on*

[9] Falconer, D. S. & Mackay, T. F. C. [1981]. *Introduction to quantitative genetics*, Longman

[10] Flury, B. [1988]. *Common Principal Components and Related Multivariate Models*, Wiley

[11] Frank, S. A. [1995]. George price's contributions to evolutionary genetics, *Journal of*

[12] Game, E. T. & Caley, M. J. [2006]. The stability of P in coral reef fishes, *Evolution*

[13] Lande, R. & Arnold, S. J. [1983]. The measurement of selection on correlated characters,

[14] Langdon, W. B. [1998a]. *Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic Programming!*, The Kluwer international series in engineering

[15] Langdon, W. B. [1998b]. Size fair and homologous tree genetic programming crossovers.

[18] Manderick, B., de Weger, M. & Spiessens, P. [1991]. The genetic algorithm and the structure of the fitness landscape, *in* R. K. Belew & L. B. Booker (eds), *Proc. of the Fourth*

*Int. Conf. on Genetic Algorithms*, Morgan Kaufmann, San Mateo, CA, pp. 143–150. [19] Mühlenbein, H. [1997]. The equation for response to selection and its use for prediction,

[16] Langdon, W. B. & Poli, R. [2002]. *Foundations of Genetic Programming*, Springer-Verlag. [17] Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Popovici, E., Sullivan, K., Harrison, J., Bassett, J., Hubley, R., Chircop, A., Compton, J., Haddon, W., Donnelly, S., Jamil, B. &

O'Beirne, J. [2010]. ECJ: A java-based evolutionary computation research.

URL: *http://www.sciencedirect.com/science/article/B6WMD-45R8FXC-3N/2/01fea9e865de0a05*

*Computation*, Vol. 2, ACM Press, Washington DC, USA, pp. 1371–1378. URL: *http://www.cs.bham.ac.uk/ wbl/biblio/gecco2005/docs/p1371.pdf*

series in probability and mathematical statistics, Wiley, New York.

Seattle, WA, USA, pp. 914–922.

in genetic programming, pp. 716–723.

*Theoretical Biology* 175(3): 373–388.

New York.

*4158ee82d6237ef7*

*Evolution* 37(6): 1210–1226.

URL: *http://www.jstor.org/stable/2408842*

URL: *http://cs.gmu.edu/ eclab/projects/ecj/*

*Evolutionary Computation* 5(3): 303–346.

genetic programming and evolvable machines.

60(4): 814–823.

*Evolutionary Computation*, IEEE Press, pp. 1398–1405.

URL: *http://dx.doi.org/10.1111/j.0014-3820.2006.tb01159.x*

and computer science, Kluwer Academic Publishers, Boston.

• **Homologous Crossover with Tournament Selection size 2** Figure 11 shows various quantitative genetic metrics similar to previous experiments for quintic regression problem.

#### 3. **Symbolic Regression Observations**

