**2. Background**

Since the introduction of the Knapsack problems some algorithm techniques such as brute force, conventional algorithms, dynamic programming, greedy approach and approximation algorithm have been proposed (Ahmed & Younas, 2011).

Evolutionary algorithms (EAs) have been widely applied to the MKP and have shown to be effective for searching and finding good quality solutions (Chu & Beasley, 1998). It is important to note that MKP is considered a NP hard problem; hence any dynamic programming solution will produce results in exponential time. In the last few years, Genetic Algorithms (GAs) have been used to solve the NP-complete problems and have shown to be very well suited for solving larger Knapsack Problems (Fukunaga & Tazoe, 2009; Gunther, 1998; Sivaraj & Ravichandran, 2011). For larger knapsack problems, the efficiency of approximation algorithms is limited in both solution quality and computational 386 Bio-Inspired Computational Algorithms and Their Applications

economics (Tavares et al., 2008). MKP has received wide attention from the operations research community, because it embraces many practical problems. In addition, the MKP can be seen as a general model for any kind of binary problems with positive coefficients

Many researchers have proposed the high potential of the hybrid-model for the solution of problems (Gallardo et al., 2007). The algorithms presented in this work to solve MKP are a combination of CAs with a Multi Population model. The Multi Population model is the division of a population into several smaller ones, usually called the island model. Each sub-population runs a standard sequential evolution proceeds, as if it were isolated from the rest, with occasional migration of individuals between sub-populations (Tomassini,

In order to conduct an investigation to discover improvements for MKP, this work is centered in the knowledge produced from CAs through the evolutionary process that utilizes a population-based Genetic Algorithm model, using various MKP benchmarks found in the literature. In addition, there is an interest in investigating how to deal with the

So as to compare test results, we implemented the follows algorithms: the standard cultural algorithm with Single Population (also known as standard CA or CA-S) and Cultural Algorithm with Multi Population defined as CA-IM with two versions: CA-IM\_1 which has fixed values for genetic operators (recombination and mutation) and CA-IM\_2 which does not have fixed values for genetic operators because these values are generated randomly. In order to evaluate the performance of the CA-IM algorithms, some comparison testing will be conducted with other two algorithms based on Distributed GA, called DGA and DGA-

The outline of the paper is as follows: in Section 2, a description with formal denition of the MKP problem and an overview of Cultural Algorithms are presented. Section 3 shows an alternative approach that explores the multi population model with Cultural Algorithms and explores how the interaction process occurs among various sub-populations. Our experimental results are shown in Section 4 and finally we show some conclusions in

Since the introduction of the Knapsack problems some algorithm techniques such as brute force, conventional algorithms, dynamic programming, greedy approach and

Evolutionary algorithms (EAs) have been widely applied to the MKP and have shown to be effective for searching and finding good quality solutions (Chu & Beasley, 1998). It is important to note that MKP is considered a NP hard problem; hence any dynamic programming solution will produce results in exponential time. In the last few years, Genetic Algorithms (GAs) have been used to solve the NP-complete problems and have shown to be very well suited for solving larger Knapsack Problems (Fukunaga & Tazoe, 2009; Gunther, 1998; Sivaraj & Ravichandran, 2011). For larger knapsack problems, the efficiency of approximation algorithms is limited in both solution quality and computational

approximation algorithm have been proposed (Ahmed & Younas, 2011).

Cultural Algorithms considering a population-based in Genetic Algorithms.

(Glover & Kochenberger, 1996).

SRM (Aguirre et al., 2000).

Section 5.

**2. Background** 

2005).

cost (Ahmed & Younas, 2011). Spillman's experiment, which applies the GA to the knapsack problem, shows that the GA does not have a good performance in relatively small size problem, but works quite well in problems that include a huge number of elements (Spillman, 1995). There are many packing problems where evolutionary methods have been applied. The simplest optimization problem and one of the most studied is the onedimensional (zero–one or 0-1) knapsack problem (Ahmed & Younas, 2011), which given a knapsack of a certain capacity, and a set of items, each one having a particular size and value, nds the set of items with maximum value which can be accommodated in the knapsack. Various real-world problems are of this type: for example, the allocation of communication channels to customers who are charged at different rates (Back et al., 1997).

During a study of 0-1 knapsack, a number of extensions and variants have been developed such as (Ahmed & Younas, 2011): Multiple Knapsack Problems (MKP), Multidimensional Knapsack Problems (MDKP), Multi Choice Knapsack Problems (MCKP) and Multiple Choice Multidimensional Knapsack Problems (MMKP). It is also important to consider other extensions such as (Chu & Beasley, 1998): Multiconstraint Knapsack Problem, and also the term "Multidimensional Zero-one Knapsack Problem". Using alternative names for the same problem is potentially confusing, but since, historically, the designation **MKP** has been the most widely used (Chu & Beasley, 1998). Consequently, Multidimensional Knapsack Problem (MKP) is the designation selected for this work. In our previous research it was introduced a Multi Population Model on the cultural structure identified as "Multi Population Cultural Genetic Algorithm" (MCGA) (Silva & Oliveira, 2009). In MCGA model several sub-populations are connected with as ring structure, where the migration of individuals occurs after a generation interval (according to the migration based on parameter interval) with best-worst migration policy implementation. The results were satisfactory in relation to other algorithms in the literature. In another research two versions of Distributed GA (DGA) are presented as follows: standard Distributed GA (DGA) and an improved DGA (DGA-SRM), which two genetic operators are applied in parallel mode to create offspring. The term SRM represents "Self-Reproduction with Mutation", that is applied to various 0/1 multiple knapsack problems so as to improve the search performance (Aguire et al., 2000). Hybridization of memetic algorithms with Branch-and-Bound techniques (BnB) is also utilized for solving combinatorial optimization problems (Gallardo et al., 2007). BnB techniques use an implicit enumeration scheme for exploring the search space in an "intelligent" way. Yet another research utilizes adaptive GA for 0/1 Knapsack problems where special consideration is given to the penalty function where constant and self-adaptive penalty functions are adopted (Zoheir, 2002). Fitness landscape analysis techniques are used to better understand the properties of different representations that are commonly adopted when evolutionary algorithms are applied to MKP (Tavares et al., 2008). Other investigation utilizes multiple representations in a GA for the MKP (Representation-Switching GA) know as RSGA (Fukunaga, 2009). Other recent works consider two heuristics and utilize them for making comparisons to the well-known multiobjective evolutionary algorithms (MOEAs) (Kumar & Singh, 2010). While comparing MOEAs with the two heuristics, it was observed that the solutions obtained by the heuristics are far superior for larger problem instances than those obtained by MOEAs.

Performance Study of Cultural Algorithms Based on

probability of applying an operator, etc.).

identification that is given by the bit index.

Fig. 1. Knapsack example for two chromosomes.

**2.2.1 Genotype** 

are: 2, 3, 5 and 6.

**2.2.2 Initial population** 

**2.2.3 Evaluation function** 

population is generated at random.

Genetic Algorithm with Single and Multi Population for the MKP 389

constant values for parameters that are used by the algorithm (such as population size,

The natural representation of the MKP would be the binary representation, in which every bit represents the existence or not of a certain element in the Knapsack. A bit set to 1 indicates that the corresponding item is packed into the knapsack and a bit set to 0 indicates that it is not packed. Hence a typical population of two individuals for a six elements in Knapsack would be represented as showed in Figure 1. Thus, each element has an

In Figure 1 (a) there are three elements in the knapsack, corresponding to the following positions: 1, 4 and 6. In Figure 1 (b) there are four elements in the knapsack, whose positions

The population is the solution representation that consists of a set of codified chromosomes. There are many ways to generate the initial population such as random chromosome or chromosome with the solution closer to the optimum. In most applications the initial

In GA each individual is evaluated by fitness function. Some individuals produce more children than others due to their fitness. By this mechanism, individuals that have chromosomes with better fitness have better chances of leaving their genes. This leads to better average performance of the whole population as generations proceed (Ku & Lee, 2001). A feasible vector solution *x* needs to satisfy constraint (2), otherwise it is infeasible. Hence, a penalty is applied to all infeasible solutions in order to decrease their corresponding "fitness". Therefore, the two types of evaluation functions used in this research are based on static (constant) and adaptive penalty functions. The standard

> *i n i*

= =

Maximum Profit Possible (MaxP) = 1

( [ ] [ ]) ( )

× − (5)

(6)

[ ] *i n i p i* = =

*x i p i Pen x*

evaluation function for each individual is given by the following expressions:

Evaluation (x) = 1

A vector solution x is optimal when Evaluation (x) =MaxP.

#### **2.1 Multidimensional Knapsack Problem**

As mentioned earlier, the MKP is a well-known nondeterministic-polynomial time-hard combinatorial optimization problem, with a wide range of applications (Tavares et al., 2008). The classical 0-1 knapsack problem is one of the most studied optimization and involves the selection of a subset of available items having maximum profit so that the total weight of the chosen subset does not exceed the knapsack capacity. The problem can be described as follows: given two sets of *n* items and *m* knapsacks constraints (or resources), for each item *j*, a prot *pj* is assigned, and for each constraint *i*, a consumption value *rij* is designated. The goal is to determine a set of items that maximizes the total prot, not exceeding the given constraint capacities *ci*. Formally, this is stated as follows (Tavares et al., 2008):

*j*

*n*

Maximize 1

$$\sum\_{j=1}^{n} p\_j \mathbf{x}\_{j,n} \tag{1}$$

$$\text{Subject to } \qquad \sum\_{j=1}^{r} \mathbf{r}\_{i,j} \mathbf{x}\_{j} \le \mathbf{c}\_{i,} \quad \mathbf{i} = \mathbf{1}, \dots, m \tag{2}$$

$$\mathbf{x}\_{j} \in \{0, 1\}, \quad j = 1, \ldots, n \tag{3}$$

$$\text{With } \qquad \qquad p\_{\downarrow} > 0 \text{ , } r\_{i,\downarrow} \ge 0 \text{ and } \ c\_{i} \ge 0 \tag{4}$$

The knapsack constraint is represented by each of the *m* constraints described in Eq. (2). The decision variable is the binary vector x =(x1,...,xn). Each item *j* is mapped to a bit and when *xj* **=1**, the corresponding item is considered to be part of the solution. The special case of *m =1* is generally known as the Knapsack Problem or the Unidimensional Knapsack Problem.

For single constraint the problem is not strongly NP-hard and effective approximation algorithms have been developed for obtaining near-optimal solutions. A review of the single knapsack problem and heuristic algorithms is given by Martello and Toth (Martello & Toth, 1990). Exact techniques and exhaustive search algorithms, such as branch-andbound, are only of practical use in solving MKP instances of small size since they are, in general, too time-consuming (e.g., instances with 100 items or less, and depending on the constraints).

#### **2.2 Evolutionary approach for the MKP**

In a resolution of specific problems that implements an Evolutionary Algorithm, as for example, a simple Genetic Algorithm (GA), it is necessary the definition of five components (Agapie et al., 1997). The first component is the genotype or a genetic representation of the potential problem (individual representation scheme). The second is a method for creating an initial population of solutions. The third is a function verifying the fitness of the solution (*objective function* or *fitness function*). The fourth are genetic operators and the fifth are some constant values for parameters that are used by the algorithm (such as population size, probability of applying an operator, etc.).

#### **2.2.1 Genotype**

388 Bio-Inspired Computational Algorithms and Their Applications

As mentioned earlier, the MKP is a well-known nondeterministic-polynomial time-hard combinatorial optimization problem, with a wide range of applications (Tavares et al., 2008). The classical 0-1 knapsack problem is one of the most studied optimization and involves the selection of a subset of available items having maximum profit so that the total weight of the chosen subset does not exceed the knapsack capacity. The problem can be described as follows: given two sets of *n* items and *m* knapsacks constraints (or resources), for each item *j*, a prot *pj* is assigned, and for each constraint *i*, a consumption value *rij* is designated. The goal is to determine a set of items that maximizes the total prot, not exceeding the given constraint capacities *ci*. Formally, this is stated as follows

*n*

*j p x* =

*ij j i*

*rx c*

1 *n*

*j*

=

*j j*

{0,1}, *<sup>j</sup> x* ∈ j =1,….,*n* (3)

With 0 *<sup>j</sup> p* > , , 0 *i j r* ≥ and 0 *<sup>i</sup> c* ≥ (4)

The knapsack constraint is represented by each of the *m* constraints described in Eq. (2). The decision variable is the binary vector x =(x1,...,xn). Each item *j* is mapped to a bit and when *xj* **=1**, the corresponding item is considered to be part of the solution. The special case of *m =1* is generally known as the Knapsack Problem or the Unidimensional

For single constraint the problem is not strongly NP-hard and effective approximation algorithms have been developed for obtaining near-optimal solutions. A review of the single knapsack problem and heuristic algorithms is given by Martello and Toth (Martello & Toth, 1990). Exact techniques and exhaustive search algorithms, such as branch-andbound, are only of practical use in solving MKP instances of small size since they are, in general, too time-consuming (e.g., instances with 100 items or less, and depending on the

In a resolution of specific problems that implements an Evolutionary Algorithm, as for example, a simple Genetic Algorithm (GA), it is necessary the definition of five components (Agapie et al., 1997). The first component is the genotype or a genetic representation of the potential problem (individual representation scheme). The second is a method for creating an initial population of solutions. The third is a function verifying the fitness of the solution (*objective function* or *fitness function*). The fourth are genetic operators and the fifth are some

, (1)

<sup>≤</sup> i =1,…,*m* (2)

**2.1 Multidimensional Knapsack Problem** 

Maximize 1

Subject to , ,

**2.2 Evolutionary approach for the MKP** 

(Tavares et al., 2008):

Knapsack Problem.

constraints).

The natural representation of the MKP would be the binary representation, in which every bit represents the existence or not of a certain element in the Knapsack. A bit set to 1 indicates that the corresponding item is packed into the knapsack and a bit set to 0 indicates that it is not packed. Hence a typical population of two individuals for a six elements in Knapsack would be represented as showed in Figure 1. Thus, each element has an identification that is given by the bit index.

In Figure 1 (a) there are three elements in the knapsack, corresponding to the following positions: 1, 4 and 6. In Figure 1 (b) there are four elements in the knapsack, whose positions are: 2, 3, 5 and 6.

Fig. 1. Knapsack example for two chromosomes.

### **2.2.2 Initial population**

The population is the solution representation that consists of a set of codified chromosomes. There are many ways to generate the initial population such as random chromosome or chromosome with the solution closer to the optimum. In most applications the initial population is generated at random.

#### **2.2.3 Evaluation function**

In GA each individual is evaluated by fitness function. Some individuals produce more children than others due to their fitness. By this mechanism, individuals that have chromosomes with better fitness have better chances of leaving their genes. This leads to better average performance of the whole population as generations proceed (Ku & Lee, 2001). A feasible vector solution *x* needs to satisfy constraint (2), otherwise it is infeasible. Hence, a penalty is applied to all infeasible solutions in order to decrease their corresponding "fitness". Therefore, the two types of evaluation functions used in this research are based on static (constant) and adaptive penalty functions. The standard evaluation function for each individual is given by the following expressions:

$$\text{Evaluating (x)} = \sum\_{i=1}^{i=n} (x[i] \times p[i]) - Pen(x) \tag{5}$$

$$\text{Maximum Profit Posible (MaxP)} = \sum\_{i=1}^{i=n} p[i] \tag{6}$$

A vector solution x is optimal when Evaluation (x) =MaxP.

Performance Study of Cultural Algorithms Based on

Genetic Algorithm with Single and Multi Population for the MKP 391

(Reynolds & Chung, 1996). As a result, CAs can provide an explicit mechanism for global knowledge and a useful framework within which to model self-adaptation in an EC system. The CAs are based on knowledge of an evolutionary system that implements a dual mechanism of inheritance. This mechanism allows the CAs to explore as much microevolution as macroevolution. Microevolution is the evolution that happens in the population level. Macroevolution occurs on the culture itself, i.e. the belief space evolution. The belief space is the place where the information on the solution of the problem is refined and stored. It is acquired through the population space over the evolutionary process. The belief space has the goal to guide individuals in search of better regions. In the CAs evolution occurs more quickly than in population without the mechanism of macroevolution. The characteristics and behaviors of individuals are represented in the Population Space and as mentioned earlier the population space can support any population-based computational model such as Genetic Algorithms among others (Jin & Reynolds, 1999). The communications protocols dictate the rules about individuals that can contribute to knowledge in the Belief Space (function of acceptance) and how the Belief

Space will influence new individuals (Function of Influence), as shown in Figure 2.

The two most used ways to represent knowledge in the belief space are (Reynolds & Peng, 2004): Situational Knowledge and Normative Knowledge. Situational Knowledge represents the best individuals found at a certain time of evolution and it contains a number of individuals considered as a set of exemplars to the rest of the population. The number of exemplars may vary according to the implementation, but it is usually small. For example, the structure used to represent this type of knowledge is shown in Figure 3. Each individual

The Situational Knowledge is updated when the best individual of the population is found. This occurs when its fitness value exceeds the fitness value of the worst individual stored.

Fig. 2. Framework of Cultural Algorithm (Reynolds & Peng, 2004).

is stored within its parameters and fitness value (Iacoban et al. 2003).

Fig. 3. Representation of Situational Knowledge.

### **2.2.4 Genetic operators**

To implement the GA process, many factors should be considered such as the representation scheme of chromosomes, the mating strategy, the size of population, and the design of the genetic operators such as selection, mutation and recombination (Ku & Lee, 2001).


The tournament is the selection type chosen for this work since it is more used and it presents good performance. For a binary representation, classical crossover and mutation operators can be used, such as n-point crossover or uniform crossover, and bit-ip mutation. In CAs the influence of information from Belief Space on recombination and mutation process such as: best chromosome or set of best chromosomes information is expected.

#### **2.2.5 Constant values parameters**

An Evolutionary Algorithm involves different strategy parameters, e.g.: mutation rate, crossover rate, selective pressure (e.g., tournament size) and population size. Good parameter values lead to good performance. There are three major types of parameter control (Eiben & Smith, 2008):


The implementation of a deterministic parameter control is easier, provided that the parameter values used are tested to verify the best performance.

#### **2.3 Cultural algorithms**

Cultural Algorithms (CAs) have been developed so as to model the evolution of the cultural component of an evolutionary computational system over time as it accumulates experience 390 Bio-Inspired Computational Algorithms and Their Applications

To implement the GA process, many factors should be considered such as the representation scheme of chromosomes, the mating strategy, the size of population, and the design of the

i. **Selection** - is an operator that prevents low fitness individuals from reproduction and permits high fitness individuals to offspring more children to improve average fitness of population over generations. There are various selections types, such as stochastic

ii. **Recombination or Crossover** - is an operator that mixes the chromosomes of two individuals. Typically two children are generated by applying this operator, which are similar to the parents but not same. Crossover causes a structured, yet randomized exchange of genetic material between solutions, with the possibility that the "fittest" solutions generate "better" ones. A crossover operator should preserve as much as

iii. **Mutation** - introduces totally new individuals to population. It helps extend the domain of search and will restrain the diversity of the population. Mutation involves the modification of each bit of an individual with some probability Pm. Although the mutation operator has the effect of destroying the structure of a potential solution, chances are it will yield a better solution. Mutation in GAs restores lost or unexplored genetic material into the population to prevent the premature convergence of the GA. The tournament is the selection type chosen for this work since it is more used and it presents good performance. For a binary representation, classical crossover and mutation operators can be used, such as n-point crossover or uniform crossover, and bit-ip mutation. In CAs the influence of information from Belief Space on recombination and mutation process such as: best chromosome or set of best chromosomes information is expected.

An Evolutionary Algorithm involves different strategy parameters, e.g.: mutation rate, crossover rate, selective pressure (e.g., tournament size) and population size. Good parameter values lead to good performance. There are three major types of parameter

• **deterministic**: a rule modifies strategy parameter without feedback from the search

• **adaptive**: a feedback rule based on some measure monitoring search progress (quality); • **self-adaptative**: parameter values evolve along with the solutions; encoded onto

The implementation of a deterministic parameter control is easier, provided that the

Cultural Algorithms (CAs) have been developed so as to model the evolution of the cultural component of an evolutionary computational system over time as it accumulates experience

genetic operators such as selection, mutation and recombination (Ku & Lee, 2001).

remainder, elitism, crowding factor model, tournament, and roulette wheel.

possible from the parents while creating an offspring.

**2.2.5 Constant values parameters** 

(based on some type of a counter);

chromosomes they undergo variation and selection.

parameter values used are tested to verify the best performance.

control (Eiben & Smith, 2008):

**2.3 Cultural algorithms** 

**2.2.4 Genetic operators** 

(Reynolds & Chung, 1996). As a result, CAs can provide an explicit mechanism for global knowledge and a useful framework within which to model self-adaptation in an EC system. The CAs are based on knowledge of an evolutionary system that implements a dual mechanism of inheritance. This mechanism allows the CAs to explore as much microevolution as macroevolution. Microevolution is the evolution that happens in the population level. Macroevolution occurs on the culture itself, i.e. the belief space evolution. The belief space is the place where the information on the solution of the problem is refined and stored. It is acquired through the population space over the evolutionary process. The belief space has the goal to guide individuals in search of better regions. In the CAs evolution occurs more quickly than in population without the mechanism of macroevolution. The characteristics and behaviors of individuals are represented in the Population Space and as mentioned earlier the population space can support any population-based computational model such as Genetic Algorithms among others (Jin & Reynolds, 1999). The communications protocols dictate the rules about individuals that can contribute to knowledge in the Belief Space (function of acceptance) and how the Belief Space will influence new individuals (Function of Influence), as shown in Figure 2.

Fig. 2. Framework of Cultural Algorithm (Reynolds & Peng, 2004).

The two most used ways to represent knowledge in the belief space are (Reynolds & Peng, 2004): Situational Knowledge and Normative Knowledge. Situational Knowledge represents the best individuals found at a certain time of evolution and it contains a number of individuals considered as a set of exemplars to the rest of the population. The number of exemplars may vary according to the implementation, but it is usually small. For example, the structure used to represent this type of knowledge is shown in Figure 3. Each individual is stored within its parameters and fitness value (Iacoban et al. 2003).

Fig. 3. Representation of Situational Knowledge.

The Situational Knowledge is updated when the best individual of the population is found. This occurs when its fitness value exceeds the fitness value of the worst individual stored.

Performance Study of Cultural Algorithms Based on

migrate and the frequency of migrations (Sinvanadam, 2007).

Nowostawski and Poli (Nowostawski & Poli, 1999).

**2.5 Island model (Multi Population Genetic Algorithms)** 

Genetic Algorithm with Single and Multi Population for the MKP 393

like migration rate, topology, migration scheme like best/worst/random individuals to

Other authors classify Parallel Genetic Algorithm in four main categories (Aguirre & Tanaka, 2006): global master-slave, island, cellular, and hierarchical parallel GAs. In a global master-slave GA there is a single population and the evaluation of tness is distributed among several processors. The important characteristic in a global master-slave GA is that the entire population is considered by genetic operators as selection, crossover and mutation. An island GA, also known as coarse-grained or distributed GA, consists of several sub-populations evolving separately with occasional migration of individuals between subpopulations. A cellular category also known as "ne-grained GA" consists of one spatially structured population, whose selection and mating are restricted to a small neighborhood. The neighborhoods are allowed to overlap permitting some interaction among individuals. Finally, a hierarchical parallel GA category, combines an island model with either a masterslave or cellular GA. The global master-slave GA does not affect the behavior of the algorithm and can be considered only as a hardware accelerator. However, the other parallel formulations of GAs are very different from canonical GAs, especially, with regard to population structure and selection mechanisms. These modications change the way the GA works, affecting its dynamics and the trajectory of evolution. For example, the utilization of parameters as sub-population size, migration rate and migration frequency are crucial to the performance of island models. Cellular, island and hierarchical models perform as well as or better than canonical versions and have the potential of being more than just hardware accelerators (Aguirre & Tanaka, 2006). A new taxonomy about PGAs is also presented by

In recent studies about MKP Silva and Oliveira (Silva & Oliveira, 2009) have shown that good results are reached in the benchmark tests when taking into consideration the implementation of sub-populations and the migration process from the island model. The results presented were better than canonical version of Cultural Algorithm in most cases.

Multi population Genetic Algorithms (MGAs) or Island Model, is an extension of traditional single-population Genetic Algorithms (SGAs) by dividing a population into several subpopulations within which the evolution proceeds and individuals are allowed to migrate from one sub-population to another. Different values for parameters such as selection, recombination and mutation rate can be chosen for each sub-population. Normally, the

In order to control the migration of individuals, several parameters were defined such as: (i) the communication topology that defines the connections between sub-populations, (ii) a migration rate that controls how many individuals migrate, and (iii) a migration interval that affects the frequency of migration. In addition, migration must include strategies for

The sub-populations size, communication topology (its degree of connectivity), migration rate and migration frequency are important factors related to the performance of distributed GAS. In general, it has been shown that distributed GAs can produce solutions with similar or better quality than single population GAs, while reducing the overall time

basic island model uses the same values for these parameters in all sub-populations.

migrant selection and for their inclusion in their new sub-populations (Aguire, 2000).

Normative Knowledge represents a set of intervals that characterize the range of values given by the features that make the best solutions (Iacoban et al., 2003).

Figure 4 shows the structure used by Reynolds and his students, where are stored the minimum and maximum values on the individual's characteristics.

Fig. 4. Representation of Normative knowledge

These intervals are used to guide the adjustments (mutations) that occur in individuals. With these minimum values, (li) and maximum (ui), the fitness values are also stored. This value results from the individuals that produced each extreme Li and Ui respectively.

The adjustment of the range of Normative Knowledge varies according to the best individual. That is, if the individual was accepted by the *acceptance function* and its range is less than the range stored in the belief space, the range is adjusted, and vice versa.

The resolution of problems produces experiences from individuals in the population space, which are selected to contribute to the acceptance by the belief space, where the knowledge is generalized and stored. In the initial population, the individuals are evaluated by the fitness function. Then, the information on the performance of the function is used as a basis for the production of generalizations for next generations. The experiences of the individuals selected will be used to make the necessary adjustments on the knowledge of the current belief space.

#### **2.4 Parallel Genetic Algorithms**

The definition of Parallel Genetic Algorithms (PGAs) is related with execution of various GAs in parallel mode. The main goal of PGAs is to reduce the large execution times that are associated with simple genetic algorithms for nding near-optimal solutions in large search spaces and to nd better solutions.

The PGAs can be implemented through two approaches (Sivanandam, 2007): standard parallel approach and the decomposition approach. In the rst approach, the sequential GA model is implemented on a parallel computer by dividing the task of implementation among the processors. The standard parallel approaches are also known as *master-slave GAs.* In the decomposition approach, the full population exists in distributed form. Other characteristic in the decomposition approach is that the population is divided into a number of sub-populations called demes. Demes are separated from one another and individuals compete only within a deme. An additional operator called migration is used to move the individuals from one deme to another. If the individuals can migrate to any other deme, the model is called *island model* or *Multiple-population GAs* when implemented in parallel or distributed environments (Braun, 1991). Migration can be controlled by various parameters 392 Bio-Inspired Computational Algorithms and Their Applications

Normative Knowledge represents a set of intervals that characterize the range of values

Figure 4 shows the structure used by Reynolds and his students, where are stored the

These intervals are used to guide the adjustments (mutations) that occur in individuals. With these minimum values, (li) and maximum (ui), the fitness values are also stored. This value results from the individuals that produced each extreme Li and Ui respectively.

The adjustment of the range of Normative Knowledge varies according to the best individual. That is, if the individual was accepted by the *acceptance function* and its range is

The resolution of problems produces experiences from individuals in the population space, which are selected to contribute to the acceptance by the belief space, where the knowledge is generalized and stored. In the initial population, the individuals are evaluated by the fitness function. Then, the information on the performance of the function is used as a basis for the production of generalizations for next generations. The experiences of the individuals selected will be used to make the necessary adjustments on the knowledge of

The definition of Parallel Genetic Algorithms (PGAs) is related with execution of various GAs in parallel mode. The main goal of PGAs is to reduce the large execution times that are associated with simple genetic algorithms for nding near-optimal solutions in large search

The PGAs can be implemented through two approaches (Sivanandam, 2007): standard parallel approach and the decomposition approach. In the rst approach, the sequential GA model is implemented on a parallel computer by dividing the task of implementation among the processors. The standard parallel approaches are also known as *master-slave GAs.* In the decomposition approach, the full population exists in distributed form. Other characteristic in the decomposition approach is that the population is divided into a number of sub-populations called demes. Demes are separated from one another and individuals compete only within a deme. An additional operator called migration is used to move the individuals from one deme to another. If the individuals can migrate to any other deme, the model is called *island model* or *Multiple-population GAs* when implemented in parallel or distributed environments (Braun, 1991). Migration can be controlled by various parameters

less than the range stored in the belief space, the range is adjusted, and vice versa.

given by the features that make the best solutions (Iacoban et al., 2003).

minimum and maximum values on the individual's characteristics.

Fig. 4. Representation of Normative knowledge

the current belief space.

**2.4 Parallel Genetic Algorithms** 

spaces and to nd better solutions.

like migration rate, topology, migration scheme like best/worst/random individuals to migrate and the frequency of migrations (Sinvanadam, 2007).

Other authors classify Parallel Genetic Algorithm in four main categories (Aguirre & Tanaka, 2006): global master-slave, island, cellular, and hierarchical parallel GAs. In a global master-slave GA there is a single population and the evaluation of tness is distributed among several processors. The important characteristic in a global master-slave GA is that the entire population is considered by genetic operators as selection, crossover and mutation. An island GA, also known as coarse-grained or distributed GA, consists of several sub-populations evolving separately with occasional migration of individuals between subpopulations. A cellular category also known as "ne-grained GA" consists of one spatially structured population, whose selection and mating are restricted to a small neighborhood. The neighborhoods are allowed to overlap permitting some interaction among individuals. Finally, a hierarchical parallel GA category, combines an island model with either a masterslave or cellular GA. The global master-slave GA does not affect the behavior of the algorithm and can be considered only as a hardware accelerator. However, the other parallel formulations of GAs are very different from canonical GAs, especially, with regard to population structure and selection mechanisms. These modications change the way the GA works, affecting its dynamics and the trajectory of evolution. For example, the utilization of parameters as sub-population size, migration rate and migration frequency are crucial to the performance of island models. Cellular, island and hierarchical models perform as well as or better than canonical versions and have the potential of being more than just hardware accelerators (Aguirre & Tanaka, 2006). A new taxonomy about PGAs is also presented by Nowostawski and Poli (Nowostawski & Poli, 1999).

In recent studies about MKP Silva and Oliveira (Silva & Oliveira, 2009) have shown that good results are reached in the benchmark tests when taking into consideration the implementation of sub-populations and the migration process from the island model. The results presented were better than canonical version of Cultural Algorithm in most cases.

### **2.5 Island model (Multi Population Genetic Algorithms)**

Multi population Genetic Algorithms (MGAs) or Island Model, is an extension of traditional single-population Genetic Algorithms (SGAs) by dividing a population into several subpopulations within which the evolution proceeds and individuals are allowed to migrate from one sub-population to another. Different values for parameters such as selection, recombination and mutation rate can be chosen for each sub-population. Normally, the basic island model uses the same values for these parameters in all sub-populations.

In order to control the migration of individuals, several parameters were defined such as: (i) the communication topology that defines the connections between sub-populations, (ii) a migration rate that controls how many individuals migrate, and (iii) a migration interval that affects the frequency of migration. In addition, migration must include strategies for migrant selection and for their inclusion in their new sub-populations (Aguire, 2000).

The sub-populations size, communication topology (its degree of connectivity), migration rate and migration frequency are important factors related to the performance of distributed GAS. In general, it has been shown that distributed GAs can produce solutions with similar or better quality than single population GAs, while reducing the overall time

Performance Study of Cultural Algorithms Based on

(M) among generations and the best-worst migration policy is used.

the belief space that is the main characteristic present in CAs.

shows the framework correspondent to the proposed structure.

Fig. 6. Framework of model proposed

recreated randomly by algorithm.

rest of the population (Situational Knowledge).

Genetic Algorithm with Single and Multi Population for the MKP 395

structure. Individuals in classical island model are migrated after every migration-interval

The approach utilized in this work is an adaption and implementation of the island model on the cultural structure here identified as "Cultural Island Model" (CA-IM), briey introduced in Silva & Oliveira (Silva & Oliveira, 2009). The implementations have become simple because the same CAs structures were used as much the evolutionary structure as

 The main characteristic present in CA-IM is the link between main belief spaces (from main population) and secondary belief space (from multi population). They store information about independent evolution for main population and sub-populations respectively, i.e. the cultural evolutions occur in parallel among the main population and the sub-populations of the islands. The link of communication between two *Belief Spaces,* allows migration between the best individuals stored in the cultural knowledge structure implemented. Figure 6

Migrations from islands occur through *Belief Space Multipopulation* structure that perform the communication process among sub-populations and send the best individuals through *Accept Migration*. It occurs in a predefined interval whose parameter is *M* (every *M* generation) where the best individuals are evaluated by acceptance function and updated in each belief space. The migration from *Belief Space Multipopulation* to *Main Belief Space* is implemented as a number of individuals which are considered as a set of exemplars to the

It is important to note that CA-IM provides a continuous verification between the last solution (optimum value) found and the current solution. Then, it computes the number of generations where don't occur improvements. Thus, if the distance between the last generation, where the current solution was found, and the current generation is high then the sub-populations are eliminated and recreated randomly. As for CA-IM, there is a fixed difference for this occurrence in the range of 60 to 100 generations. If a new solution is not found in this range, then the sub-populations of the islands (Multipopulation Space) as well as the cultural information about all sub-populations (Belief Space Multipopulation) are

to completion in a factor that is almost in reciprocal proportion to the number of processors (Aguire, 2000).

In the island model GA, the sub-populations are isolated during selection, breeding and evaluation. Islands typically focus on the evolutionary process within sub-populations before migrating individuals to other islands, or conceptual processors, which also carry out an evolutionary process. At predetermined times, during the search process, islands send and receive migrants to other islands. There are many variations of distributed models, e.g. islands, demes, and niching methods, where each requires numerous parameters to be dened and tuned (Gustafson, 2006).

An example of the communication topology, can be defined as a graph in which the subpopulations *Pi* (i = 0, 1,..., K - 1) are the vertices and each defined edge Li,j specifies a communication link between the incident vertices Pi and Pj (neighbor sub-populations) (Aguire, 2000). In general, assuming a directed graph for each defined link Li,j we can indicate the number of individuals Ri,j that will migrate from P to Pj (migration rate) and the number of generations M between migration events (migration interval). The communication topology and migration rates could be static or dynamic and migration could be asynchronous or synchronous.

Various strategies for choosing migrants have been applied. Two strategies often used to select migrants are selection of the best and random selection. For example, the migration can implement a synchronous elitist broadcast strategy occurring every M generation. Each sub-population broadcasts a copy of its R best individuals to all its neighbor subpopulations.

Hence, every sub-population in every migration event receives migrants. Figure 5 illustrates a communication topology +l+2 island model in which each sub-population is linked to two neighbors (L = 2). In this example, the sub-population P0 can send individuals only to P1 and P2 and receive migrants only from P4 and P5.

Fig. 5. +1+2 communication topology.

#### **3. Cultural Island Model (CA-IM)**

In this section is presented an approach about the communication topology for migration process implemented in a Cultural Algorithm based on the island model. As noted earlier in the classical island model implementation, there are sub-populations connected with as ring 394 Bio-Inspired Computational Algorithms and Their Applications

to completion in a factor that is almost in reciprocal proportion to the number of processors

In the island model GA, the sub-populations are isolated during selection, breeding and evaluation. Islands typically focus on the evolutionary process within sub-populations before migrating individuals to other islands, or conceptual processors, which also carry out an evolutionary process. At predetermined times, during the search process, islands send and receive migrants to other islands. There are many variations of distributed models, e.g. islands, demes, and niching methods, where each requires numerous parameters to be

An example of the communication topology, can be defined as a graph in which the subpopulations *Pi* (i = 0, 1,..., K - 1) are the vertices and each defined edge Li,j specifies a communication link between the incident vertices Pi and Pj (neighbor sub-populations) (Aguire, 2000). In general, assuming a directed graph for each defined link Li,j we can indicate the number of individuals Ri,j that will migrate from P to Pj (migration rate) and the number of generations M between migration events (migration interval). The communication topology and migration rates could be static or dynamic and migration

Various strategies for choosing migrants have been applied. Two strategies often used to select migrants are selection of the best and random selection. For example, the migration can implement a synchronous elitist broadcast strategy occurring every M generation. Each sub-population broadcasts a copy of its R best individuals to all its neighbor sub-

Hence, every sub-population in every migration event receives migrants. Figure 5 illustrates a communication topology +l+2 island model in which each sub-population is linked to two neighbors (L = 2). In this example, the sub-population P0 can send individuals only to P1

In this section is presented an approach about the communication topology for migration process implemented in a Cultural Algorithm based on the island model. As noted earlier in the classical island model implementation, there are sub-populations connected with as ring

(Aguire, 2000).

populations.

dened and tuned (Gustafson, 2006).

could be asynchronous or synchronous.

and P2 and receive migrants only from P4 and P5.

Fig. 5. +1+2 communication topology.

**3. Cultural Island Model (CA-IM)** 

structure. Individuals in classical island model are migrated after every migration-interval (M) among generations and the best-worst migration policy is used.

The approach utilized in this work is an adaption and implementation of the island model on the cultural structure here identified as "Cultural Island Model" (CA-IM), briey introduced in Silva & Oliveira (Silva & Oliveira, 2009). The implementations have become simple because the same CAs structures were used as much the evolutionary structure as the belief space that is the main characteristic present in CAs.

 The main characteristic present in CA-IM is the link between main belief spaces (from main population) and secondary belief space (from multi population). They store information about independent evolution for main population and sub-populations respectively, i.e. the cultural evolutions occur in parallel among the main population and the sub-populations of the islands. The link of communication between two *Belief Spaces,* allows migration between the best individuals stored in the cultural knowledge structure implemented. Figure 6 shows the framework correspondent to the proposed structure.

Fig. 6. Framework of model proposed

Migrations from islands occur through *Belief Space Multipopulation* structure that perform the communication process among sub-populations and send the best individuals through *Accept Migration*. It occurs in a predefined interval whose parameter is *M* (every *M* generation) where the best individuals are evaluated by acceptance function and updated in each belief space. The migration from *Belief Space Multipopulation* to *Main Belief Space* is implemented as a number of individuals which are considered as a set of exemplars to the rest of the population (Situational Knowledge).

It is important to note that CA-IM provides a continuous verification between the last solution (optimum value) found and the current solution. Then, it computes the number of generations where don't occur improvements. Thus, if the distance between the last generation, where the current solution was found, and the current generation is high then the sub-populations are eliminated and recreated randomly. As for CA-IM, there is a fixed difference for this occurrence in the range of 60 to 100 generations. If a new solution is not found in this range, then the sub-populations of the islands (Multipopulation Space) as well as the cultural information about all sub-populations (Belief Space Multipopulation) are recreated randomly by algorithm.

Performance Study of Cultural Algorithms Based on

Fig. 8. Recombination pseudo-code.

Algorithm with single population.

beneficial mutations.

**4.1 DGA and DGA-SRM** 

**4. Experimental results and discussion** 

Genetic Algorithm with Single and Multi Population for the MKP 397

To evaluate the performance of the proposed algorithm CA-IM, a comparison of various tests with Distributed Genetic Algorithms utilizing the same knapsack problems was carried out. To make a comparison two kinds of algorithms based in Distributed GAs (Aguirre et al., 2000): (i) A Distributed canonical GA (denoted as DGA), and (ii) a Distributed GA-SRM (denoted as DGA-SRM) were utilized. The SRM term means "Self-Reproduction with Mutation", and introduces diversity by means of mutation inducing the appearance of

For the CA-IM algorithm there are two versions: CA-IM\_1 and CA-IM\_2. The only difference is that CA-IM\_1 has a fixed rate for mutation and recombination, while CA-IM\_2 has a random rate for mutation and recombination. The standard CA (CAs) is the Cultural

The DGA works with various 0/1 multiple knapsack problems (NP hard combinatorial) which from previous efforts seem to be fairly difficult for GAs (Aguirre et al., 2000). Those algorithms were evaluated on test problems which are taken from the literature. The problem sizes range from 15 objects to 105 and from 2 to 30 knapsacks and can be found in OR-Library (Beasley, 1990). The knapsack problems are defined by: problem *(n, m)* where *n* represents the number of objects and *m* represents the number of knapsacks. Each knapsack

### **3.1 Mutation and recombination**

In mutation operation the cultural knowledge (such as situational knowledge) as well as the standard binary mutation operation (known as "bit-flip mutation") is utilized. If the cultural knowledge is utilized during the mutation process, the mutated chromosome genes are replaced by the best genes from chromosome stored in situational knowledge with PM probability, otherwise, the genes are inverted by bit-flip mutation. The chromosome chosen among a set of chromosomes from situational knowledge can be the best chromosome or a random chromosome.

The bit-ip mutation is a common operation applied in evolutionary algorithms to solve a problem with binary representation. Consequently, each bit from current mutated chromosome is ipped, i.e. the value of the chosen gene is inverted also with probability of mutation PM. Figure 7 shows the pseudo-code of mutation utilized by CA-IM.

Fig. 7. Mutation pseudo-code.

In recombination operation the cultural knowledge as well as the standard binary recombination operation (known as "uniform recombination") is also utilized. In the uniform recombination the bits are randomly copied from the first or from the second parent to genes in the offspring chromosomes, in any sequence of ones and zeros. Figure 8 shows the pseudo-code of CA-IM recombination.

If the cultural knowledge is utilized during the recombination process, the chromosome genes are replaced by the best genes from chromosome stored in situational knowledge with PR probability. Otherwise, the genes are replaced with genes from their parents. Here only the best chromosome is chosen from situational knowledge during the recombination process.

396 Bio-Inspired Computational Algorithms and Their Applications

In mutation operation the cultural knowledge (such as situational knowledge) as well as the standard binary mutation operation (known as "bit-flip mutation") is utilized. If the cultural knowledge is utilized during the mutation process, the mutated chromosome genes are replaced by the best genes from chromosome stored in situational knowledge with PM probability, otherwise, the genes are inverted by bit-flip mutation. The chromosome chosen among a set of chromosomes from situational knowledge can be the best chromosome or a

The bit-ip mutation is a common operation applied in evolutionary algorithms to solve a problem with binary representation. Consequently, each bit from current mutated chromosome is ipped, i.e. the value of the chosen gene is inverted also with probability of

In recombination operation the cultural knowledge as well as the standard binary recombination operation (known as "uniform recombination") is also utilized. In the uniform recombination the bits are randomly copied from the first or from the second parent to genes in the offspring chromosomes, in any sequence of ones and zeros. Figure 8

If the cultural knowledge is utilized during the recombination process, the chromosome genes are replaced by the best genes from chromosome stored in situational knowledge with PR probability. Otherwise, the genes are replaced with genes from their parents. Here only the best chromosome is chosen from situational knowledge during the recombination

mutation PM. Figure 7 shows the pseudo-code of mutation utilized by CA-IM.

**3.1 Mutation and recombination** 

random chromosome.

Fig. 7. Mutation pseudo-code.

process.

shows the pseudo-code of CA-IM recombination.

Fig. 8. Recombination pseudo-code.
