**4. Artificial immune systems**

In the human body, the immune system is the collection of cells, tissues and molecules that defend the body against infections [20]. Its basic function is to eradicate and prevent infections.

Naturally in our body there are two types of immunity: innate immunity and adaptive immunity, both share the same objective, but have different tasks and time reactions. Innate immunity acts in the first hours whereas adaptive immunity operates through days.

Adaptive immunity can further be classified as humeral immunity and cell-mediated immun‐ ity, containing different responding, effectors cells and functions.

A detailed biologically inclined overview of the Artificial Immune System can be found in [20]; while a computationally inclined overview can be found at [21].

Simulating active immunity is of great interest for the CODA algorithm. The most common example is vaccination of an individual, in this case the "naive individual" is exposed to antigens in order to mount an active response and be able to eradicate the infection. After this process the individual will be immune to that microbe and that is because it has already built resistance for a later infection. In contrast, passive immunity is shown in new-borns that do not have an immune system mature enough to fight against pathogens, but are protected by their mother's antibodies through the placenta and milk.

Artificial Immune Systems where developed with active immunity in mind since at the very first they were implemented for intrusion detection in computer servers. The main goal of the system was to detect insiders and external intruders that were committing abuse or misuse of the computer systems.

actuators. The reward could be the similarity between the learnt (desired) data and the produced data on the present state, and could be bigger if similarity is higher and reward

**Figure 3.** Reinforcement learning cycle, agent performs action, receives reward and ends up on a new state.

In the human body, the immune system is the collection of cells, tissues and molecules that defend the body against infections [20]. Its basic function is to eradicate and prevent infections.

Naturally in our body there are two types of immunity: innate immunity and adaptive immunity, both share the same objective, but have different tasks and time reactions. Innate

Adaptive immunity can further be classified as humeral immunity and cell-mediated immun‐

A detailed biologically inclined overview of the Artificial Immune System can be found in [20];

Simulating active immunity is of great interest for the CODA algorithm. The most common example is vaccination of an individual, in this case the "naive individual" is exposed to antigens in order to mount an active response and be able to eradicate the infection. After this process the individual will be immune to that microbe and that is because it has already built resistance for a later infection. In contrast, passive immunity is shown in new-borns that do not have an immune system mature enough to fight against pathogens, but are protected by

immunity acts in the first hours whereas adaptive immunity operates through days.

ity, containing different responding, effectors cells and functions.

while a computationally inclined overview can be found at [21].

their mother's antibodies through the placenta and milk.

should be lower if similarity is not likely.

260 Recent Advances in Robotic Systems

**4. Artificial immune systems**

There are several crucial characteristics that are important for the CODA algorithm to inherit the robustness, adaptability and specificity among other characteristics that are desirable for a real problem-solving algorithm. The characteristics that are considered important and should be applied in the computational version of the immune system are shown in **Table 2**.


**Table 2.** Natural Immune System characteristics that make it such a potent tool in the human body. This features are imitated by the computer counterparts of the NIS the AIS.

*Specificity and diversity*: This is an important feature that lets the immune system distinguish from similar antigens, helping us specify the response for a certain antigen and no response to any other even if they are quite similar. Therefore lymphocyte repertoire could be over millions of different units, all of them cloned, built specifically for different antigens, leading to a much extended antigen distinguishing.

*Memory*: When the system is exposed to an antigen for the first time, the response is called primary immune response and is mediated by lymphocytes, called naive lymphocytes. The second encounter is called secondary immune response, and it is supposed to be rapid, larger and more effective.

In order to implement and use an Artificial Immune System, it is quite important to understand deeply how the Artificial Immune systems works for the several processes that occur within the system and manage several tasks. In [22], a very explicit description of the AIS based upon the natural model from the human body of the Natural Immune System (NIS) can be found. According to the explanation of the model, the immune system is an example of a mechanism that is capable of learning and remembering. This memory is capable not only of string previous interactions but also forgetting information with little use.

The NIS is an example of a yet adaptive, decentralised and effective system. The B and T cells are just examples of how the NIS has a working structure that delegates all tasks. All the previous characteristics are desirable for a problem-solving algorithm that can offer novel methods and thus testing is required to compare this paradigm across all the machine learning methods.

In [22] the authors test the algorithm with a simple pattern recognition problem as a first test. In order to have a better overview of the performance of the AIS, it was applied to a real-world problem such as the recognition of the promoters in DNA sequences. According to the results presented, all were consistent with other approaches such as ANN and Quinlan ID3 obtaining similar results. The performance was better than Nearest Neighbour Algorithm.

The diversification of the AIS is an important characteristic since it does not focus on a global optima, instead the antibodies evolve in order to handle all the variety that the antigens can represent. **Figure 6** shows how the antigen is taken and mutated to produce all the antibodies represented by the blue circles. It is also quickly adaptable to changing situations, naturally the system can handle event-response situations. This is one of the most important features of the NIS as well, and the response must be active in a matter of minutes or hours in order to protect the human body.

According to [22] a remarkable characteristic of the AIS is the genetic mechanism that can mutate and reproduce the antibodies, memory and its self-organising properties. In CODA algorithm these concepts are implemented [22], for example, the self-organising characteristic implemented with a clustering step running over the antibodies in order to organise all data.

It is important to notice that the computational model presented in this document or in any of the articles that were revised does not aim to model the human NIS perfectly, nor it is an attempt to provide explanations of how the system works within the body. Rather, the stated features are emulated in order to solve problems that need such characteristics presented in **Table 1** and **Table 2**, such as specificity, diversity and memory, among others.

Farmer et al. presented Eq. (3) as a mathematical model of the immune system, in which *N* represents the antibodies, *n* represents the antigens, *c* is a rate constant that depends on the number of comparisons how antibodies are being stimulated, *a* represents the current B cell, *xej* represents the *jth* B cell's epitope, *xpj* represents the *jth* B cell object paratope and y represents the current antigen. Finally the first term in the sum is the affinity between antibodies and neighbours; the second term is the enmity between the previous named objects; the third represents how well the antibodies are capable of binding with the antigen; and the last term models the tendency of cells to die if no interactions are present.

$$\text{Minimulation} = c \left[ \sum\_{j=1}^{N} m(a, \mathbf{x} e\_j) - k\_1 \sum\_{j=1}^{N} m(a, \mathbf{x} p\_j) + k\_2 \sum\_{j=1}^{N} m(a, \mathbf{y}) \right] - k\_3,\tag{3}$$

The model used by Farmer et al. presented three types of mutation presented as follows.

*Multi-point mutation*: Each element in the antibody is processed in turn. If a randomly gener‐ ated number is above the mutation threshold, then the element is mutated.

*Substring regeneration*: Two points are selected at random in the antibody's paratope. Then all the elements between these two points are replaced by randomly generated elements, resulting in a partial regeneration of the antibody.

methods and thus testing is required to compare this paradigm across all the machine learning

In [22] the authors test the algorithm with a simple pattern recognition problem as a first test. In order to have a better overview of the performance of the AIS, it was applied to a real-world problem such as the recognition of the promoters in DNA sequences. According to the results presented, all were consistent with other approaches such as ANN and Quinlan ID3 obtaining

The diversification of the AIS is an important characteristic since it does not focus on a global optima, instead the antibodies evolve in order to handle all the variety that the antigens can represent. **Figure 6** shows how the antigen is taken and mutated to produce all the antibodies represented by the blue circles. It is also quickly adaptable to changing situations, naturally the system can handle event-response situations. This is one of the most important features of the NIS as well, and the response must be active in a matter of minutes or hours in order to

According to [22] a remarkable characteristic of the AIS is the genetic mechanism that can mutate and reproduce the antibodies, memory and its self-organising properties. In CODA algorithm these concepts are implemented [22], for example, the self-organising characteristic implemented with a clustering step running over the antibodies in order to organise all data.

It is important to notice that the computational model presented in this document or in any of the articles that were revised does not aim to model the human NIS perfectly, nor it is an attempt to provide explanations of how the system works within the body. Rather, the stated features are emulated in order to solve problems that need such characteristics presented in

Farmer et al. presented Eq. (3) as a mathematical model of the immune system, in which *N* represents the antibodies, *n* represents the antigens, *c* is a rate constant that depends on the number of comparisons how antibodies are being stimulated, *a* represents the current B cell,

the current antigen. Finally the first term in the sum is the affinity between antibodies and neighbours; the second term is the enmity between the previous named objects; the third represents how well the antibodies are capable of binding with the antigen; and the last term

11 1

The model used by Farmer et al. presented three types of mutation presented as follows.

ated number is above the mutation threshold, then the element is mutated.

*Multi-point mutation*: Each element in the antibody is processed in turn. If a randomly gener‐

é ù = - +- ê ú ë û

*NN N j j jj j Stimulation c m a xe k m a xp k m a y k* == =

represents the *jth* B cell object paratope and y represents

1 23

åå å (3)

(, ) (, ) (, ) ,

**Table 1** and **Table 2**, such as specificity, diversity and memory, among others.

models the tendency of cells to die if no interactions are present.

similar results. The performance was better than Nearest Neighbour Algorithm.

methods.

262 Recent Advances in Robotic Systems

protect the human body.

represents the *jth* B cell's epitope, *xpj*

*xej*

*Simple substitution*: Operator uses the roulette wheel [23] algorithm to select another B cell object from which elements will be substituted into the current B cell object.

The mutation procedure is an important procedure since it has been designed to introduce diversity into the system, as antibodies are created, the previous examples are just a few techniques that can be used during the mutation process, but there are several ways in which the mutation can be implemented.

The AIS systems is compared with other several machine learning approaches such as ANN, LCS, CBR and others showing that AIS offers certain advantages over some of them specifically the self-organising features and the unsupervised nature of the algorithm [22]. It also let us know that it can be noise tolerant and that the algorithms inherently generalises such a problem with ANN that can over fit. Also the self-organising feature makes it easier to handle rather than ANN that can be tedious and time consuming in order to tune them for a certain application checking the bias and variance of a certain configuration of the ANN.

The AIS is a paradigm that yet seems to have certain characteristics that attract engineers and scientist as well. It is a powerful example of a learning system that not only adapts rapidly but it also is a non-linear network, has a content addressable memory and it is self-organised. With this in mind, it is important to stop and analyse about the contributions that the paradigm has brought to different areas where it has been applied. CODA wants to use these features and apply them to the grasping problem in robotics, this is the main reason why the algorithm uses previous released machine learning tools plus it emulates the AIS in order to solve real problems and use AIS in important applications.

Precisely the study [24] explores how the AIS has been applied in "out of the lab" applications, where real world problems state new challenges, this question of how have the AIS has perform in the real world applications or industry leads us to the importance of focusing the efforts on developing theory that will serve as hard evidence of this paradigm.

Since according to the no-free lunch theorem [6], there is no one "do-it-all algorithm" that can outperform all the machine learning methods available so, it is important for any paradigm to contain features that may not be present in any of the previous techniques. In this manner the new algorithm can be described as a truly novel method.

Authors of [22] disagree with the definition proposed in [25] in which effectiveness is meas‐ ured in terms of how an algorithm performs better or not, compared to other in benchmark test. For example they measured the time it took to complete the task. The same work expos‐ es certain problems where AIS has done a magnificent job, these applications are presented in **Table 3**.


**Table 3.** Major and minor areas where AIS have been used.

**Table 3** reflects how the articles can be grouped according to the main areas where AIS has been applied. The studies in [3, 26–28] talk about Anomaly Detection, [29–31] expose works on optimisation, image processing [5], robot control [32–35] and web mining [36]. It is important to note that this table does not represent all publications related to AIS. It is a general picture based on information contained in ICARIS 2004 [37], ICARIS 2005 [38] and in the bibliography produced by de Castro [39].

Something important that must be noticed about most of the applications in several papers is that the AIS is tried in benchmark problems and robotic applications tend to be simulated rather than in real environments and almost all of them are small and simplified.

The article [22] concludes that there is a necessity to construct a robust framework that will allow the AIS to have a more biologically grounded theory, but it also remarks the need of multidisciplinary work between computer scientist, engineers, biologists and mathematicians.

Another important aspect discussed is that AIS algorithms are not so generic and most of them are quite specific to the task on duty. More generalised algorithms would be develop if the theoretical aspects of the AIS are improved in such a manner that generalisation would be easier. The authors close their work [22] with a phrase that is important for the development of the AIS algorithms:

*"However, all this futuristic discussion is interesting, but what is needed is well-grounded immune inspired techniques that are applied in a logical and coherent matter"*.

CODA algorithm pretends to handle data that could contain noise or that may be a small amount. Therefore the AIS should be an algorithm that can handle data in those circumstances, from the several bibliography revised it was found that in [40], the use of the AIS helps us in visualising how the AIS can be an unsupervised machine learning method but also how it can handle an analysis on the data and help visualise information.

The article describes the procedures that are internal to the AIS such as the primary and secondary response, how the antibody/antigen binding occurs, the B cell stimulation, the immune network among others that can serve as a really good introductory text for the reader. But it is not until the description of the B cell cloning were interest in such concept was really awaken.

The text is very explicit on how the AIS clones and mutates the B cells to build up the memory mechanism where all the information will be stored and will help on the recognition tasks where similar patterns must be identified. With all this data it is necessary for the memory to have an organising mechanism to form clusters that later will identify patterns.

**Major Minor**

Computer security Control Numeric function optimisation Robotics Combinatoric optimisation Virus detection Learning Web mining

**Table 3.** Major and minor areas where AIS have been used.

264 Recent Advances in Robotic Systems

bibliography produced by de Castro [39].

of the AIS algorithms:

awaken.

Clustering/classification Bio-informatics Anomaly detection Image processing

**Table 3** reflects how the articles can be grouped according to the main areas where AIS has been applied. The studies in [3, 26–28] talk about Anomaly Detection, [29–31] expose works on optimisation, image processing [5], robot control [32–35] and web mining [36]. It is important to note that this table does not represent all publications related to AIS. It is a general picture based on information contained in ICARIS 2004 [37], ICARIS 2005 [38] and in the

Something important that must be noticed about most of the applications in several papers is that the AIS is tried in benchmark problems and robotic applications tend to be simulated

The article [22] concludes that there is a necessity to construct a robust framework that will allow the AIS to have a more biologically grounded theory, but it also remarks the need of multidisciplinary work between computer scientist, engineers, biologists and mathematicians.

Another important aspect discussed is that AIS algorithms are not so generic and most of them are quite specific to the task on duty. More generalised algorithms would be develop if the theoretical aspects of the AIS are improved in such a manner that generalisation would be easier. The authors close their work [22] with a phrase that is important for the development

*"However, all this futuristic discussion is interesting, but what is needed is well-grounded*

CODA algorithm pretends to handle data that could contain noise or that may be a small amount. Therefore the AIS should be an algorithm that can handle data in those circumstances, from the several bibliography revised it was found that in [40], the use of the AIS helps us in visualising how the AIS can be an unsupervised machine learning method but also how it can

The article describes the procedures that are internal to the AIS such as the primary and secondary response, how the antibody/antigen binding occurs, the B cell stimulation, the immune network among others that can serve as a really good introductory text for the reader. But it is not until the description of the B cell cloning were interest in such concept was really

*immune inspired techniques that are applied in a logical and coherent matter"*.

handle an analysis on the data and help visualise information.

rather than in real environments and almost all of them are small and simplified.

As it is stated before in this document, there is a need for the AIS to be implemented in a simple yet logical manner, and the authors of [40] build up a really simple method to measure the affinity between the B cells with the Euclidean distance shown in Eq. (4).

$$\text{Affinity}(a, b) = \sqrt{\sum\_{n=1}^{\text{ND}} (a(n) - b(n))^2},\tag{4}$$

This implementation was tested with the "Fisher iris data set", which consist of 150 instances from three different classes, specifically plants. The attributes taken in consideration for the three classes are: sepal length, sepal width, petal length and petal width. The dataset cannot be separated linearly in two groups (Iris Virginica and Iris Versicolor) but it is possible to segregate the third group linearly corresponding to the Iris Setosa class. First principal component analysis was used to represent the data in a lower dimensional space in order to make possible a two dimensional graph. The AIS was able to recognise three classes inde‐ pendently from the distribution of the data set that may facilitate to classes and have more problems in the recognition of the third class.

The created network was capable of receiving unseen data and still able to recognise between classes, concluding that the networks were effective as a simple classification, one of the main objectives for using this AIS. The second valuable characteristic is that they seem to generalise in a wider region of the input space making it quite interesting since this feature could be from great importance in environments where little data can be collected, where the data could be corrupted with noise or demonstrations of the task are limited.

It has been seen all over the bibliography that a common point is the development of a more reliable theoretical point of view for the AIS, this was one of the first statements when approaching the AIS. The main reason the ABBAS book of Basic Immunology was revised was most of the documents found are a general overview of the NIS and difficult to understand if you have not had any previous experience. One interesting work is [40] and "Where" should be "where". Where the clonal selection mechanism was treated in a really interesting and straight forward manner. The clonal selection theory (CST) [41] is basically used to explain how the NIS response to any antigen stimulus. The main idea of the CST is that those cloned cells are in charge of recognising antigens.

In [20], a recognition region is presented but it is defined as spherical, a characteristic that could limit how the recognition ball could work. In [40] the author does not limit this region with a spherical geometry, leading to advancement in the theoretical framework for the AIS.

In [40] the B cell Algorithm (BCA) is defined as an iterative process that improves candidate solutions in a specific problem, with the use of tools such as cloning, mutation and selection. Again the Euclidean distance is used as a measure for affinity, which suggests it could be a standard for this process. But how the BCA can be differentiated from one of the most popular evolutionary algorithm such as genetic algorithms? The reader should notice that no cross over is employed in the cloning process, there is no necessity for this method to be applied in the BCA in order to increase diversity.

Specifically talking about AIS algorithms and more precisely about clonal selection ones, the authors consider it really important to make the population variation according to probabilistic rules and follow the nature of the model so this probabilities for the transitions to a new state depend only on the current state so the Markov chain could be satisfied. This also means that the algorithm should converge and find at least one global optimum solution with probability equal to one as *t* →*∞*. One of the very first papers that introduced this was [42] and should be revised for a detailed lecture on the theme.
