**1. Introduction**

22 Recurrent Neural Networks and Soft Computing

Wray, J., Green, G.G.R. (1994). Neural Networks, Approximation Theory and Finite

FRS (Fuzzy Rule Systems) and ANNs (Artificial Neural Networks) have gained much popularity due to their capabilities in modeling human knowledge and in learning from data, respectively. Fuzzy systems have the advantage of allowing users to incorporate available experts' knowledge directly in the fuzzy model [2]. Thus, decisions made by fuzzy systems are transparent to the user (i.e. the reasons behind the decisions made are clearly understood by tracing the decision and finding out which rules fired and contributed to it). However, there are many parameters whose values are arbitrary. These values have to be "guessed" by a fuzzy system designer, yet they largely influence the system behavior. Thus, a fuzzy system is as good as its programmer.

On the other hand, ANNs have the advantage of being universal functions approximators, requiring only sample data points and no expert knowledge [3]. Despite their advantages, they are essentially black box models. This means that the reasons behind their decisions are concealed in the knowledge acquired in the trained weights. However, these usually have no clear logical interpretation. Thus, their reliability is questionable.

To combine the advantages of both systems while overcoming their disadvantages, two approaches have been proposed in literature. The first is rule extraction from weights of trained ANNs [4]-[7]. However, the proposed approaches often yield some "un-plausible" rules, thus rule pruning and retraining is often required. For examples, some rules may be impossible i.e. their firing depends on conditions that can never occur in reality (*impossible antecedents*). For example, a rule dictating that a certain action is to be taken in case time is negative. The second approach is ANFIS [8], which attempts to cast the fuzzy system as an ANN with five layers. Although, only two of these layers are adaptable, this model is still more complicated to build and train than a conventional feed-forward ANN for two main reasons. First, the user's expertise is required to choose appropriate consequent (output) membership functions. Second, the desired output needs to be known a priori. This may not be possible for several applications including design problems, inverse problems and high dimensional problems. For example, in a robot path tracking problem, the ANN is required to predict the correct control input. In such application, the desired performance is known but no real-solid rules exist, especially, if the robot is required to be self-adaptive. Similarly,

A Framework for Bridging the Gap Between Symbolic and Non-Symbolic AI 25

It has been proved in [1] that the sigmoid response to a sum of inputs is equivalent to combining the sigmoid response to each input using the fuzzy logic operator "ior" (interactive or). The truth table of the "ior" operator is shown in table 1. The truth table can be readily generalized to an arbitrary number of inputs. Eq.(1) can, thus, be interpreted as the output of a fuzzy inference system, where the weight *wjk* is the action recommended by fuzzy rule *j*. However, this *wjk* does not contribute directly to the ANN output. Instead, its

The sigmoid term corresponds to the degree of firing of the rule, which judges to what extent rule 'j' should participate in the ANN final decision. Moreover, the inference is based on 'ior' rather than the product/'and' fuzzy operator used in ANFIS. It is clear from the 'ior' truth table that the 'ior' operator decides that a rule fully participate in the ANN final decision if all its inputs satisfy their corresponding constraints or if some of them does, while the others are neutral. On the other hand, it decides that the rule should not participate if one or some of the inputs do not satisfy their constraints, while the others are neutral. In the case that some of the inputs completely satisfy the constraints; while others completely violate them, the rule becomes neutral participating by half-weighted recommended action in the final ANN output. The mathematical expression for "ior" is as

1

*i= sig w x + b* .

*p hij j i*

(2)

*Ni*

contribution to the output is weighted by the sigmoid term

1

of rules taking the following format:

1 2

............

*ior(a ,a , ,a ) =*

*p*

overcome these disadvantages, our approach is to modify Eq.(1) as follows:

1 1

*2 n*

1 2 2

 / / 1,2,...

*SLANCV x > b N w then o = w i = ,N*

Despite the successful deployment of the "ior" based rule extraction in several applications ([1], [6] and [7]), it has several disadvantages. For example, the weights and biases of a hidden neuron have no direct clear logical interpretation. This makes the incorporation of available knowledge difficult. Such knowledge is of great use in accelerating the ANN training procedure. Besides, leaving weights and biases values unconstrained often lead to some un-plausible rules (rules with impossible antecedent) that need pruning. Therefore, to

> *N N h i c cc <sup>p</sup> k jk hij ij i j= i=*

 

(3)

*o = w sig w (x b )*

. ................. 1 . 1 ........... 1 . ............

In linguistic terms, an antecedent formed by "ior-ing" several conditions, is equivalent to replacing the conventional phrase: "*if* A & B & --- then" with "So long as none of the conditions A, B, … are violated --- then". Throughout the paper we will use the Mnemonic "SLANCV" as a shortcut for this phrase. Thus we can say that Eq. 1 can be restated as a set

*aa a ( a )( a ) ( a )+a a a*

*n*

*i j i hij jk jk i*

*n1 n*

follows [1]:

consider a car shape optimization problem. The ANN is required to estimate the shape parameters required to achieve certain air resistance during car motion. Constraints on the shape parameters exist; however, no clear rule database exists relating shape parameters to the desired performance.

The present work proposes a new approach that combines the advantages of fuzzy systems and ANNs through a simple modification of ANN's activation calculations. The proposed approach yields weights that are readily interpretable as logical consistent fuzzy rules because it includes the "semantic" of both input and output variables in the learning/optimization process.

The rest of the chapter is organized as follows. Section II describes the proposed framework. Section III demonstrates its effectiveness through a case study. Section IV shows how it to can be generalized to solve optimization problems. An illustrative example is given for this purpose. Finally, Section V concludes the chapter with a summary of the advantages of the proposed approach.
