**2. Time-delay recurrent neural network (TDRNN)**

Figure 1 depicts the proposed time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism. In the figure, <sup>1</sup> *Z* denotes a one-step time delay, the notation "" represents the memory neurons in the input layer with selffeedback gain (0 1) , which improves the resolution ratio of the inputs.

Fig. 1. Architecture of the modified Elman network

It is a type of recurrent neural network with different layers of neurons, namely: input nodes, hidden nodes, output nodes and, specific of the approach, context nodes. The input and output nodes interact with the outside environment, whereas the hidden and context nodes do not. The context nodes are used only to memorize previous activations of the output nodes. The feed-forward connections are modifiable, whereas the recurrent connections are fixed. More specifically, the proposed TDRNN possesses self-feedback links with fixed coefficient in the context nodes. Thus the output of the context nodes can be described by

$$y\_{Cl}(k) = ay\_{Cl}(k-1) + y\_{l}(k-1) \quad \text{ ( $l=1,2,\cdots,m$ )}\,. \tag{1}$$

where ( ) *Cl y k* and ( ) *<sup>l</sup> y k* are, respectively, the outputs of the *l*th context unit and the *l*th output unit and (0 1 ) is the self-feedback coefficient. If we assume that there are *r* nodes in the input layer, *n* nodes in the hidden layer, and *m* nodes in the output layer and context layers respectively, then the input *u* is an *r* dimensional vector, the output *x* of the hidden layer is *n* dimensional vector, the output *y* of the output layer and the output *Cy* of

the context nodes are *m* dimensional vectors, and the weights *W* <sup>1</sup> , *W* <sup>2</sup> and *W* 3 are *n r*, *m n* and *m m* dimensional matrices, respectively.

The mathematical model of the proposed TDRNN can be described as follows.

$$\mathbf{y}(k) = \mathbf{g}(\mathcal{W}^2 \mathbf{x}(k) + \mathcal{W}^3 \mathbf{y}\_{\mathcal{C}}(k)) \,. \tag{2}$$

( ) ( 1) ( 1) *C C y k y k yk* , (3)

$$\mathbf{x}(k) = f(\mathcal{W}^1 \mathbf{z}(k)) \,. \tag{4}$$

$$\pi z(k) = \mu(k) + \beta \sum\_{i=1}^{\tau} \mu(k - i) + \gamma z(k - 1) \,. \tag{5}$$

where 0 , , 1, 1, *z*(0) 0 , and is the step number of time delay. *f* ( ) *x* is often taken as the sigmoidal function

<sup>1</sup> ( ) <sup>1</sup> *<sup>x</sup> f x e* . (6)

and *g x*( ) is often taken as a linear function, that is

116 Recurrent Neural Networks and Soft Computing

identifying and controlling dynamic systems. The proposed identification and control schemes are examined by the numerical experiments for identifying and controlling some

The rest of this chapter is organized as follows. Section 2 proposes a novel time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism; moreover, a dynamic recurrent back propagation algorithm is developed according to the gradient descent method. Section 3 derives the optimal adaptive learning rates to guarantee the global convergence in the sense of discrete-type Lyapunov stability. Thereafter, the proposed identification and control schemes based on TDRNN models are examined by numerical experiments in Section 4. Finally, some conclusions are made in

Figure 1 depicts the proposed time-delay recurrent neural network (TDRNN) by introducing the time-delay and recurrent mechanism. In the figure, <sup>1</sup> *Z* denotes a one-step time delay, the notation "" represents the memory neurons in the input layer with self-

<sup>1</sup> *W*

, which improves the resolution ratio of the inputs.

*kz* )( *kx* )(

<sup>2</sup> *W*

It is a type of recurrent neural network with different layers of neurons, namely: input nodes, hidden nodes, output nodes and, specific of the approach, context nodes. The input and output nodes interact with the outside environment, whereas the hidden and context nodes do not. The context nodes are used only to memorize previous activations of the output nodes. The feed-forward connections are modifiable, whereas the recurrent connections are fixed. More specifically, the proposed TDRNN possesses self-feedback links

<sup>3</sup> *W*

1

1

*ky* )(

typical nonlinear systems.

**2. Time-delay recurrent neural network (TDRNN)** 

Z-1

Z-1

Z-1 

Z-1 

Fig. 1. Architecture of the modified Elman network

Section 5.

feedback gain

*ku* )(

 (0 1) 

$$\mathbf{y}(k) = \mathcal{W}^2 \mathbf{x}(k) + \mathcal{W}^3 y\_\mathbb{C}(k) \,. \tag{7}$$

Taking expansion for *z k*( 1) , *z k*( 2) ,…, *z*(1) by using Eq.(5), then we have

$$\pi z(k) = \sum\_{i=0}^{\tau} u(k-i) + \sum\_{i=1}^{k-\tau} \gamma^i u(k-\tau-i) - \gamma^k u(0) \,. \tag{8}$$

From Eq.(8) it can be seen that the memory neurons in the input layer include all the previous input information and the context nodes memorize previous activations of the output nodes, so the proposed TDRNN model has far higher memory depth than the popular neural networks. Furthermore, the neurons in the input layer can memory accurately the inputs from time *k* to time *k* , and this is quite different from the memory performance of popular recurrent neural networks. If the delay step is moderate large, the TDRNN possesses higher memory resolution ratio.

Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 119

2

2 2

*i i*

1 1

. (19)

2

. (20)

(21)

. (22)

. (18)

1 1

*jq jq*

<sup>2</sup> <sup>2</sup>

<sup>2</sup> <sup>2</sup>

2 2

*i i*

*W W*

*W W*

. (15)

. (16)

, () () () *i di i e k y k y k* . (17)

max ( ) max( ( )) *k ij k ij*

*nr z k W k*

2 1 <sup>1</sup> () () <sup>2</sup> *m i i Ek e k* 

1 <sup>1</sup> ( ) ( 1) ( ) ( 1) ( ) <sup>2</sup> *m*

*i Ek Ek Ek e k e k*

1 1 1 1

*n r n r <sup>i</sup> <sup>i</sup> i i jq i jq*

*j q jq j q jq*

1 1 1 1 ( ) ( ) () ()() ()() *<sup>i</sup> <sup>i</sup>*

 

1 1 1

1 1

1 1

*i*

*W*

*i i*

 

1 1 1 1

*y k y k k k W W*

<sup>1</sup> ( ) ( ) () 2 () <sup>2</sup>

*W*

*W W*

( ) ( ) ( 1) ( ) ( )

Furthermore, the modification of weights associated with the input and hidden layers is

*jq i i*

*m T*

*yk yk Ek e k k*

*e k y k ek ek W ek <sup>W</sup>*

*e k <sup>y</sup> <sup>k</sup> W k ke k ke k*

<sup>1</sup> () () () () 1 () <sup>1</sup>

<sup>1</sup> ( ) () 1 () <sup>1</sup>

*y k ek k*

*<sup>m</sup> <sup>i</sup>*

<sup>1</sup> ( ) () 1 1 () <sup>2</sup>

 

 

*y k k k*

1

Then the error during the learning process can be expressed as

1

Hence, from Eqs.(18-20) we obtain

2

*i i*

2

*i i m*

2 1

*ek k*

*i i*

() ()

1

2

2

1

*i*

1

1

*i*

**Proof.** Define the Lyapunov energy function as follows.

Where

Where

 

*k*

<sup>8</sup> 0 ()

And consequently, we can obtain the modification of the Lyapunov energy function

Let the *k*th desired output of the system be ( ) *<sup>d</sup> y k* . We can then define the error as

$$E = \frac{1}{2} (y\_d(k) - y(k))^T (y\_d(k) - y(k))\,. \tag{9}$$

Differentiating *E* with respect to *W* <sup>3</sup> , *W* 2 and *W* 1 respectively, according to the gradient descent method, we obtain the following equations

$$
\Delta w\_{il}^{\mathcal{S}} = \eta\_{\mathcal{S}} \delta\_i^0 y\_{\mathcal{C},l}(k) \qquad \text{ (i = 1, 2, \dots, m; l = 1, 2, \dots, m\text{)}, \tag{10}
$$

$$
\Delta w\_{ij}^2 = \eta\_2 \delta\_i^0 \{ \mathbf{x}\_j(k) + w\_{ii}^3 \frac{\partial y\_{\subset,i}(k)}{\partial w\_{ij}^2} \} \quad \text{(\$i = 1, 2, \dots, m\$; j = 1, 2, \dots, n\$)}\tag{11}
$$

$$
\Delta w\_{j\eta}^1 = \eta\_1 \sum\_{t=1}^n \delta\_t^0 w\_{\dagger j}^2 f\_j'(\cdot) z\_q(k) \qquad \text{( $j = 1, 2, \dots, n$ ;  $\eta = 1, 2, \dots, r$ )}\tag{12}
$$

which form the learning algorithm for the proposed TDRNN, where 1 , 2 and 3 are learning steps of *W* <sup>1</sup> , *W* <sup>2</sup> and *W* <sup>3</sup> , respectively, and

$$
\delta\_i^0 = (y\_{d,i}(k) - y\_i(k))\mathbf{g}\_i'(\cdot) \,. \tag{13}
$$

$$\frac{\partial y\_{\mathbb{C},i}(k)}{\partial w\_{\vec{\imath}\vec{\jmath}}^2} = \alpha \frac{\partial y\_{\mathbb{C},i}(k-1)}{\partial w\_{\vec{\imath}\vec{\jmath}}^2} + \frac{\partial y\_i(k-1)}{\partial w\_{\vec{\imath}\vec{\jmath}}^2} \,. \tag{14}$$

If *g x*( ) is taken as a linear function, then () 1 *<sup>i</sup> g* . Clearly, Eqs. (11) and (14) possess recurrent characteristics.

#### **3. Convergence of proposed time-delay recurrent neural network**

In Section 2, we have proposed a TDRNN model and derived its dynamic recurrent back propagation algorithm according to the gradient descent method. But the learning rates in the update rules have a direct effect on the stability of dynamic systems. More specifically, a large learning rate can make the modification of weights over large in each update step, and this will induce non-stability and non-convergence. On the other hand, a small learning rate will induce a lower learning efficiency. In order to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis. The following theorems give sufficient conditions for the convergence of the proposed TDRNN when the functions *f* ( ) and *g*( ) in Eqs. (4) and (2) are taken as sigmoidal function and linear function respectively.

Suppose that the modification of the weights of the TDRNN is determined by Eqs. (10-14). For the convergence of the TDRNN we have the following theorems.

**Theorem 1.** The stable convergence of the update rule (12) on *W* 1 is guaranteed if the learning rate 1 ( ) *k* satisfies that

$$0 < \eta\_1(k) < \frac{8}{nr \left| \max\_k z\_k(k) \right| \left| \max\_{ij} (\mathcal{W}^2\_{ij}(k)) \right|} \tag{15}$$

**Proof.** Define the Lyapunov energy function as follows.

$$E(k) = \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \,. \tag{16}$$

Where

118 Recurrent Neural Networks and Soft Computing

<sup>1</sup> ( ( ) ( )) ( ( ) ( ))

Differentiating *E* with respect to *W* <sup>3</sup> , *W* 2 and *W* 1 respectively, according to the gradient

3 , ( ) ( 1,2, , ; 1,2, , ) *w y k i ml m il i C l*

*w wf z k j nq r*

, ( ( ) ( )) ( ) *i di i i*

2 22 *C i*( ) ( 1) *C i* ( 1) *<sup>i</sup> ij ij ij*

If *g x*( ) is taken as a linear function, then () 1 *<sup>i</sup> g* . Clearly, Eqs. (11) and (14) possess

In Section 2, we have proposed a TDRNN model and derived its dynamic recurrent back propagation algorithm according to the gradient descent method. But the learning rates in the update rules have a direct effect on the stability of dynamic systems. More specifically, a large learning rate can make the modification of weights over large in each update step, and this will induce non-stability and non-convergence. On the other hand, a small learning rate will induce a lower learning efficiency. In order to train neural networks more efficiently, we propose three criterions of selecting proper learning rates for the dynamic recurrent back propagation algorithm based on the discrete-type Lyapunov stability analysis. The following theorems give sufficient conditions for the convergence of the proposed TDRNN when the functions *f* ( ) and

*yk yk y k w ww*

 

*ij y k w xk w i m <sup>j</sup> <sup>n</sup>*

*w*

which form the learning algorithm for the proposed TDRNN, where

0

, ,

**3. Convergence of proposed time-delay recurrent neural network** 

*g*( ) in Eqs. (4) and (2) are taken as sigmoidal function and linear function respectively.

For the convergence of the TDRNN we have the following theorems.

Suppose that the modification of the weights of the TDRNN is determined by Eqs. (10-14).

**Theorem 1.** The stable convergence of the update rule (12) on *W* 1 is guaranteed if the

( ) ( () ) ( 1,2, , ; 1,2, , ) *C i*

, (11)

( ) ( ) ( 1,2, , ; 1,2, , )

. (12)

*<sup>T</sup> E y k yk y k yk d d* . (9)

, (10)

*y k y k g* , (13)

1 , 2 and 3 are

. (14)

Let the *k*th desired output of the system be ( ) *<sup>d</sup> y k* . We can then define the error as

2

descent method, we obtain the following equations

   

*ij i j ii*

1 02 1 1

 

learning steps of *W* <sup>1</sup> , *W* <sup>2</sup> and *W* <sup>3</sup> , respectively, and

*n jq t tj j q t*

20 3 , 2 2

3 0

recurrent characteristics.

learning rate 1

( ) *k* satisfies that

$$e\_i(k) = y\_{d,i}(k) - y\_i(k) \,. \tag{17}$$

And consequently, we can obtain the modification of the Lyapunov energy function

$$
\Delta E(k) = E(k+1) - E(k) = \frac{1}{2} \sum\_{i=1}^{m} \left[ e\_i^2 \left( k + 1 \right) - e\_i^2 \left( k \right) \right]. \tag{18}
$$

Then the error during the learning process can be expressed as

$$e\_i(k+1) = e\_i(k) + \sum\_{j=1}^n \sum\_{q=1}^r \frac{\partial e\_i(k)}{\partial \mathcal{W}^1\_{jq}} \Delta \mathcal{W}^1\_{jq} = e\_i(k) - \sum\_{j=1}^n \sum\_{q=1}^r \frac{\partial y\_i(k)}{\partial \mathcal{W}^1\_{jq}} \Delta \mathcal{W}^1\_{jq} \tag{19}$$

Furthermore, the modification of weights associated with the input and hidden layers is

$$
\Delta \mathcal{W}\_{jq}^{1}(k) = \eta\_1(k) e\_i(k) \frac{\partial e\_i(k)}{\partial \mathcal{W}\_{jq}^{1}} = -\eta\_1(k) e\_i(k) \frac{\partial \mathcal{y}\_i(k)}{\partial \mathcal{W}\_{jq}^{1}} \,. \tag{20}
$$

Hence, from Eqs.(18-20) we obtain

$$\begin{split} \Delta E(k) &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_1(k) \left[ \frac{\partial y\_i(k)}{\partial \mathcal{W}^1} \right]^T \left[ \frac{\partial y\_i(k)}{\partial \mathcal{W}^1} \right] \right)^2 - 1 \right] \\ &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_1(k) \left\| \frac{\partial y\_i(k)}{\partial \mathcal{W}^1} \right\|^2 \right)^2 - 1 \right] \\ &= - \sum\_{i=1}^{m} e\_i^2(k) \mathcal{J}\_i^1(k) \end{split} \tag{21}$$

Where

$$\begin{split} \mathcal{J}\_{i}^{1}(k) &= \frac{1}{2} \Bigg[ 1 - \Big( 1 - \eta\_{1}(k) \left\| \frac{\partial \mathcal{y}\_{i}(k)}{\partial \mathcal{V}^{1}} \right\|^{2} \Bigg)^{2} \Bigg] \\ &= \frac{1}{2} \eta\_{1}(k) \left\| \frac{\partial \mathcal{y}\_{i}(k)}{\partial \mathcal{V}^{1}} \right\|^{2} \Bigg( 2 - \eta\_{1}(k) \left\| \frac{\partial \mathcal{y}\_{i}(k)}{\partial \mathcal{V}^{1}} \right\|^{2} \Bigg) \end{split} \tag{22}$$

Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 121

<sup>1</sup> ( ) () 1 1 () <sup>2</sup>

 

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, and neglect the dependence relation between ( ) *Cy k* and the weights <sup>2</sup> *wij* , we obtain

> 0 <sup>2</sup> ( ) *i j*

*<sup>E</sup> x k*

( ) ( ) 1 ( 1,2, , ; 1,2, , ) *<sup>i</sup>*

*y k x k i mj n <sup>W</sup>*

2 ( ) *<sup>i</sup> i y k <sup>n</sup> W* 

() 0 *<sup>i</sup>* 

According to the Lyapunov stability theory, this shows that the training error will converges

**Theorem 3.** The stable convergence of the update rule (10) on *W* 3 is guaranteed if the

3 2

*m yk*

3 3

*W*

<sup>1</sup> () () () () 1 () <sup>1</sup>

*<sup>T</sup> <sup>m</sup> i i*

*yk yk Ek e k k*

<sup>1</sup> ( ) () 1 () <sup>1</sup>

*y k ek k*

*<sup>m</sup> <sup>i</sup>*

<sup>2</sup> 0 ()

*k*

 

2

*i i*

2

*i i m*

2 3

*ek k*

*i i*

() ()

1

2

2

1

1

*i*

,

3 3 3

<sup>2</sup> <sup>2</sup>

*W W*

max( ( )) *C l <sup>l</sup>*

*ij*

*w*

*y k k k*

2

*i*

2

According to the definition of the Euclidean norm we have

*n*

, we have <sup>2</sup>

*ij*

<sup>2</sup> 0 () *<sup>k</sup>*

to zero as *t* . This completes the proof.

( ) *k* satisfies that

**Proof.** Similarly, as the above proof, we have

 *j*

<sup>2</sup> <sup>2</sup>

. (29)

. (31)

*k* , then from Eq.(27) we obtain *E k*() 0 .

. (32)

2

. (33)

. (28)

2 2

*i*

*W*

*i*

. (30)

Where

Hence,

Therefore, while <sup>2</sup>

learning rate 3

*W* 1 represents an *n r* dimensional vector and denotes the Euclidean norm.

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, we have 0 ( ) 1/4 *f x* . Thus,

$$\begin{aligned} \left| \frac{\partial y\_j(k)}{\partial \mathcal{W}^1\_{jq}} \right| = \left| \mathcal{W}^2\_{ij}(k) f\_j^\cdot(\cdot) z\_q(k) \right| &\le \frac{1}{4} \left| \max\_q z\_q(k) \right| \left| \max\_{ij} \{ \mathcal{W}^2\_{ij}(k) \} \right| \\\ (i = 1, 2, \dots, m; j = 1, 2, \dots, n; q = 1, 2, \dots, r) \end{aligned} \tag{23}$$

According to the definition of the Euclidean norm we have

$$\left\|\frac{\partial y(k)}{\partial W^1}\right\| \le \sqrt{\frac{nr}{4} \left| \max\_q z\_q(k) \right| \left| \max\_{ij} (\mathcal{W}^2\_{ij}(k)) \right|}\right. \tag{24}$$

Therefore, while <sup>1</sup> 2 <sup>8</sup> 0 () max ( ) max( ( )) *q ij q ij k nr z k W k* , we have <sup>1</sup> () 0 *<sup>i</sup> k* , then from

Eq.(21) we obtain *E k*() 0 . According to the Lyapunov stability theory, this shows that the training error will converges to zero as *t* . This completes the proof.

**Theorem 2.** The stable convergence of the update rule (11) on *W* 2 is guaranteed if the learning rate 2 ( ) *k* satisfies that

$$0 < \eta\_2(k) < \frac{2}{n}.\tag{25}$$

**Proof.** Similarly, the error during the learning process can be expressed as

$$e\_i(k+1) = e\_i(k) + \sum\_{j=1}^n \frac{\partial e\_i(k)}{\partial \mathcal{W}\_{ij}^2} \Delta \mathcal{W}\_{ij}^2 = e\_i(k) - \sum\_{j=1}^n \frac{\partial y\_i(k)}{\partial \mathcal{W}\_{ij}^2} \Delta \mathcal{W}\_{ij}^2 \,. \tag{26}$$

Therefore,

$$\begin{split} \Delta E(k) &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_2(k) \left[ \frac{\partial y\_i(k)}{\partial \mathcal{W}\_i^2} \right]^\top \left[ \frac{\partial y\_i(k)}{\partial \mathcal{W}\_i^2} \right] \right)^2 - 1 \right] \\ &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_2(k) \left\| \frac{\partial y\_i(k)}{\partial \mathcal{W}\_i^2} \right\|^2 \right)^2 - 1 \right] \\ &= - \sum\_{i=1}^{m} e\_i^2(k) \mathcal{H}\_i^2(k) \end{split} \tag{27}$$

Where

120 Recurrent Neural Networks and Soft Computing

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal

2 ' 2

2

*<sup>W</sup>* . (24)

2 2

. (26)

2

2 2

2 2 2

<sup>2</sup> <sup>2</sup>

*W W*

. (23)

() 0 *<sup>i</sup>* 

. (25)

*k* , then from

. (27)

( ) <sup>1</sup> ( ) ( ) ( ) max ( ) max( ( )) <sup>4</sup>

( 1,2, , ; 1,2, , ; 1,2, , )

( ) max ( ) max( ( )) <sup>4</sup> *q ij q ij y k nr zk W k*

, we have <sup>1</sup>

Eq.(21) we obtain *E k*() 0 . According to the Lyapunov stability theory, this shows that

**Theorem 2.** The stable convergence of the update rule (11) on *W* 2 is guaranteed if the

2 <sup>2</sup> 0 () *<sup>k</sup>*

( ) ( ) ( 1) ( ) ( )

*e k y k ek ek W ek <sup>W</sup>*

*ii ij i ij*

<sup>1</sup> () () () () 1 () <sup>1</sup>

*i i i*

*<sup>T</sup> <sup>m</sup> i i*

*yk yk Ek e k k*

**Proof.** Similarly, the error during the learning process can be expressed as

2

*i*

2

*i*

2 2

*ek k*

*i i*

() ()

1

2

2

1

1

*i*

*m*

max ( ) max( ( )) *q ij q ij*

*nr z k W k*

the training error will converges to zero as *t* . This completes the proof.

2

*n*

1 1

2 2

*W*

<sup>1</sup> ( ) () 1 () <sup>1</sup>

*y k ek k*

*<sup>m</sup> <sup>i</sup>*

*i i*

*n n <sup>i</sup> <sup>i</sup>*

*j j ij ij*

*W W*

*i mj nq r*

*ij j q <sup>q</sup> ij <sup>q</sup> ij jq y k W kf z k z k W k <sup>W</sup>*

*W* 1 represents an *n r* dimensional vector and denotes the Euclidean norm.

type, we have 0 ( ) 1/4 *f x* . Thus,

Therefore, while <sup>1</sup>

learning rate 2

Therefore,

1

According to the definition of the Euclidean norm we have

1

<sup>8</sup> 0 ()

*k*

 

( ) *k* satisfies that

*i*

$$\beta\_i^2(k) = \frac{1}{2} \left[ 1 - \left( 1 - \eta\_2(k) \left\| \frac{\partial \boldsymbol{y}\_i(k)}{\partial \boldsymbol{W}\_i^2} \right\|^2 \right)^2 \right]. \tag{28}$$

Notice that the activation function of the hidden neurons in the TDRNN is the sigmoidal type, and neglect the dependence relation between ( ) *Cy k* and the weights <sup>2</sup> *wij* , we obtain

$$\frac{\partial E}{\partial w\_{ij}^{2}} = -\delta\_{i}^{0} \mathbf{x}\_{j}(k) \,. \tag{29}$$

Hence,

$$\left|\frac{\partial y\_i(k)}{\partial \mathcal{W}\_{ij}^2}\right| = \left|x\_j(k)\right| < 1 \qquad \text{(\$i = 1, 2, \dots, m\$; j = 1, 2, \dots, n\$)}\tag{30}$$

According to the definition of the Euclidean norm we have

$$\left\| \frac{\partial \mathcal{y}\_i(k)}{\partial \mathcal{W}\_i^2} \right\| < \sqrt{n} \text{ .} \tag{31}$$

Therefore, while <sup>2</sup> <sup>2</sup> 0 () *<sup>k</sup> n* , we have <sup>2</sup> () 0 *<sup>i</sup> k* , then from Eq.(27) we obtain *E k*() 0 . According to the Lyapunov stability theory, this shows that the training error will converges to zero as *t* . This completes the proof.

**Theorem 3.** The stable convergence of the update rule (10) on *W* 3 is guaranteed if the learning rate 3 ( ) *k* satisfies that

$$0 < \eta\_3(k) < \frac{2}{m \left| \max\_l (y\_{\subset,l}(k)) \right|^2}. \tag{32}$$

**Proof.** Similarly, as the above proof, we have

$$\begin{split} \Delta E(k) &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_3(k) \left[ \frac{\partial \boldsymbol{y}\_i(k)}{\partial \boldsymbol{\mathcal{W}}^3} \right]^T \left[ \frac{\partial \boldsymbol{y}\_i(k)}{\partial \boldsymbol{\mathcal{W}}^3} \right] \right)^2 - 1 \right] \\ &= \frac{1}{2} \sum\_{i=1}^{m} e\_i^2(k) \left[ \left( 1 - \eta\_3(k) \left\| \frac{\partial \boldsymbol{y}\_i(k)}{\partial \boldsymbol{\mathcal{W}}^3} \right\|^2 \right)^2 - 1 \right] \\ &= - \sum\_{i=1}^{m} e\_i^2(k) \boldsymbol{\beta}\_i^3(k) \end{split} \tag{33}$$

Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 123

recurrent network proposed by Elman [12]. Some parameters on the TDRNN in our experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are

Figure 2 shows the identification result, where the "Actual curve" is the real output curve of the dynamic system, represented by the solid line; the "Elman curve" is the output curve identified using the ENN model, and represented by the dash line; the "TDRNN curve" is the output curve identified by the proposed TDRNN model, and represented by the dash dot line. Figure 3 shows the identification error curves obtained with the TDRNN and ENN respectively, in which the error is the absolute value of the difference between identification result and the actual output. From the two figures it can be seen that the proposed method is superior to the ENN method. These results demonstrate the power and potential of the

0 20 40 60 80 100

Actual curve Elman curve TDRNN curve

*k*

 , and 

are set as 0.4, 0.6, 0.4

initialized in the interval [-2, 2] randomly, besides,

proposed TDRNN model for identifying nonlinear systems.


Fig. 2. Identification curves with different methods

y( *k*)

respectively. The number of hidden nodes in the ENN is also taken as 6.

Where

$$\mathcal{J}\_i^3(k) = \frac{1}{2} \left| 1 - \left( 1 - \eta\_3(k) \left\| \frac{\partial \mathcal{y}\_i(k)}{\partial \mathcal{W}^3} \right\|^2 \right)^2 \right|. \tag{34}$$

Furthermore, according to the learning algorithm we have

$$\left|\frac{\partial y\_s(k)}{\partial \mathcal{W}\_{il}^{\mathcal{S}}}\right| = \left|\mathcal{S}\_{is} y\_{\mathcal{C},h}(k)\right| = \mathcal{S}\_{is} \left|y\_{\mathcal{C},l}(k)\right| \le \mathcal{S}\_{is} \left|\max\_{l} (y\_{\mathcal{C},l}(k))\right|.\tag{35}$$
 
$$(i = 1, 2, \cdots, m; s = 1, 2, \cdots, m; l = 1, 2, \cdots, m)$$

Where

$$
\mathcal{S}\_{is} = \begin{cases} 1 & i = s \\ 0 & i \neq s \end{cases} \tag{36}
$$

According to the definition of the Euclidean norm we have

$$\left\|\frac{\partial y(k)}{\partial \mathcal{W}^3}\right\| \le \sqrt{m} \left|\max\_{l} (y\_{\mathcal{C},l}(k))\right|.\tag{37}$$

Therefore, from Eq.(34), we have <sup>3</sup> () 0 *<sup>i</sup> k* , while <sup>3</sup> <sup>2</sup> , <sup>2</sup> 0 () max( ( )) *C l <sup>l</sup> k m yk* . Then from

Eq.(33) we obtain *E k*() 0 . According to the Lyapunov stability theory, this shows that the training error will converges to zero as *t* . This completes the proof.

#### **4. Numerical results and discussion**

The performance of the proposed time-delay recurrent neural network for identifying and controlling dynamic systems is examined by some typical test problems. We provide four examples to illustrate the effectiveness of the proposed model and algorithm.

#### **4.1 Nonlinear time-varying system identification**

We have carried out the identification for the following nonlinear time-varying system using the TDRNN model as an identifier.

$$y(k+1) = \frac{y(k)}{1 + 0.68\sin(0.0005\pi k)y^2(k)} + 0.78u^3(k) + v(k)\,. \tag{38}$$

Where *v k*( ) is Gauss white noise with zero mean and constant variance 0.1. The input of system is taken as *u k*( ) sin(0.01 ) *k* .

To evaluate the performance of the proposed algorithm, the numerical results are compared with those obtained by using Elman neural network (ENN). The Elman network is a typical

<sup>1</sup> ( ) () 1 1 () <sup>2</sup>

 

3 ,, , ( ) ( ) ( ) max( ( ))

*<sup>s</sup> is C h is C l is C l <sup>l</sup> il y k <sup>y</sup> <sup>k</sup> <sup>y</sup> <sup>k</sup> <sup>y</sup> <sup>k</sup>*

 

() 0 *<sup>i</sup>* 

*W* 

training error will converges to zero as *t* . This completes the proof.

examples to illustrate the effectiveness of the proposed model and algorithm.

( 1,2, , ; 1,2, , ; 1,2, , )

1 0 *is*

3 , ( ) max( ( )) *C l <sup>l</sup> y k <sup>m</sup> <sup>y</sup> <sup>k</sup>*

Eq.(33) we obtain *E k*() 0 . According to the Lyapunov stability theory, this shows that the

The performance of the proposed time-delay recurrent neural network for identifying and controlling dynamic systems is examined by some typical test problems. We provide four

We have carried out the identification for the following nonlinear time-varying system

( ) ( 1) 0.78 ( ) ( ) 1 0.68sin(0.0005 ) ( ) *y k <sup>y</sup> k u <sup>k</sup> <sup>v</sup> <sup>k</sup> ky k*

Where *v k*( ) is Gauss white noise with zero mean and constant variance 0.1. The input of

To evaluate the performance of the proposed algorithm, the numerical results are compared with those obtained by using Elman neural network (ENN). The Elman network is a typical

2

*i s i s*

*i m s ml m*

*y k k k*

3 3

*i*

*W*

 

3

*i*

Furthermore, according to the learning algorithm we have

According to the definition of the Euclidean norm we have

*W*

Therefore, from Eq.(34), we have <sup>3</sup>

**4. Numerical results and discussion** 

using the TDRNN model as an identifier.

system is taken as *u k*( ) sin(0.01 )

**4.1 Nonlinear time-varying system identification** 

*k* . <sup>2</sup> <sup>2</sup>

. (34)

. (35)

. (36)

,

. Then from

. (38)

max( ( )) *C l <sup>l</sup>*

*m yk*

. (37)

<sup>2</sup> 0 ()

3

*k* , while <sup>3</sup> <sup>2</sup>

 

*k*

Where

Where

recurrent network proposed by Elman [12]. Some parameters on the TDRNN in our experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides, , and are set as 0.4, 0.6, 0.4 respectively. The number of hidden nodes in the ENN is also taken as 6.

Figure 2 shows the identification result, where the "Actual curve" is the real output curve of the dynamic system, represented by the solid line; the "Elman curve" is the output curve identified using the ENN model, and represented by the dash line; the "TDRNN curve" is the output curve identified by the proposed TDRNN model, and represented by the dash dot line. Figure 3 shows the identification error curves obtained with the TDRNN and ENN respectively, in which the error is the absolute value of the difference between identification result and the actual output. From the two figures it can be seen that the proposed method is superior to the ENN method. These results demonstrate the power and potential of the proposed TDRNN model for identifying nonlinear systems.

Fig. 2. Identification curves with different methods

Recurrent Neural Network-Based Adaptive Controller Design for Nonlinear Dynamical Systems 125

0 2 4 6 810

Reference Curve Control Curve

*t*

Reference Curve Control Curve

0 5 10 15 20 25 30

*t*

The inverted pendulum system is one of the classical examples used in many experiments dealing with classical as well as modern control, and it is often used to test the effectiveness of different controlling schemes [13-16]. So in this chapter, to examine the effectiveness of the proposed TDRNN model, we investigate the application of the TDRNN to the control of

The inverted pendulum system used here is shown in Fig.6, which is formed from a cart, a pendulum and a rail for defining position of cart. The Pendulum is hinged on the center of the top surface of the cart and can rotate around the pivot in the same vertical plane with the

0.8

Fig. 4. Control curves with line type reference

0.0

Fig. 5. Control curves with quadrate wave reference

rail. The cart can move right or left on the rail freely.

0.2

0.4

*z*(*t*)

**4.3 Inverted pendulum control** 

inverted pendulums.

0.6

0.8

1.0

0.9

1.0

*z*(*t*)

1.1

1.2

Fig. 3. Comparison of error curves obtained by different methods

#### **4.2 Bilinear DGP system control**

In this section, we control the following bilinear DGP system using the TDRNN model as a controller.

$$z(t) = 0.5 - 0.4z(t-1) + 0.4z(t-1)u(t-1) + u(t) \,. \tag{39}$$

The system output at an arbitrary time is influenced by all the past information. The control reference curves are respectively taken as:

1. Line type:

$$z(t) = 1.0 \; ; \; \tag{40}$$

2. Quadrate wave:

$$z(t) = \begin{cases} 0.0 & \text{(2k} \cdot \text{5} \le t < (2k+1) \cdot \text{5}, (k=0,1,2,\cdots)) \\ 1.0 & \text{((2k-1)\cdot 5 \le t < 2k \cdot \text{5}, (k=1,2,3,\cdots))} \end{cases} \tag{41}$$

The parameters on the TDRNN in the experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly, besides, , and are set as 0.3, 0.6, 0.4 respectively. Figures 4 and 5 show the control results. Figure 4 shows the control curve using the proposed TDRNN model when the control reference is taken as a line type. Figure 5 shows the control curve when the reference is taken as a quadrate wave type. From these results it can be seen that the proposed control model and algorithm possess a satisfactory control precision.

Fig. 4. Control curves with line type reference

0 20 40 60 80 100

Elman error curve TDRNN error curve

*zt zt zt ut ut* ( ) 0.5 0.4 ( 1) 0.4 ( 1) ( 1) ( ) . (39)

are set as 0.3, 0.6, 0.4 respectively. Figures 4 and 5 show the control

*z t*( ) 1.0 ; (40)

(41)

*k*

In this section, we control the following bilinear DGP system using the TDRNN model as a

The system output at an arbitrary time is influenced by all the past information. The control

0.0 (2 5 (2 1) 5, ( 0,1,2, )) ( ) 1.0 ((2 1) 5 2 5, ( 1,2,3, )) *k tk k*

The parameters on the TDRNN in the experiments are taken as follows. The number of hidden nodes is taken as 6, the weights are initialized in the interval [-2, 2] randomly,

results. Figure 4 shows the control curve using the proposed TDRNN model when the control reference is taken as a line type. Figure 5 shows the control curve when the reference is taken as a quadrate wave type. From these results it can be seen that the proposed control

*k tk k* 


Fig. 3. Comparison of error curves obtained by different methods

0.00

0.02

0.04

Error

**4.2 Bilinear DGP system control** 

reference curves are respectively taken as:

*z t*

model and algorithm possess a satisfactory control precision.

controller.

1. Line type:

besides,

2. Quadrate wave:

 , and 

0.06

0.08

Fig. 5. Control curves with quadrate wave reference
