**2. Problem statement**

However, the calculations could slowly occurred and may even overflow or

learning speed [3].

*Deep Learning Applications*

iterations.

certain cases.

**4**

fluctuate between the optima. These limitations encouraged researchers to look for more powerful optimizations techniques that can help reach the optimal solution in an improved fashion, guarantee the convergence of control system and increase the

A lot of optimization techniques of neural network are widely used. Among them, particle swarm optimization (PSO), the subject of this chapter, studied in different papers like in [3], and originates from the behavioral simulation of fish schooling and bird flocking. The conceptual model, at some point in the evolution of the algorithm, was an optimizer following which a number of parameters extraneous to optimization were eliminated leading to the basic PSO algorithm [3].

This technique is used in many applications of neural network optimization like identification, control and modeling. For instance, in [4], the authors used the PSO based neural network optimization for prediction of diameter errors in a boring machine. In their work, they established an improvement in the quality of optimization of the neural networks and error compensations with the use of the PSO algorithm, which achieves a better machining precision with fewer numbers of

The PSO algorithm is proposed to get optimal the parameters of ANN. This algorithm is well used because it has convergent result and not require many iterations, so in relative calculation relative quick. PSO is a population-based approach, which uses the swarm intelligence generated by the cooperation and competition between the particles in a swarm. It has been emerged successfully to a

For example, in [5], the authors compared the performance of the PSO technique with other EAs for both continuous and discrete optimization problems in terms of processing time, convergence speed, and quality of the results. In addition, in [6], the authors proposed a PSO learning algorithm that self-generates radial basis function neural network (RBFN) to deal with three non-linear problems. This proposed PSO allows a high accuracy within a short training time when determining RBFN with small number of radial basis functions. Then, in [7], a PSO algorithm was developed by the authors to find the optimum process parameters which satisfy given limit, tool wear and surface roughness values and maximize the productivity at the same time. Also, in [8], the authors described an evolutionary algorithm for evolving the ANN which was based on the PSO technique. Both the architecture and the weights of the ANN were adaptively adjusted according to the quality of the neural network until the best architecture or a terminating criterion was reached. Moreover, the performance of the basic PSO algorithm with the constriction PSO on some test functions of different dimensions was compared by [9] and they found that the use of constriction PSO with mutation provided significant improvement in

Further, in [10], it is presented an improved PSO algorithm for neural network training employing a population diversity method to avoid premature convergence. Furthermore, in [11], the authors used the PSO technique to optimize the grinding process parameters such as wheel and workpiece speed, depth and lead of dressing, etc. subjected to suitable constraints with the objective of minimizing the production cost and obtaining the finest possible surface finish. As well as, by comparing the PSO algorithm results with genetic algorithms and quadratic programming techniques, the PSO algorithm gives the global optimum solution with the objective to obtain minimum cost of manufacturing, [12]. Equally, in [13], the authors applied ANN - PSO approach for selection of optimum process parameters for minimizing burr size in drilling process. Besides that, the PSO algorithm was applied for optimization of multi objective problem in tile manufacturing process [14] and also

wide variety of search and optimization problems.

The indirect adaptive control that is used, in this chapter, is composed of two blocks: a block of neural network model and a block of control system. The proposed control system is a neural network controller. At the simulation, it is assume that the neural network controller parameter's depending of the model parameter's as given in **Figure 1**.

In this architecture of indirect control, *r k*ð Þ is the desired value, *u k*ð Þ is the control law from the controller, *y k*ð Þ is the output of the nonlinear system, *yr k*ð Þ is the output of the neural network model, *e k*ð Þ is the identification error, ^*ec*ð Þ*k* is the estimated tracking error, *ec*ð Þ*k* is the tracking error and *k* is the discrete time.

The aim of this chapter is to find a control law *u k*ð Þ to the nonlinear system, given by the Eq. (1), based on the tuning neural network controller's parameters in order that the system output *y k*ð Þ tracks, where possible, the desired value *r k*ð Þ.

**Figure 1.** *The architecture of indirect neural control.*

$$y(k+1) = f\left(y(k), \ldots, y\left(k - n\_{\mathcal{I}}\right), u(k), \ldots, u(k - n\_{u})\right) \tag{1}$$

2.algorithms based on Newton's method in which the direction of descent is the

*Tuning Artificial Neural Network Controller Using Particle Swarm Optimization Technique…*

3. the algorithms of the quasi-Newton type, in which an approximation *Hk* of the Hessian matrix evaluated in the iterates is built, the direction being then, as for

The gradient back-propagation algorithm is the most widely used for weight adaptation, the goal of which is to find the appropriate combination of connection

*yk* � *yrk*

*yk* and *yrk* being, respectively, the desired output and the actual output of the *k*

Compared to the desired outputs, the resulting errors allow the weights of the

This procedure is based on an extension of the Delta rule which involves a gradient descent and which consists in propagating an observation of the input of

the neural network through the neural layer, to obtain the output values.

output neurons to be adjusted. Without the presence of the hidden layer, the knowledge of these errors allows a direct calculation of the gradient and makes the adjustment of the weights of these single neurons, easy as shown by the Delta rule. So for a network with hidden layers, ignoring the desired outputs of the hidden neurons, it thus remains impossible to know the errors of these neurons. So, as it is, this process cannot be used for weight adjustment of hidden neurons. The intuition which solves this difficulty and which gave rise to back-propagation was as follows: the activity of a neuron is linked to neurons of the preceding layer. Thus, the error of an output neuron is due to the hidden neurons of the previous layer in proportion to their influence; therefore according to their activation and the weights that connect the hidden neurons to the output neuron. Therefore, we seek to obtain the contributions of the *L* hidden neurons which gave the error of the output neuron *k*. The back-propagation procedure consists in propagating the error gradient (error produced during the propagation of an input vector) in the network. In this phase, the propagation of an output neuron's error starts from the output layer to

It is therefore sufficient to retrace the original activation path backwards, starting from the errors of the output neurons, to obtain the error of all the neurons in the network. Once the corresponding error for each neuron is known, the weight

Another class of methods, more sophisticated than the previous one, is based on

∇*E* (7)

second order algorithms, based on Newton's method which adapts the weights

*<sup>Δ</sup><sup>w</sup>* ¼ �*H*�<sup>1</sup>

*f x*ð Þ*<sup>k</sup> dk* ¼ ∇*f x*ð Þ*<sup>k</sup>* (4)

*Hkdk* ¼ ∇*f x*ð Þ*<sup>k</sup>* (5)

� �<sup>2</sup> (6)

∇2

the method of Newton, the solution of the linear system

*<sup>E</sup>* <sup>¼</sup> <sup>1</sup> 2 X *k*

weights that minimizes the error function *E* defined by:

solution of the system

*DOI: http://dx.doi.org/10.5772/intechopen.96424*

neuron for a given input vector.

the hidden neurons.

**7**

adaptation relationships can be obtained.

**3.2 Second order optimization method**

according to the following relation:

*f*ð Þ*:* is the nonlinear function mapping specified by the model, *ny* and *nu* are the number of past output and input samples respectively required for prediction.

In this structure, the neural network controller and the neural network model must be updated at the same time. But, it is a difficult task to have a neural identifier learn the system fast enough to adapt to varying parameters. Therefore, the neural controller is ineffective on variations of the system parameters. In this chapter, to solve this problem we propose a fast learning algorithm based on a particle swarm optimization approach.
