**3. The terminology**

The basic building element of any neural network is an **artificial neural network** cell (Fig. 2 left).

Fig. 2. The artificial neural network cell (left) and the general neural network system (right)

<sup>1</sup>The measurement systems usually (for example vacuum gauge) operate indirectly. Through measurement of different parameters the observed value of the system (output) can be deduced. Such is the case with the measurement of the cathode current at inverted magnetron. The current is in nonlinear dependence with the pressure in the vacuum system. In such system the dependence of the current versus pressure is not known analytically – at least not good enough - to use the analytical expression directly. This makes ideal ground to use neural network to build the adequate model.

Neural Networks and Static Modelling 7

The process of adaptation of a neural network is called "training" or "learning". During supervised training, the input – output pairs are presented to the neural network, and the

The measured data points are consecutively presented to the neural network. For each data point, the neural network produces an output value which normally differs from the **target value**. The difference between the two is the approximation error in the particular data point. The error is then propagated back through the neural network towards the input, and the correction of the connection weights is made to lower the output error. There are numerous methods for correction of the connection weight. The most frequently used

The training process continues from the first data point included in the training set to the very last, but the queue order is not important. A single training run on a complete training data set is called an **epoch**. Usually several epochs are needed to achieve the acceptable error (**training error**) for each data point. The number of epochs depends on various

When the training achieves the desired accuracy, it is stopped. At this point, the model can reproduce the given data points with a prescribed precision for all data points. It is good practice to make additional measurements (**test data set**) to validate the model in the points not included in the training set. The model produces another error called the **test error**,

Tikk et al. (2003) and Mhaskar (1996) have provided a detailed survey on the evolution of approximation theory and the use of neural networks for approximation purposes. In both papers, the mathematical background is being provided, which has been used as the background for other researchers who studied the approximation properties of various

There is another very important work provided by Wray and Green (1994). They showed that, when neural networks are operated on digital computers (which is true in the vast majority of cases), real limitations of numerical computing should also be considered. Approximation

The majority of works have frequently been used as a misleading promise, and even proof, that neural networks can be used as the general approximation tool for any kind of function. The provided proofs are theoretically correct but do not take into account the fact that all

Nonlinearity of the neural network cell simulated on a digital computer is realized by polynomial approximation. Therefore, each neural cell produces a polynomial activation function. Polynomial equivalence for the activation function is true even in the theoretical sense, i.e., the activation function is an analytical function, and analytical functions have

properties like universal approximation and the best approximation become useless.

The overview of mentioned surveys is given in (Belič, 2006).

those neural network systems were run on digital computers.

training algorithm iteratively changes the weights of the neural network.

parameters but can easily reach numbers from 100,000 to several million.

**3.1 Creation of model – The training process** 

algorithm is called the **error backpropagation algorithm**.

which is normally higher than the training error.

**4. Some critical aspects** 

their polynomial equivalents.

neural networks.

Each artificial neural network consists of a number of **inputs (synapses)** that are connected to the **summing junction**. The values of inputs are multiplied by adequate weights **w (synaptic weights)** and summed with other inputs. The training process changes the values of connection weights, thus producing the effect of changing the input connection strengths. Sometimes there is a special input to the neural cell, called the **bias input**. The bias input value is fixed to 1, and the connection weight is adapted during the training process as well. The value of summed and weighted inputs is the argument of an **activation function** (Fig. 3) which produces the final output of an artificial neural cell. In most cases, the activation function ( ) *x* is of **sigmoidal**2 type. Some neural network architectures use the mixture of sigmoidal and linear activation functions (radial basis functions are a special type of neural network that use the neural network cells with linear activation functions in the output layer and non-sigmoidal functions in the hidden layer).

Artificial neural network cells are combined in the **neural network architecture** which is by default composed of two layers that provide communication with "outer world" (Fig. 2 right). Those layers are referred to as the **input and output layer** respectively. Between the two, there is a number of **hidden layers** which transform the signal from the input layer to the output layer. The hidden layers are called "hidden" for they are not directly connected, or visible, to the input or output of the neural network system. These hidden layers contribute significantly to the adaptive formation of the non-linear neural network inputoutput transfer function and thus to the properties of the system.

Fig. 3. The activation function ( ) *x* of an artificial neural network cell.

2 One of the sigmoidal type functions is ( ) 1/(1 ) *<sup>x</sup> x e* .

Each artificial neural network consists of a number of **inputs (synapses)** that are connected to the **summing junction**. The values of inputs are multiplied by adequate weights **w (synaptic weights)** and summed with other inputs. The training process changes the values of connection weights, thus producing the effect of changing the input connection strengths. Sometimes there is a special input to the neural cell, called the **bias input**. The bias input value is fixed to 1, and the connection weight is adapted during the training process as well. The value of summed and weighted inputs is the argument of an **activation function** (Fig. 3) which produces the final output of an artificial neural cell. In most cases, the activation

sigmoidal and linear activation functions (radial basis functions are a special type of neural network that use the neural network cells with linear activation functions in the output layer

Artificial neural network cells are combined in the **neural network architecture** which is by default composed of two layers that provide communication with "outer world" (Fig. 2 right). Those layers are referred to as the **input and output layer** respectively. Between the two, there is a number of **hidden layers** which transform the signal from the input layer to the output layer. The hidden layers are called "hidden" for they are not directly connected, or visible, to the input or output of the neural network system. These hidden layers contribute significantly to the adaptive formation of the non-linear neural network input-

( ) *x* is of **sigmoidal**2 type. Some neural network architectures use the mixture of

**0.00**

( ) *x* of an artificial neural network cell.

**x - S um o f ne ura l ne tw ork ce ll inpu ts**

**-5 -4 -3 -2 -1012 3**

*x e* .

**0.20**

**0.40**

**0.60**

**0.80**

*φ (x) = y =*

**1.00**

function

**y - activation**

Fig. 3. The activation function

and non-sigmoidal functions in the hidden layer).

output transfer function and thus to the properties of the system.

Satur ation m argin S m = 0.1 - interval 0 -1 is limited to 0.05 - 0.95

2 One of the sigmoidal type functions is ( ) 1/(1 ) *<sup>x</sup>*
