**3.4 MMC nets**

A specific type of recurrent neural networks that show fixed-point attractors and that are particularly suited to describe systems with redundant degrees of freedom are the so called MMC nets. The easiest way to construct such a net is to start with a simpler version. Given a linear equation with n variables

Each of these n equations represents the computation performed by one neuroid. So the complete network represents Multiple Solutions of the Basis Equation, and is therefore termed MSBE net. Different to Hopfield nets, the weights are in general asymmetric (apart from the special case that all parameters a are identical, i.e. a1 = a2 = a3), but follow the rule wij = 1/wji . The diagonal weights, by which each unit excites itself, could be zero, as in the example, or any positive value di, if all weights of this equation are further normalized by multiplication with 1/(di+1). Positive diagonal weights influence the dynamics to adopt low-pass filter-like properties, because the earlier state of this unit is transferred to the actual state with the factor d/(d+1). As di can arbitrarily be chosen (di ≤ 0), the weights may then not follow the rule wij = 1/wji anymore. Starting this net with any vector **a**, the net will stabilize at a vector fulfilling the basic equation. This means that the attractor points form a smooth, in this example two-dimensional, space. This is another difference to Hopfield nets which show discrete attractor points. Furthermore, there are no nonlinear characteristics necessary.

This network can be expanded to form an MMC net. MMC nets result from the combination of several MSBE nets (i.e. several basis equations) with shared units. Such MMC nets can be used to describe landmark navigation in insects and as a model describing place cells found in rodents (Cruse 2002). However, the principle can also be used to represent the kinematics of a body with complex geometry (Steinkühler and Cruse 1998). As a simple example, we will use a three-joint arm that moves in two dimensional space, therefore having one extra degree of freedom (Fig. 10.**a**)( Le Yang &Yanbo Xue, (2009).

The properties of this type of network can best be described by starting with a simple linear version (Fig. 10.**b**). As we have a two-dimensional case, the complete net consists of two identical networks. The output values correspond to the Cartesian coordinates of the six vectors shown in Fig.10.**a**, the x coordinates of the vectors given by the net shown with solid lines, the y coordinates by dashed lines. To obtain the weights, vector equations drawn from the geometrical arrangement shown in Fig. 10.**a** are used as basis equations. This means in this case there are several basis equations possible. For example, each three vectors forming a triangle can be used to provide a basis equation (e.g. **L**1 + **L**2 - **D**1 = 0). As a given variable (e.g. **L**1) occurs in different basis equations, there are several equations to determine this variable.

With respect to their dynamical properties, recurrent neural networks may be described as showing fixed point attractors. How is it possible to design a neural network with specific dynamics? Dynamical systems are often described by differential equations. In such cases the construction of a recurrent network is easily possible: Any system described by a linear differential equation of order n can be transformed into a recurrent neural network containing n units (Nauck et al., 2003). To this end, the differential equation has first to be

A specific type of recurrent neural networks that show fixed-point attractors and that are particularly suited to describe systems with redundant degrees of freedom are the so called MMC nets. The easiest way to construct such a net is to start with a simpler version. Given a

Each of these n equations represents the computation performed by one neuroid. So the complete network represents Multiple Solutions of the Basis Equation, and is therefore termed MSBE net. Different to Hopfield nets, the weights are in general asymmetric (apart from the special case that all parameters a are identical, i.e. a1 = a2 = a3), but follow the rule wij = 1/wji . The diagonal weights, by which each unit excites itself, could be zero, as in the example, or any positive value di, if all weights of this equation are further normalized by multiplication with 1/(di+1). Positive diagonal weights influence the dynamics to adopt low-pass filter-like properties, because the earlier state of this unit is transferred to the actual state with the factor d/(d+1). As di can arbitrarily be chosen (di ≤ 0), the weights may then not follow the rule wij = 1/wji anymore. Starting this net with any vector **a**, the net will stabilize at a vector fulfilling the basic equation. This means that the attractor points form a smooth, in this example two-dimensional, space. This is another difference to Hopfield nets which show discrete attractor points. Furthermore, there are no nonlinear characteristics

This network can be expanded to form an MMC net. MMC nets result from the combination of several MSBE nets (i.e. several basis equations) with shared units. Such MMC nets can be used to describe landmark navigation in insects and as a model describing place cells found in rodents (Cruse 2002). However, the principle can also be used to represent the kinematics of a body with complex geometry (Steinkühler and Cruse 1998). As a simple example, we will use a three-joint arm that moves in two dimensional space, therefore having one extra

The properties of this type of network can best be described by starting with a simple linear version (Fig. 10.**b**). As we have a two-dimensional case, the complete net consists of two identical networks. The output values correspond to the Cartesian coordinates of the six vectors shown in Fig.10.**a**, the x coordinates of the vectors given by the net shown with solid lines, the y coordinates by dashed lines. To obtain the weights, vector equations drawn from the geometrical arrangement shown in Fig. 10.**a** are used as basis equations. This means in this case there are several basis equations possible. For example, each three vectors forming a triangle can be used to provide a basis equation (e.g. **L**1 + **L**2 - **D**1 = 0). As a given variable (e.g. **L**1) occurs in different basis equations, there are several equations to determine this variable.

degree of freedom (Fig. 10.**a**)( Le Yang &Yanbo Xue, (2009).

**3.3 Linear differential equations and recurrent neural networks** 

**3.4 MMC nets** 

necessary.

linear equation with n variables

transferred into a system of n coupled differential equations of the order one.

Fig. 10. An arm consisting of three segments described by the vectors L1 L2, and L3 which are connected by three planar joints.

#### **3.5 Forward models and inverse models**

Using recurrent neural networks like Jordan nets or MMC nets for the control of behavior, i.e. use their output for direct control of the actuators. Several models are proposed to control the movement of such a redundant (or non-redundant) arm. One model corresponds to a schema shown in Fig 11.**a**, where DK and IK may represent feedforward (e. g., threelayer) networks, computing the direct (DK) and inverse kinematics (IK) solutions, respectively. (In the redundant case, IK has to represent a particular solution)( Liu Meiqin, 2006).

Fig. 11. A two-joint arm (a) and a three-joint arm (b) moving in a two dimensional (x-y) plane. When the tip of the arm has to follow a given trajectory (dotted arrow), in the redundant case (b) an infinite number of joint angle combinations can solve the problem for any given position of the tip.

Recurrent Neural Network with Human Simulator Based Virtual Reality 103

An even faster way to solve this problem would be to introduce the inverse model (IM), which corresponds to the inverse kinematics, but may also incorporate the dynamic properties of the process (Fig. 11.c**,** upper part). Given the desired new state of the arm, the inverse model does provide the necessary motor command (**x** -> α) to reach this state. Using such an inverse model, the process could be controlled without the need of an internal

A training algorithm is a procedure that adapts the free parameters of a neural network in response to the behavior of the network embedded in its environment. The goal of the adaptations is to improve the neural network performance for the given task. Most training algorithms for neural networks adapt the network parameters in such a way that a certain error measure (also called cost function) is minimized. Alternatively, the *negative* error

Error measures can be postulated because they seem intuitively right for the task, because they lead to good performance of the neural network for the task, or because they lead to a convenient training algorithm. An example is the Sum Squared Error measure, which always has been a default choice of error measure in neural network research. Error

Different error measures lead to different algorithms to minimize these error measures. Training algorithms can be classified into categories depending on certain distinguishing properties of those algorithms. Four main categories of algorithms that can be distinguished

1. *gradient-based algorithms.* The gradient of the equation of the error measure with respect to all network weights is calculated and the result is used to perform *gradient descent*. This means that the error measure is minimized in steps, by adapting the weight

2. *second-order gradient-based algorithms.* In second-order methods, not only the first derivatives of the error measure are used, but also the second-order derivatives of the

3. *stochastic algorithms*. Stochastic weight updates are made but the stochastic process is directed in such a way, that on average the error measure becomes smaller over time. A gradient of the error measure is not needed so an expression for the gradient doesn't

4. *hybrid algorithms*. Gradient-based algorithms are sometimes combined with stochastic

The following algorithms for the Fully Recurrent Neural Network and the subsets of the

The Back propagation Through Time (BPTT) algorithm is an algorithm that performs an exact computation of the gradient of the error measure for use in the weight adaptation. In

**4. Training algorithms for recurrent neural networks** 

measure or a *performance measure* can be maximized.

measures can also be obtained by accepting some general.

parameters proportional to the negative gradient vector.

elements, which may capture the advantages of both approaches.

**4.1 Back propagation through time (BPTT) algorithm for FRNN** 

feedback loop.

are:( Dijk, 1999)

error measure.

have to exist.

FRNN are investigated in this item:

Depending on the sensory system applied, the DK system might belong to the process, for example when the position of the tip of the arm is registered visually, or it might be part of the neuronal system, for example if joint angles are monitored. In principle, the process can be controlled with this feedback network. However, since in biological systems information transfer is slow, this solution is not appropriate for fast movements because long delays can lead to unstable behavior. A further complication of the control task occurs if there is a process with non negligible dynamic properties. This means that computation is more complex because not only the kinematics but also the forward and inverse dynamics have to be determined.

A simple solution of this problem would be to expand the system by introducing an internal (neuronal) model of the process (Fig. 12.b, FM). By this, the behavior of the process can be predicted in advance, e. g., inertia effects could be compensated for before they occur. This prediction system is called forward model (FM). Usually, although depicted separately in Figure 12.a, in this case the DK is considered part of the forward model. In other words, the FM predicts the next state of the arm (e.g. position, velocity) if the actual state and the motor command is known. Note that the network consists of serially connected IK and FM (including DK) forming a loop within the system and can therefore be represented by a recurrent neural network. The additional external feedback loop is omitted in Fig.12.b, but is still necessary if external disturbances occur. It should be mentioned that the output of the FM, i.e. the predicted sensory feedback, could also be compared with the real sensory feedback. This could be used to distinguish sensors effects produced by own actions from those produced by external activities.( Karaoglan D. Aslan, 2011).

Fig. 12. Three ways of controlling the movement of a redundant (or non redundant) arm as shown in Fig. 11. DK, IK: networks solving the direct or inverse kinematics, respectively. FM: forward model of the process. IM: inverse model of the process. The dashed lines in (c) denote the error signals used to train the inverse model

Depending on the sensory system applied, the DK system might belong to the process, for example when the position of the tip of the arm is registered visually, or it might be part of the neuronal system, for example if joint angles are monitored. In principle, the process can be controlled with this feedback network. However, since in biological systems information transfer is slow, this solution is not appropriate for fast movements because long delays can lead to unstable behavior. A further complication of the control task occurs if there is a process with non negligible dynamic properties. This means that computation is more complex because not only the kinematics but also the forward and inverse dynamics have to

A simple solution of this problem would be to expand the system by introducing an internal (neuronal) model of the process (Fig. 12.b, FM). By this, the behavior of the process can be predicted in advance, e. g., inertia effects could be compensated for before they occur. This prediction system is called forward model (FM). Usually, although depicted separately in Figure 12.a, in this case the DK is considered part of the forward model. In other words, the FM predicts the next state of the arm (e.g. position, velocity) if the actual state and the motor command is known. Note that the network consists of serially connected IK and FM (including DK) forming a loop within the system and can therefore be represented by a recurrent neural network. The additional external feedback loop is omitted in Fig.12.b, but is still necessary if external disturbances occur. It should be mentioned that the output of the FM, i.e. the predicted sensory feedback, could also be compared with the real sensory feedback. This could be used to distinguish sensors effects produced by own actions from

Fig. 12. Three ways of controlling the movement of a redundant (or non redundant) arm as shown in Fig. 11. DK, IK: networks solving the direct or inverse kinematics, respectively. FM: forward model of the process. IM: inverse model of the process. The dashed lines in (c)

those produced by external activities.( Karaoglan D. Aslan, 2011).

denote the error signals used to train the inverse model

be determined.

An even faster way to solve this problem would be to introduce the inverse model (IM), which corresponds to the inverse kinematics, but may also incorporate the dynamic properties of the process (Fig. 11.c**,** upper part). Given the desired new state of the arm, the inverse model does provide the necessary motor command (**x** -> α) to reach this state. Using such an inverse model, the process could be controlled without the need of an internal feedback loop.
