**2. CPG based locomotion controller**

Animal produces may periodic movements such as walking and running, which are called gaits. It was found (Grillner, 1985) that those undirected movements are produced by Central Pattern Generators (CPGs). CPG is a neural network that can produce rhythmic patterned outputs without any input, and they underlie the production of most rhythmic motor patterns. Many researchers developed mathematical CPG models to generate motion in biped robots (Takeshi Mori et. al, 2004), snake robots (Crespi and Ijspeert, 2006), and quadruped robot (Cappelletto et. al, 2007). Those biological inspired controllers insure the production of more natural and simple motion. Also, those oscillators can entrain and adapt the dynamic of the system. For example, Matsuoka, Van-der-pol, Hopf and Rayleigh are oscillators used in robotics control.

Basically, a single neuron is the basic unit of a CPG network. Two neurons when coupled can produce a rhythmic output and this is called a Neural Oscillators (NO). The rhythmic output requires two or more neurons that interact with each other to oscillate and therefore passes by its starting condition (Hooper SL., 2000). Unlike linear systems, nonlinear oscillators can produce stable limit cycles without an oscillatory input of the same frequency, and thus they are very suited to model some parts of the nervous system.

Fig. 1. Matsuoka neural oscillator model. Black dots corresponds to inhibitory connections and the white dots to excitery. Using the notation in (Zaier and Nagashima, 2002), the symbol ">0" reprents a switcher that take the positive part of the input, for example the input from x1 becomes [x1]+

#### **2.1 Matsuoka based oscillator**

42 The Future of Humanoid Robots – Research and Applications

then the Piecewise Linear Function Based Oscillator as our proposed approach will be detailed. The fourth section will present the Experiment Results and Discussion, and finally

Animal produces may periodic movements such as walking and running, which are called gaits. It was found (Grillner, 1985) that those undirected movements are produced by Central Pattern Generators (CPGs). CPG is a neural network that can produce rhythmic patterned outputs without any input, and they underlie the production of most rhythmic motor patterns. Many researchers developed mathematical CPG models to generate motion in biped robots (Takeshi Mori et. al, 2004), snake robots (Crespi and Ijspeert, 2006), and quadruped robot (Cappelletto et. al, 2007). Those biological inspired controllers insure the production of more natural and simple motion. Also, those oscillators can entrain and adapt the dynamic of the system. For example, Matsuoka, Van-der-pol, Hopf and Rayleigh are

Basically, a single neuron is the basic unit of a CPG network. Two neurons when coupled can produce a rhythmic output and this is called a Neural Oscillators (NO). The rhythmic output requires two or more neurons that interact with each other to oscillate and therefore passes by its starting condition (Hooper SL., 2000). Unlike linear systems, nonlinear oscillators can produce stable limit cycles without an oscillatory input of the same

> τr

τa

v1 >0

1

x1

>0

β

x2

β

v2

τr

>0

>0 τa

1

Fig. 1. Matsuoka neural oscillator model. Black dots corresponds to inhibitory connections and the white dots to excitery. Using the notation in (Zaier and Nagashima, 2002), the symbol ">0" reprents a switcher that take the positive part of the input, for example the

µ12 µ21

+

>0

>0

Extensor Neuron

\_

y

Flexor Neuron

frequency, and thus they are very suited to model some parts of the nervous system.

the conclusion will highlight the possible of future developments of the method.

**2. CPG based locomotion controller** 

oscillators used in robotics control.

Tonic Input ( c )

hj

hj

max(g,0)


input from x1 becomes [x1]+

Matsuoka proposed the neural oscillator shown in Figure 1 that consists of a flexor and extensor neuron (Matsuoka, 1985, 1987). Each neuron is presented by a nonlinear differential equation. Each neuron produces a periodic signal to inhabit the other neuron to control the limbs motion (i.e. extending and flexing the elbow). Compared to other models, Matsuoka model uses significantly less computational resources, has less parameters requiring tuning, and has no need for post-processing of the neural output signals (i.e., filtering of the spikes). The mathematical model of the Matsuoka neural oscillator can be expressed by equations (2- 7) and as quoted from Williamson (Williamson, 1999) and illustrated in Fig. 1 using the neuron model of equation 1 and as decribed in (Zaier and Nagashima, 2002):

$$\begin{aligned} \varepsilon \frac{d\mathbf{x}\_1}{dt} + \mathbf{x}\_1 &= \sum \text{inputs} \tag{1} \end{aligned} \tag{1}$$

where ε is the time delay of the neuron.

`

$$\tau\_r \frac{d\mathbf{x}\_1}{dt} + \mathbf{x}\_1 = -\beta v\_1 - \mu\_{21} \left[\mathbf{x}\_2\right]^+ - \sum h\_j \left[\mathbf{g}\_j\right]^+ + c \tag{2}$$

$$
\pi\_a \frac{dv\_1}{dt} + v\_1 = \begin{bmatrix} x\_1 \end{bmatrix}^+ \tag{3}
$$

$$\sigma\_r \frac{d\mathbf{x}\_2}{dt} + \mathbf{x}\_2 = -\beta v\_2 - \mu\_{12} \left[\mathbf{x}\_1\right]^+ - \sum h\_j \left[\mathbf{g}\_j\right]^- + c \tag{4}$$

$$
\pi\_a \frac{dv\_2}{dt} + v\_2 = \left[\chi\_2\right]^+ \tag{5}
$$

$$\mathbf{y}\_i = \begin{bmatrix} \mathbf{x}\_i \end{bmatrix}^+ = \max(\mathbf{0}\_\prime \mathbf{x}\_i) \tag{6}$$

$$\mathbf{y}\_{out} = \begin{bmatrix} \mathbf{x}\_1 \end{bmatrix}^+ - \begin{bmatrix} \mathbf{x}\_2 \end{bmatrix}^+ = y\_1 - y\_2 \tag{7}$$

$$\left[\mathbf{x}\_{i}\right]^{+} = \max(0, \mathbf{x}\_{i}) \tag{8}$$

The extensor neuron 1 in Fig. 1 is governed by equations (2) and (3) and flexor neuron 2 by equations (4) and (5). The variable xi is the neuron potential or firing rate of the ith neuron, *v*<sup>i</sup> is the variable that represents the degree of adaptation or self-inhibition, c is the external tonic input with a constant rate, β specify the time constant for the adaptation, τa and τr are the adaptation and rising rates of the neuron potential, μij is the weight that represents the strength of inhibition connection between neurons, and gj is an external input which is usually the sensory feedback weighted by gain hj. Where the positive part of the input [gj ]+ is applied to one neuron and the negative part [gj]- =-min(gj,0) is applied to the other neuron. The positive part of the output of each neuron is denoted by yi=[x1 ]+ and the final output of the oscillator yout is the difference between the two neurons' outputs. However, the parameters should be adjusted correctly to suite the application the oscillator will be used

Design of Oscillatory Neural Network for Locomotion Control of Humanoid Robots 45

The equation 9 has a periodic solution that attracts other solutions except the trivial one at

In the case of *μ* 0 , the system will be unstable, oscillate forever and the phase plot will never converge to an equilibrium point, as shown in Figure 4 (a) and (b). While in the case of μ 0 , the system will converge to an equilibrium point as shown in Figure 4 (c) and (d). In this case, the decaying is directly proportional to the negative value of μ , where the oscillation dies faster as the constant goes larger in the negative direction. However, if μ is set to zero, then the system has no damping and will oscillate sinusoidally forever as shown in Figure 4 (e). The phase plot draws an exact circle in this case. In conclusion, the Van der Pol oscillator has the ability to produce a periodic behavior which can represent a periodic

The control strategy here is based on piecewise linear oscillators and inspired by the solution of the Van der Pol system in Figure 4b, and with few parameters that can be easily obtained. Moreover, the motivation is to provide much flexibility to the design of the motion pattern generator so that modulation of the output by other circuits such as circuits generating reflexes can be realized without complexity or re-design of the motion pattern generator (Zaier and Kanda, 2007). The method requires not to satisfy constraints on robot's foot or ZMP stability margin, it simply uses piecewise-linear functions and a first order lowpass filter generated by an original recurrent neural network (RNN), where the "integrate and fire" neuron model (Gerstner, 1995) has been used. Indeed, the piecewise linear control is much easier to analyze than the control based directly on non-linear equations. Moreover, this type of control provides much intuition about the system behavior. Also, our method for generating walking motion, although inspired by the inverted pendulum, it considers not strictly the system dynamics. The stability of the system is assured by a gyro feedback loop, while terrain irregularities are compensated by adjusting the pitching motion using a virtual spring-damper systems. The proposed method is straightforward with respect to

the unique equlibrum point (origin of the phase plote).

locomotion pattern of robots (Veskos P., 2005).

Fig. 3. Simulation of Van der Pol Oscillator Behavior.

**3. Piecewise linear function based oscillator** 

for. This can be done by experiments or by specifying the constraints. According to Williamson (Williamson, 1999), for stable oscillations, *τ τ r a* / should be in the range 0.1-0.5. Keeping the ratio *τ τ r a* / constant makes the natural frequency of the oscillator *ωn* (the frequency of the oscillator without an input) proportional to <sup>r</sup> 1 /τ . According to Matsuoka (Matsuoka, 1987), the parameters should be selected based on the following criteria to

ensure a stable rhythm 1 (1 ) *μji β* and 12 21 . 1 *<sup>r</sup> a τ μ μ τ* . Some researchers used Genetic

Algorithm (GA) technique to find and optimize the oscillator's parameters (Inada, 2004).

### **2.2 Van der Pol based oscillator**

The Van der Pol Oscillator, described by second order nonlinear differential equation (13), can be regraded as a mass-spring-damper system. The circuit of this oscillator is shown in Figure 2, where the neuron notation in (Zaier and Nagashima, 2002) is used. It has been adopted by Van de Pol in 1920th to study the oscillations in vacum tube circuits. Recently researchers built CPG model based on Van der Pol oscillator (Senda K, Tanaka T, 2000). Equation (10) is for the forced oscillotor where the right hand side term is the periodic forcing term. The unforced Van der Pol oscillator is investigated using MATLAB/Simulink© block diagram as shown in Figure 3.

$$
\ddot{\mathbf{x}} + \mu \left(\mathbf{x}^2 - \mathbf{1}\right)\dot{\mathbf{x}} + \mathbf{x} = \mathbf{0} \tag{9}
$$

$$\ddot{\mathbf{x}} + \mu \left(\mathbf{x}^2 - \mathbf{1}\right)\dot{\mathbf{x}} + \mathbf{x} = a\sin(2\pi v\tau) \tag{10}$$

Where x and x are the states of the system and μ is a control parameter that represents the degree of nonlinearity of the system or its damping strength.

44 The Future of Humanoid Robots – Research and Applications

for. This can be done by experiments or by specifying the constraints. According to Williamson (Williamson, 1999), for stable oscillations, *τ τ r a* / should be in the range 0.1-0.5. Keeping the ratio *τ τ r a* / constant makes the natural frequency of the oscillator *ωn* (the frequency of the oscillator without an input) proportional to <sup>r</sup> 1 /τ . According to Matsuoka (Matsuoka, 1987), the parameters should be selected based on the following criteria to

and 12 21 . 1 *<sup>r</sup>*

The Van der Pol Oscillator, described by second order nonlinear differential equation (13), can be regraded as a mass-spring-damper system. The circuit of this oscillator is shown in Figure 2, where the neuron notation in (Zaier and Nagashima, 2002) is used. It has been adopted by Van de Pol in 1920th to study the oscillations in vacum tube circuits. Recently researchers built CPG model based on Van der Pol oscillator (Senda K, Tanaka T, 2000). Equation (10) is for the forced oscillotor where the right hand side term is the periodic forcing term. The unforced Van der Pol oscillator is investigated using MATLAB/Simulink©

> τr

 τr

v1

1

τr

>0

x2

1

Fig. 2. Van der Pol oscillator model. Black dots corresponds to inhibitory connections.

 <sup>2</sup> *x x xx* 

<sup>2</sup> *x x xxa v*

Where x and x are the states of the system and μ is a control parameter that represents

1 sin(2 )

the degree of nonlinearity of the system or its damping strength.

τr

v2

1

 

x1

>0

1

*μ μ*

Algorithm (GA) technique to find and optimize the oscillator's parameters (Inada, 2004).

*a τ*

Extensor Neuron

1

Flexor Neuron

1 0 (9)

(10)

 τr *μ*

. Some researchers used Genetic

*τ*

ensure a stable rhythm 1

**2.2 Van der Pol based oscillator** 

block diagram as shown in Figure 3.

input

hj

(1 )

 *μji β*

The equation 9 has a periodic solution that attracts other solutions except the trivial one at the unique equlibrum point (origin of the phase plote).

In the case of *μ* 0 , the system will be unstable, oscillate forever and the phase plot will never converge to an equilibrium point, as shown in Figure 4 (a) and (b). While in the case of μ 0 , the system will converge to an equilibrium point as shown in Figure 4 (c) and (d). In this case, the decaying is directly proportional to the negative value of μ , where the oscillation dies faster as the constant goes larger in the negative direction. However, if μ is set to zero, then the system has no damping and will oscillate sinusoidally forever as shown in Figure 4 (e). The phase plot draws an exact circle in this case. In conclusion, the Van der Pol oscillator has the ability to produce a periodic behavior which can represent a periodic locomotion pattern of robots (Veskos P., 2005).

Fig. 3. Simulation of Van der Pol Oscillator Behavior.
