**5.1. Autonomous behavior initiators**

As mentioned in the introduction, the *Drosophila* brain involves nonlinearity and the competence of only a few neurons in the final fly's behavior-initiating mechanism, deep buried in its brain. So, we are interested in neural structures with few neurons and genuine spontaneity. In the previous work [12], we presented a solution where the term behavior is defined as a finite sequence of events distributes in the space time. The initiation of these sequences is fired by using an n-flop, a robust network constructed with sigmoidal-type neurons sharing a common self-activating excitatory input K [20]. Being robust, it serves as foundation for other large-scale optimization structures that solve difficult jobs, such as the travel salesman problem (TSP). The n-flop is the basic building block beyond the concept of programming with neurons [20], and the term is derived from the flip-flop, a computer circuit that has only two stable states. n-flops have n-stable states and the rooted capacity to solve high-dimensional problems [21].

In an n-flop, neurons are programmed by their weight interconnections to solve the constraint that only one of them will be active when the system is in equilibrium. To this end, the output of each neuron is connected with an inhibitory input weight (-1) to each of the other n-1 neuron inputs (lateral inhibition). In addition, each neuron receives a common excitatory input K which, on controlled situations, tends to force all neuron outputs toward 1. A solution or desired output is self-activated by rising K and forcing all neurons to some near-equilibrium but unstable "high-energy" state. At this point, K is set to almost zero, forcing the network to seek a low-energy or equilibrium-stable state. The solution given by a non-biased n-flop is a unique but unpredictable winner, which may be used as a behavior initiator, where "behavior" corresponds to a finite sequence of events, distributes in the space time. A unique winner guarantees a conflict-free operation in terms of robotic conduct. A good stabilized n-flop will always satisfy the syntactic rule "only one winner," even when neurons in the n-flop community share input weights with outside-world neurons, including other n-flops. This conduces toward a proactive scenery where it is possible to control, with events that happen inside or outside the robot, the initiation of behaviors that are being pushed from the inside (**Figure 4**).

Like in biological brains, the proposed behavior initiator constantly consumes energy, and since it controls behaviors, it can affect the whole information processing of the robot.

**Figure 4.** The n-flop as a behavior pusher. A set of sigmoidal-type neurons inhibit each other with −1 value weights and share a common self-activating excitatory input K. Under controlled situations, K raises and forces all neuron outputs toward a 1 output. Once in this high-energy position, K comes down to almost zero, forcing the network to seek a lowenergy or equilibrium state. For each K cycle, a new, unique, random winner is generated. Neurons in the n-flop can share input weights with other insider or outsider neurons. This results in a proactive scenery where behavior initiation is being pushed from the inside and all behaviors can be eventually awakened, but it is still possible for events inside or outside of the robot to modify the statistical occurrence of behaviors.

spatial interaction between the space sensors and the ball, a complex input pattern is produced. Applying a mass center algorithm, the position of the ball can be estimated in terms of

**Figure 5.** The wave generator. A shallow network receives as input 16 decoded timing signals produced by a timing generator. By using a moving wave form as target, the net is trained to generate the chosen waveform and to move it, when the input timing signal moves. Since output neurons directly drive robot's muscles, moving mechanical waves can be created in the robot's joint structure. The chosen hidden layer has nine neurons so data compression occurs.

Autonomous Robots and Behavior Initiators http://dx.doi.org/10.5772/intechopen.71958 133

The ball presence is trained to indicate that the ball is touching the sensors somewhere along the robot's body. It is a first front detector, and its training includes white noise as counter example so that the net learns to distinguish the specific sensor excitation pattern produced

**Figure 6.** The timing generator. After training, the output vector behaves as a programmable shift register, with left, stop, right, commands and a winner-takes-all output, where only one of n-defined output stays on (near to 1), while all other

a unique joint number and given as target to the net.

**5.5. The ball presence detector**

by the ball (**Figure 8**).

m-1 outputs stay off (near to 0).

#### **5.2. The wave generator**

Waves are important in living creatures, and some forms of contraction waves are always used for locomotion and other activities [22].

We use a wave generator network to control the robot's muscles through a one-to-one assignment so that one output neuron controls one muscle. The net is pre-trained with a set of inputs that produce output values in the range 0–1. This moves the joint associated to the muscle in the range (α, −α) where α is a target angle measured in degrees. With the appropriated targets, the net learns to reproduce moving wave forms in its outputs (**Figure 5**).

The training objective is to create a neural moving wave that in turn produces a mechanical moving wave through the robot's body.

#### **5.3. The timing generator**

Timing is important in living creatures making it possible to control complex thing, from walking to sleeping [23]. We use a neural timing generator trained with backpropagation so that its output vector behaves as a programmable shift register with left, stop, right commands and a winner-takes-all output; the winner stays near 1, while all other m−1 outputs stay off (near to 0). The training objective is to create neural timing signals that produce mechanical timing through the robot's body (**Figure 6**).

#### **5.4. The position predictor**

The position predictor is trained to indicate the position of the ball when it touches the sensors. The predictor receives sensor signals in the range 0–1 as input and predicts the mean position of the detected ball in terms of one joint number. For example, in **Figure 7**, due to

**Figure 5.** The wave generator. A shallow network receives as input 16 decoded timing signals produced by a timing generator. By using a moving wave form as target, the net is trained to generate the chosen waveform and to move it, when the input timing signal moves. Since output neurons directly drive robot's muscles, moving mechanical waves can be created in the robot's joint structure. The chosen hidden layer has nine neurons so data compression occurs.

spatial interaction between the space sensors and the ball, a complex input pattern is produced. Applying a mass center algorithm, the position of the ball can be estimated in terms of a unique joint number and given as target to the net.

#### **5.5. The ball presence detector**

**5.2. The wave generator**

132 Human-Robot Interaction - Theory and Application

used for locomotion and other activities [22].

outside of the robot to modify the statistical occurrence of behaviors.

moving wave through the robot's body.

timing through the robot's body (**Figure 6**).

**5.3. The timing generator**

**5.4. The position predictor**

Waves are important in living creatures, and some forms of contraction waves are always

**Figure 4.** The n-flop as a behavior pusher. A set of sigmoidal-type neurons inhibit each other with −1 value weights and share a common self-activating excitatory input K. Under controlled situations, K raises and forces all neuron outputs toward a 1 output. Once in this high-energy position, K comes down to almost zero, forcing the network to seek a lowenergy or equilibrium state. For each K cycle, a new, unique, random winner is generated. Neurons in the n-flop can share input weights with other insider or outsider neurons. This results in a proactive scenery where behavior initiation is being pushed from the inside and all behaviors can be eventually awakened, but it is still possible for events inside or

We use a wave generator network to control the robot's muscles through a one-to-one assignment so that one output neuron controls one muscle. The net is pre-trained with a set of inputs that produce output values in the range 0–1. This moves the joint associated to the muscle in the range (α, −α) where α is a target angle measured in degrees. With the appropriated tar-

The training objective is to create a neural moving wave that in turn produces a mechanical

Timing is important in living creatures making it possible to control complex thing, from walking to sleeping [23]. We use a neural timing generator trained with backpropagation so that its output vector behaves as a programmable shift register with left, stop, right commands and a winner-takes-all output; the winner stays near 1, while all other m−1 outputs stay off (near to 0). The training objective is to create neural timing signals that produce mechanical

The position predictor is trained to indicate the position of the ball when it touches the sensors. The predictor receives sensor signals in the range 0–1 as input and predicts the mean position of the detected ball in terms of one joint number. For example, in **Figure 7**, due to

gets, the net learns to reproduce moving wave forms in its outputs (**Figure 5**).

The ball presence is trained to indicate that the ball is touching the sensors somewhere along the robot's body. It is a first front detector, and its training includes white noise as counter example so that the net learns to distinguish the specific sensor excitation pattern produced by the ball (**Figure 8**).

**Figure 6.** The timing generator. After training, the output vector behaves as a programmable shift register, with left, stop, right, commands and a winner-takes-all output, where only one of n-defined output stays on (near to 1), while all other m-1 outputs stay off (near to 0).

**Figure 7.** The position predictor is trained to indicate the position of the sensed ball in terms of a joint number. From 48 sensors, 16 outputs are derived.

#### **5.6. The autonomous neural controller**

Our next step is to assemble the above-defined neural devices into an integrated autonomous neural controller (ANC) that combines the different capacities of the participant networks. As shown in **Figure 9**, a five-flop is established as a basal behavior initiator, where neuron 1 is connected with a positive weight to a constant output, becoming the neuron with the highest probability to win, assuming the role of a default network. When the ball presence network becomes active, the state 3 may become a winner activating the three-flop, which feeds random values to the wave generator producing a random moving wave. Notice that a single event (ball presence) activates a complex assembly of neural devices that work by themselves, creating a mechanical wave running through the robot's body.

Specifically, the outputs of the ball position predictor (16) are weight connected with the neurons of the 3-flop, making possible for the ball position to control the direction of the moving

**Figure 9.** The autonomous neural controller. A 5-flop acts as basal behavior initiator; its number 1 neuron is connected with a positive weight value to the constant 1 line, making the neuron the highest probability to win (default state). When the ball presence network (bp) becomes active, the state 3 becomes a likely winner that in turn activates the 3-flop. Next, random values are fed to the wave generator, producing a random moving mechanical wave in the robot. To promote complex behaviors, selected outputs are allowed to develop connecting weights with other neurons, for instance, the 3-flop neurons have connective input weights with the position predictor network. Through weight mutations, complex behaviors emerge.

Autonomous Robots and Behavior Initiators http://dx.doi.org/10.5772/intechopen.71958 135

The hidden weight in the wave generator (144 weights) is also allowed to mutate, opening opportunities for different wave forms and wave movements to appear. The overall behavior of the free-running autonomous neural controller is thus governed by these 192 variables.

Once the autonomous neural controller (ANC) begins to behave like an orchestra conductor, issuing timing signals to coordinate activities among different control regions, humans begin to interact with the robot through a keyboard and a visual interface (**Figure 10**). Humans are asked to play the coconut dance game, in which a couple tries, without using their hands, to move upward a coconut placed between them at their waist level; in our case, the human player uses the robot as dancing partner. We choose this activity because it requires a close, coordinate interaction between the two participants, and it doesn't have a trivial solution. The coconut (ball) is subjected to gravity force and is released somewhere between the dancers. The human, represented by a flexible wall, must use the keyboard to move toward the robot and trap the ball between the two bodies; he/she then uses the keyboard to manipulate a moving body bending that pushes the ball up. The game is won when the ball is pushed up, out

Animated by internal n-flops, the robot behaves proactively, burning energy and initiating

wave. This comprises 16 × 3= 48 weights.

**6. The human-robot interaction**

behaviors independently of the outside world.

of the body's reach.

To promote complex behaviors, a set of selected outputs are allowed to have connecting weights with a selected set of neurons (**Figure 9**).

**Figure 8.** The ball presence detector is trained to indicate that the ball is touching the sensors somewhere along the robot's body.

**Figure 9.** The autonomous neural controller. A 5-flop acts as basal behavior initiator; its number 1 neuron is connected with a positive weight value to the constant 1 line, making the neuron the highest probability to win (default state). When the ball presence network (bp) becomes active, the state 3 becomes a likely winner that in turn activates the 3-flop. Next, random values are fed to the wave generator, producing a random moving mechanical wave in the robot. To promote complex behaviors, selected outputs are allowed to develop connecting weights with other neurons, for instance, the 3-flop neurons have connective input weights with the position predictor network. Through weight mutations, complex behaviors emerge.

Specifically, the outputs of the ball position predictor (16) are weight connected with the neurons of the 3-flop, making possible for the ball position to control the direction of the moving wave. This comprises 16 × 3= 48 weights.

The hidden weight in the wave generator (144 weights) is also allowed to mutate, opening opportunities for different wave forms and wave movements to appear. The overall behavior of the free-running autonomous neural controller is thus governed by these 192 variables.
