**3. Experimental results**

Proposed experimental scenario is: 1) a human teacher demonstrates a behavior by manually moving the right arm of ISAC to reach, grasp and move the Knight on the piece; 2) ISAC records the demonstrations using encoders on its right arm and the cameras on the head; 3) ISAC generates several behaviors to complete tasks in a similar but different situations;

Using the cognitive segmentation method proposed in this paper, each of the three demonstrations are segmented into 2 parts as shown in Fig.6. Each part is considered as a behavior. The black circle demonstrates the point of the change of the world states because the Knight began to move with the end-effector. Based on the assumption in section 2.1, the first behavior of each demonstration is considered as the same type of a behavior, and the same consideration for the second behavior. As shown in Fig.6, three demonstrations are segmented into 2 behaviors with a changing point which is considered as the grasping behavior.

Fig. 7. Segmented behavior sequences.

In Fig. 7 and Fig. 8, Behaviors are categorized according to the formula in Section 2.4. Behavior 1is a Common Behavior and Behavior 2 is a Special Behavior.

Fig. 8. Recorded behavior 1.

200 The Future of Humanoid Robots – Research and Applications

�� is the generated velocity correspondingly. α�, β�, and τ are constants in the equation. From the original paper of the DMP, α�, β�, and τ are chosen as 1, ¼ and 1 heurisitically to achieve

When the position value of the Knight is given, the CEA generates a new trajectory which

After the grasping, behavior 2 is generated strictly following the obtained behavior in

Proposed experimental scenario is: 1) a human teacher demonstrates a behavior by manually moving the right arm of ISAC to reach, grasp and move the Knight on the piece; 2) ISAC records the demonstrations using encoders on its right arm and the cameras on the head; 3) ISAC generates several behaviors to complete tasks in a similar but different

Using the cognitive segmentation method proposed in this paper, each of the three demonstrations are segmented into 2 parts as shown in Fig.6. Each part is considered as a behavior. The black circle demonstrates the point of the change of the world states because the Knight began to move with the end-effector. Based on the assumption in section 2.1, the first behavior of each demonstration is considered as the same type of a behavior, and the same consideration for the second behavior. As shown in Fig.6, three demonstrations are segmented into 2 behaviors with a changing point which is considered as the grasping

In Fig. 7 and Fig. 8, Behaviors are categorized according to the formula in Section 2.4.

Behavior 1is a Common Behavior and Behavior 2 is a Special Behavior.

has similar dynamics to the demonstration of behavior 1.

the convergence.

**3. Experimental results** 

Fig. 7. Segmented behavior sequences.

section.

situations;

behavior.

Fig. 9. Recorded Behavior 2.

Implementation of a Framework for Imitation

Demo1

Fig. 11. Stored knowledge of behavior 1.





PositionY(mm)




600 700 800

PositionX(mm)

Knight on the board, the coordinates of which is (450, 215, -530).

GeneratedPositionX

Fig. 12. Generated New Behavior 1 in X, Y, and Z direction.

0 0.5 1

Time(s)

450

PositionZ(mm)

Learning on a Humanoid Robot Using a Cognitive Architecture 203

Fig. 11 displays the generated behaivor 1 when ISAC is asked to reach, grasp, and move the

GeneratedPositionY

0 0.5 1

Time(s)



PositionZ(mm)

700 800 900

650 700 750

PositionX(mm)


GeneratedPositionZ

0 0.5 1

Time(s)






PositionZ(mm)










PositionY(mm)


0

10

20

30

Obtained

PositionX(mm)







PositionY(mm)



0

10

20

Demo3

500 600 700

PositionX(mm)

10

20

30

40

50

PositionY(mm)

60

70

80

90

100

Demo2

In Fig. 9, on each dimension, the sampled trajectories of different behaviors in each demonstration are normalized in time and magnitude. The magnitude was normalized in the range of (0, 1) for each dimension of the trajectory. Because DMP is chosen in the trajectory generation part and DMP only requires the dynamics of the trajectory. In practical, DMP algorithm automatically normalizes the sampled trajectory, therefore different time periods and magnitudes will not affect the results of the experiments.

In Fig.7, the first row are the trajectories of Behavior 1 in X-direction, the second row are the trajectories of Behavior 1 in Y-direction, and the third row are the trajectories of Behavior 1 in Z-direction. The first column is the trajectory of Behavior 1 in demonstration 1, the second column is the trajectory of Behavior 1 in demonstration 1, the third column is the trajectory of Behavior 1 in demonstration 1, and the fourth column is the processed trajectory of Behavior 1 which will be used for future behavior generation.

Fig. 10. Stored knowledge of behavior 1.

Behavior 2 is calculated by getting the average value of the sampled position values on the common timing points.

In Fig. 9, the left figure is the trajectory of Behavior 2 in demonstration 2 on X-Y plane, the left middle figure is the trajectory of Behavior 2 in demonstration 2 on X-Y plane, the right middle figure is the trajectory Behavior 2 on X-Y plane, and the right figure is the processed trajectory of Behavior 2 which will be used for future behavior generation.

202 The Future of Humanoid Robots – Research and Applications

In Fig. 9, on each dimension, the sampled trajectories of different behaviors in each demonstration are normalized in time and magnitude. The magnitude was normalized in the range of (0, 1) for each dimension of the trajectory. Because DMP is chosen in the trajectory generation part and DMP only requires the dynamics of the trajectory. In practical, DMP algorithm automatically normalizes the sampled trajectory, therefore

In Fig.7, the first row are the trajectories of Behavior 1 in X-direction, the second row are the trajectories of Behavior 1 in Y-direction, and the third row are the trajectories of Behavior 1 in Z-direction. The first column is the trajectory of Behavior 1 in demonstration 1, the second column is the trajectory of Behavior 1 in demonstration 1, the third column is the trajectory of Behavior 1 in demonstration 1, and the fourth column is the processed trajectory of

Behavior 2 is calculated by getting the average value of the sampled position values on the

In Fig. 9, the left figure is the trajectory of Behavior 2 in demonstration 2 on X-Y plane, the left middle figure is the trajectory of Behavior 2 in demonstration 2 on X-Y plane, the right middle figure is the trajectory Behavior 2 on X-Y plane, and the right figure is the processed

trajectory of Behavior 2 which will be used for future behavior generation.

different time periods and magnitudes will not affect the results of the experiments.

0 0.5 1

0 0.5 1

0 0.5 1

Time(s)

Obtained

0 0.5 1

Time(s)

Obtained

0 0.5 1

Time(s)

Obtained

0

0

0

0.5

PositionZ(mm)

1

0.5

PositionY(mm)

1

0.5

PositionX(mm)

1

Time(s)

Demo 3

0 0.5 1

Time(s)

Demo 3

0 0.5 1

Time(s)

Demo 3

0

0

0

0.5

PositionZ(mm)

1

0.5

PositionY(mm)

1

0.5

PositionX(mm)

1

Time(s)

Demo 2

0 0.5 1

Time(s)

Demo 2

0 0.5 1

Time(s)

Demo 2

Behavior 1 which will be used for future behavior generation.

0

0

0

0.5

PositionZ(mm)

1

0.5

PositionY(mm)

1

0.5

PositionX(mm)

1

Fig. 10. Stored knowledge of behavior 1.

0 0.5 1

Time(s)

Demo 1

0 0.5 1

Time(s)

Demo 1

0 0.5 1

Time(s)

Demo 1

0

0

0

0.5

PositionZ(mm)

1

0.5

PositionY(mm)

1

0.5

PositionX(mm)

1

common timing points.

Fig. 11. Stored knowledge of behavior 1.

Fig. 11 displays the generated behaivor 1 when ISAC is asked to reach, grasp, and move the Knight on the board, the coordinates of which is (450, 215, -530).

Fig. 12. Generated New Behavior 1 in X, Y, and Z direction.

Implementation of a Framework for Imitation

move the chess in the expected way.

Fig. 14. ISAC reach, grasp, and move the knight on the board

and it can reach, grasp, and move the piece in an expected way.

**4. Discussion and future work** 

knowledge for other cognitive functions.

experiment.

similar to the demonstrations.

Learning on a Humanoid Robot Using a Cognitive Architecture 205

has been changed, the movement of the Knight is the same as the demonstration. From Fig. 13, it is concluded that this algorithm successfully trained the robot to reach, grasp and

Practical experiments were carried out on ISAC robot as shown in Fig.13. The Knight (blue piece) was placed at (450, 215, -530) which is far away from the place in the demonstrations and ISAC was asked to reach, grasp and move it to the red grid in an expected way which is

The experimental results demonstrates that ISAC successfully learns this behavior sequence,

The main idea of this paper is to demonstrate the effectiveness of cognitive segmentation of the behavior sequence. From the simulation and the practical experiments, we can conclude that this method is successful and it is useful for future study on cognitive imitation learning. Because ISAC is pneumatically driven and it is not very precise, although it can move the end-effector according to the behavior sequence and complete the movement successfully, it cannot grasp the piece every time. Fig.12 only shows one successful

We have finished current work on applying the proposed method on simulation platform and ISAC robot. Our long term goal is to design a robust behavior generation method which enables robots to work in a dynamic human-robot interaction environment safely. This requires that system have the ability to learn knowledge from demonstration, develop new skills through learning and adapt the skills to new situations. Therefore, robots should learn the demonstration, store the knowledge and retrieve the knowledge for future tasks. Therefore, in cognitive robotics, the functions of robotic control, perception, planning, and interaction are always incorporated into cognitive architectures. Upon that, researchers can use general cognitive processes to enhance imitation learning processes and stores the

By comparing Fig. 11 with the fourth column of Fig. 9, similar dynamics of the two trajectories in X, Y, Z-direction can be found.

Grasping is added between Behavior 1 and Behavior 2 based on the assumption in section 2.2.

Fig.12 shows the generated behavior sequences for the new task constraints. The red line is the behavior 1, the blue line is the behavior 2, and the intersection point of the two behaviors is the 'grasping' behavior. The simulation results shows that the end-effector moves to reach the Knight at (450, 215, -530) with similar dynamics to the behavior 1 in the demonstration, then it move the Knight in the same way as behavior 2. Although the position of the Knight has been changed, the movement of the Knight is the same as the demonstration. From Fig. 11, it is concluded that this algorithm successfully trained the robot to reach, grasp and move the chess in the expected way.

Fig. 13. A generated new behavior sequence in cartesian space

Fig. 13 shows the generated behavior sequences for the new task constraints. The red line is the behavior 1, the blue line is the behavior 2, and the intersection point of the two behaviors is the 'grasping' behavior. The simulation results shows that the end-effector moves to reach the Knight at (450, 215, -530) with similar dynamics to the behavior 1 in the demonstration, then it move the Knight in the same way as behavior 2. Although the position of the Knight 204 The Future of Humanoid Robots – Research and Applications

By comparing Fig. 11 with the fourth column of Fig. 9, similar dynamics of the two

Grasping is added between Behavior 1 and Behavior 2 based on the assumption in section

Fig.12 shows the generated behavior sequences for the new task constraints. The red line is the behavior 1, the blue line is the behavior 2, and the intersection point of the two behaviors is the 'grasping' behavior. The simulation results shows that the end-effector moves to reach the Knight at (450, 215, -530) with similar dynamics to the behavior 1 in the demonstration, then it move the Knight in the same way as behavior 2. Although the position of the Knight has been changed, the movement of the Knight is the same as the demonstration. From Fig. 11, it is concluded that this algorithm successfully trained the robot to reach, grasp and

trajectories in X, Y, Z-direction can be found.

move the chess in the expected way.

Fig. 13. A generated new behavior sequence in cartesian space

Fig. 13 shows the generated behavior sequences for the new task constraints. The red line is the behavior 1, the blue line is the behavior 2, and the intersection point of the two behaviors is the 'grasping' behavior. The simulation results shows that the end-effector moves to reach the Knight at (450, 215, -530) with similar dynamics to the behavior 1 in the demonstration, then it move the Knight in the same way as behavior 2. Although the position of the Knight

2.2.

has been changed, the movement of the Knight is the same as the demonstration. From Fig. 13, it is concluded that this algorithm successfully trained the robot to reach, grasp and move the chess in the expected way.

Practical experiments were carried out on ISAC robot as shown in Fig.13. The Knight (blue piece) was placed at (450, 215, -530) which is far away from the place in the demonstrations and ISAC was asked to reach, grasp and move it to the red grid in an expected way which is similar to the demonstrations.

Fig. 14. ISAC reach, grasp, and move the knight on the board

The experimental results demonstrates that ISAC successfully learns this behavior sequence, and it can reach, grasp, and move the piece in an expected way.
