**4. Discussion and future work**

The main idea of this paper is to demonstrate the effectiveness of cognitive segmentation of the behavior sequence. From the simulation and the practical experiments, we can conclude that this method is successful and it is useful for future study on cognitive imitation learning. Because ISAC is pneumatically driven and it is not very precise, although it can move the end-effector according to the behavior sequence and complete the movement successfully, it cannot grasp the piece every time. Fig.12 only shows one successful experiment.

We have finished current work on applying the proposed method on simulation platform and ISAC robot. Our long term goal is to design a robust behavior generation method which enables robots to work in a dynamic human-robot interaction environment safely. This requires that system have the ability to learn knowledge from demonstration, develop new skills through learning and adapt the skills to new situations. Therefore, robots should learn the demonstration, store the knowledge and retrieve the knowledge for future tasks. Therefore, in cognitive robotics, the functions of robotic control, perception, planning, and interaction are always incorporated into cognitive architectures. Upon that, researchers can use general cognitive processes to enhance imitation learning processes and stores the knowledge for other cognitive functions.

Implementation of a Framework for Imitation

many candidate policies can be design in a probabilistic way.

human beings learn from the demonstrations from childhood.

learning when the task constraints were changed a lot.

architectures with the imitation learning technologies is successful.

interdiciplinary research.

**5. Conclusion** 

behavior generation.

**6. References** 

Learning on a Humanoid Robot Using a Cognitive Architecture 207

Another possible future work is to design a probabilistic imitation learning method especially for the generation stage. It is known that the sensed information, and the results of the actions are uncertain. Therefore, robots should make a probabilistic decision in the imitation learning to complete tasks. Generally, the imitation learning is considered to be a learning of control policies[Argall, 2009]. Therefore, how can robots choose a policy from

Machine learning normally is a iterative process, in which, computers or robots obtain the experience from the exercises, either good or devil. The decision making is improved by utilizing the results fromthe iterative learnings in the practice. In Atkeson's famous experiment, inverted pendulum experiment, the robot tried to improve the performance through a feedback process. Initially, the robot may fail serveral times to hold pendulum in a balanced position. However, it can obtain the experiences from the failure, and finally got success. Currently, there are two types of teaching methods can be used for robots to improve and fasten the learning process: one is to provide the demonstration at the begining and robots try to complete a similar but different task from reproducing the demonstration; the other one is to train robots to learn strarting from scratch, and correct the behavior of the robots in the learning process.Both methods are effective for robotic imitation learning. And we expect that we can combine the two kinds of methods by simulating the way in which

Currently, Cognitive Sciences receives broad attention from the robotics research community. It is expected that robots can behave like a real human and live with us in the future, and it is reasonable that researchers may seek the solution or motivation from the

This paper proposes a cognitive framework to incorporate cognitive segmentation and the DMP algorithm in a cognitive architecture to deal with generating new behaviors in similar but different situations using imitation learning method. The simulation and experimental results demonstrates that this method is effective to solve basic problems in imitation

The main contribution of this paper is that it provides a framework and architecture for robots to complete some complicated tasks, especially in the situation where several task constraints have been changed. A cognitive segmentation method is proposed in this paper. And the experimental resutls demonstrates that the integration of robotic cognitive

Basically, the current research in imitation learning for robots is still a control problem in which the sensory information increases largely. Cognitive robots should understand the target of the task, incorporate the perceptual information, and use cognitive methods to generate suitable behaviors in a dynamic environment. This paper provides a possible solution, which can be used in different cognitive architectures, for the future cognitive

Albus, J. and Barbera, A. (2005). RCS: A Cognitive Architecture for Intelligent Multi-Agent

Systems. *Annual Reviews in Control,* Vol.29, No.1, pp. 87-99, 2005

Some researchers are inrested in transferring skills and behaviors between robots. The skill transfer between robots does not implemented in this paper. However, it is still can be incorporated in this cognitive architecture.

Skill Transfer is divided into two parts: demonstration and observation.

Assume that ISAC is asked to transfer the skills to another robot, named Motoman. ISAC demonstrates the behavior sequences to Motoman strictly follows the demonstrations from the human teacher. Therefore, there are three demonstrations to reach, grasp and move the Knight.

Fig. 15. ISAC demonstrates the behavior sequences with the knight at different locations

The only difference between the observation by ISAC and Motoman is that Motoman should observe the demonstrations using the camera and convert the recorded data in the tasks space, which is the movements of the right hand of ISAC, to the joint space.

O�� � �O�� ��, which is a ��� matrix, records the position values of the end-effector on the right arm of ISAC and the temporal information related to the sampling points.

$$\boldsymbol{\Theta}\_{\rm{ors}} = \text{inverse kinematics}(\mathbf{O}\_{\rm{rs}}) \tag{13}$$

P�� � �P�� �� is still a ��� matrix recording the positions of the Knight on the chess board in the Cartesian space in the demonstration and the temporal information related to the sampling points

After the observation, Motoman uses the same process as described in former paragraphs to segment, recognize, and generates new behaviors in simialr but slightly different situations.

In the future, our lab intend to implemente the skill stransfer using this framework and cognitive architecture described in this paper.

The existing problem is this cognitive framework largely relies on the vision system. In this paper, we proposed a cognitive segmentation method using the visual information from the cameras. As known, vision system is not very stable, and it is easy to be affected by the environmental issues, e.g., light, perspective, and noises. Therefore, how to design a stable vision system is crucial for robotics research, especially for humanoid robots and cognitive robots[Tan and Liao, 2007a].

Another possible future work is to design a probabilistic imitation learning method especially for the generation stage. It is known that the sensed information, and the results of the actions are uncertain. Therefore, robots should make a probabilistic decision in the imitation learning to complete tasks. Generally, the imitation learning is considered to be a learning of control policies[Argall, 2009]. Therefore, how can robots choose a policy from many candidate policies can be design in a probabilistic way.

Machine learning normally is a iterative process, in which, computers or robots obtain the experience from the exercises, either good or devil. The decision making is improved by utilizing the results fromthe iterative learnings in the practice. In Atkeson's famous experiment, inverted pendulum experiment, the robot tried to improve the performance through a feedback process. Initially, the robot may fail serveral times to hold pendulum in a balanced position. However, it can obtain the experiences from the failure, and finally got success. Currently, there are two types of teaching methods can be used for robots to improve and fasten the learning process: one is to provide the demonstration at the begining and robots try to complete a similar but different task from reproducing the demonstration; the other one is to train robots to learn strarting from scratch, and correct the behavior of the robots in the learning process.Both methods are effective for robotic imitation learning. And we expect that we can combine the two kinds of methods by simulating the way in which human beings learn from the demonstrations from childhood.

Currently, Cognitive Sciences receives broad attention from the robotics research community. It is expected that robots can behave like a real human and live with us in the future, and it is reasonable that researchers may seek the solution or motivation from the interdiciplinary research.
