**5. Conclusion**

We have shown the properties and the possible use of the proposed two-layered movement imitation system. The system can be used for both waveform learning and frequency extraction. Each of these has preferable properties for learning and controlling of robots by imitation. On-line waveform learning can be used for effective and natural learning of robotic movement, with the demonstrator in the loop. During learning the demonstrator can observe the learned behavior of the robot and if necessary adapt his movement to achieve better robot performance. The table wiping task is an example of such on-line learning, where not only is the operator in the loop for the initial trajectory, but online learning adapts the trajectory based on the measured force signal.

Online frequency extraction, essential for waveform learning, can additionally be used for synchronization to external signals. Examples of such tasks are drumming, cooperative rope turning and synchronization to a measured EMG signal. Specially the synchronization to an EMG signal shows at great robustness of the system. Furthermore, the process of synchronizing the movement of the robot and the actuated device can be applied in a similar manner to different tasks and can be used as a common generic algorithm for controlling periodic tasks. It is also easy to implement, with hardly any parameter tuning at all.

The overall structure of the system, based on nonlinear oscillators and dynamic movement primitives is, besides computationally extremely light, also inherently robust. The properties of dynamic movement primitives, which allow on-line modulation, and return smoothly to the desired trajectory after perturbation, allow learning of whole families of movement with one demonstrated trajectory. An example of such is learning of circular movement, which can be increased in amplitude, or we can change the center of the circular movement. Additional modifications, such as slow-down feedback and virtual repulsive forces even expand these properties into a coherent and complete block for control of periodic robotic movements.

#### **6. References**


22 Will-be-set-by-IN-TECH

We have shown the properties and the possible use of the proposed two-layered movement imitation system. The system can be used for both waveform learning and frequency extraction. Each of these has preferable properties for learning and controlling of robots by imitation. On-line waveform learning can be used for effective and natural learning of robotic movement, with the demonstrator in the loop. During learning the demonstrator can observe the learned behavior of the robot and if necessary adapt his movement to achieve better robot performance. The table wiping task is an example of such on-line learning, where not only is the operator in the loop for the initial trajectory, but online learning adapts the trajectory

Online frequency extraction, essential for waveform learning, can additionally be used for synchronization to external signals. Examples of such tasks are drumming, cooperative rope turning and synchronization to a measured EMG signal. Specially the synchronization to an EMG signal shows at great robustness of the system. Furthermore, the process of synchronizing the movement of the robot and the actuated device can be applied in a similar manner to different tasks and can be used as a common generic algorithm for controlling

The overall structure of the system, based on nonlinear oscillators and dynamic movement primitives is, besides computationally extremely light, also inherently robust. The properties of dynamic movement primitives, which allow on-line modulation, and return smoothly to the desired trajectory after perturbation, allow learning of whole families of movement with one demonstrated trajectory. An example of such is learning of circular movement, which can be increased in amplitude, or we can change the center of the circular movement. Additional modifications, such as slow-down feedback and virtual repulsive forces even expand these properties into a coherent and complete block for control of periodic robotic movements.

An, C., Atkeson, C. & Hollerbach, J. (1988). *Model-based control of a robot manipulator*, MIT

Asfour, T., Regenstein, K., Azad, P., Schröder, J., Bierbaum, A., Vahrenkamp, N. & Dillmann, R.

Buchli, J., Righetti, L. & Ijspeert, A. (2006). Engineering entrainment and adaptation in

Buchli, J., Righetti, L. & Ijspeert, A. J. (2005). A dynamical systems approach to learning:

*in Computer Science*, Springer Berlin / Heidelberg, pp. 210–220.

*IEEE-RAS Int. Conf. on Humanoid Robots (Humanoids 2006)*, Genoa, Italy. Atkeson, C., Moore, A. & Schaal, S. (1997). Locally weighted learning, *AI Review* 11: 11–73. Bailey, A. (2004). *Biomimetic control with a feedback coupled nonlinear oscillator: insect experiments, design tools, and hexapedal robot adaptation results*, PhD thesis, Stanford University. Buchli, J. & Ijspeert, A. (2004). A simple, adaptive locomotion toy-system, *in* S. Schaal,

*Behavior (SAB'04)*, MIT Press, pp. 153–162.

*Cybernetics* 95(6): 645–664.

(2006). ARMAR-III: An Integrated Humanoid Platform for Sensory-Motor Control,

A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam & J. Meyer (eds), *From Animals to Animats 8. Proceedings of the Eighth International Conference on the Simulation of Adaptive*

limit cycle systems – from biological inspiration to applications in robotics, *Biological*

A frequency-adaptive hopper robot, *in* M. S. Capcarrere, A. A. Freitas, P. J. Bentley, C. G. Johnson & J. Timmis (eds), *Advances in Artificial Life*, Vol. 3630 of *Lecture Notes*

periodic tasks. It is also easy to implement, with hardly any parameter tuning at all.

**5. Conclusion**

**6. References**

Press.

based on the measured force signal.


Petriˇc, T., Curk, B., Cafuta, P. & Žlajpah, L. (2010). Modeling of the robotic powerball: a

<sup>27</sup> Performing Periodic Tasks: On-Line Learning,

Pikovsky, A., Author, Rosenblum, M., Author, Kurths, J., Author, Hilborn, R. C. & Reviewer

Righetti, L., Buchli, J. & Ijspeert, A. (2006). Dynamic hebbian learning in adaptive frequency

Righetti, L. & Ijspeert, A. J. (2006). Programmable Central Pattern Generators: an application

Ronsse, R., Lefevre, P. & Sepulchre, R. (2007). Rhythmic feedback control of a blind planar

Sato, T., Hashimoto, M. & Tsukahara, M. (2007). Synchronization based control using online

*Biomimetics, 2007. ROBIO 2007. IEEE International Conference on*, pp. 652 –657. Schaal, S. (1999). Is imitation learning the route to humanoid robots?, *Trends in cognitive sciences*

Schaal, S. & Atkeson, C. (1993). Open loop stable control strategies for robot juggling, *Robotics*

Schaal, S. & Atkeson, C. (1998). Constructive incremental learning from only local

Schaal, S., Mohajerian, P. & Ijspeert, A. (2007). *Dynamics systems vs. optimal control –*

Sentis, L., Park, J. & Khatib, O. (2010). Compliant control of multicontact and center-of-mass behaviors in humanoid robots, *Robotics, IEEE Transactions on* 26(3): 483 –501. Spong, M. (1995). The swing up control problem for the acrobot, *Control Systems Magazine,*

Spong, M. & Bullo, F. (2005). Controlled symmetries and passive walking, *Automatic Control,*

Strogatz, S. H. (1986). *The mathematical structure of the human sleep-wake cycle*, Springer-Verlag

Thompson, S. E. & Patel, R. V. (1987). Formulation of joint trajectories for industrial robots using b-splines, *Industrial Electronics, IEEE Transactions on* IE-34(2): 192 –199. Tsuda, S., Zauner, K.-P. & Gunji, Y.-P. (2007). Robot control with biological cells,

Ude, A., Atkeson, C. G. & Riley, M. (2000). Planning of joint trajectories for humanoid robots

van der Pol, B. (1934). The nonlinear theory of electric oscillations, *Proceedings of the Institute*

*Biosystems* 87(2-3): 215 – 223. Papers presented at the Sixth International Workshop on Information Processing in Cells and Tissues, York, UK, 2005 - IPCAT 2005,

*Computer Modelling of Dynamical Systems* . 10.1080/13873954.2010.484237. Petriˇc, T., Gams, A., Tomšiˇc, M. & Žlajpah, L. (2011). Control of rhythmic robotic movements

*Proceedings*, IEEE.

*of Physics* 70(6): 655–655.

Adaptation and Synchronization with External Signals

*Robotics and Automation*.

(3): 233–242.

oscillators, *Physica D* 216(2): 269–281.

juggler, *Robotics, IEEE Transactions on* 23(4): 790 –802.

information, *Neural Computation* 10: 2047–2084.

10.1016/S0079-6123(06)65027-9.

*IEEE Transactions on* 50(7): 1025 – 1031.

New York, Inc., New York, NY, USA.

Information Processing in Cells and Tissues.

using b-spline wavelets, *ICRA*, pp. 2223–2228.

*of Radio Engineers* 22(9): 1051 –1086.

*IEEE* 15(1): 49 –55.

nonholonomic, underactuated, and variable structure-type system, *Mathematical and*

through synchronization with human muscle activity, *2011 IEEE International Conference on Robotics and Automation, ICRA 2011, May 9-13, 2011, Shanghai, China,*

(2002). Synchronization: A universal concept in nonlinear science, *American Journal*

to biped locomotion control, *Proceedings of the 2006 IEEE International Conference on*

design of dynamics and its application to human-robot interaction, *Robotics and*

*and Automation, 1993. Proceedings., 1993 IEEE International Conference on*, pp. 913–918.

*a unifying view*, Vol. 165 of *Progress in Brain Research*, Elsevier, pp. 425 – 445.


24 Will-be-set-by-IN-TECH

Inoue, K., Ma, S. & Jin, C. (2004). Neural oscillator network-based controller for meandering

Jalics, L., Hemami, H. & Zheng, Y. (1997). Pattern generation using coupled oscillators for

*Proceedings., 1997 IEEE International Conference on*, Vol. 1, pp. 179 –184 vol.1. Jin, H.-L., Ye, Q. & Zacksenhouse, M. (2009). Return maps, parameterization, and cycle-wise

Jin, H.-L. & Zackenhouse, M. (2002). Yoyo dynamics: Sequence of collisions captured

Jin, H.-L. & Zacksenhouse, M. (2003). Oscillatory neural networks for robotic yo-yo control,

Jindai, M. & Watanabe, T. (2007). Development of a handshake robot system based on

Kasuga, T. & Hashimoto, M. (2005). Human-robot handshaking using neural oscillators,

Kimura, H., Akiyama, S. & Sakurama, K. (1999). Realization of dynamic walking and

Kralemann, B., Cimponeriu, L., Rosenblum, M., Pikovsky, A. & Mrowka, R. (2008). Phase dynamics of coupled oscillators reconstructed from data, *Phys. Rev. E* 77(6): 066205. Kun, A. & Miller, W.T., I. (1996). Adaptive dynamic balance of a biped robot using

Liu, C., Chen, Q. & Zhang, J. (2009). Coupled van der pol oscillators utilised as central pattern

Matsuoka, K. (1985). Sustained oscillations generated by mutually inhibiting neurons with

Matsuoka, K., Ohyama, N., Watanabe, A. & Ooshima, M. (2005). Control of a giant swing

Matsuoka, K. & Ooshima, M. (2007). A dish-spinning robot using a neural oscillator,

Mirollo, R. E., Steven & Strogatz, H. (1990). Synchronization of pulse-coupled biological

Miyamoto, H., Schaal, S., Gandolfo, F., Gomi, H., Koike, Y., Osu, R., Nakano, E., Wada, Y. &

Morimoto, J., Endo, G., Nakanishi, J. & Cheng, G. (2008). A biologically inspired biped

coupled oscillator model, *Robotics, IEEE Transactions on* 24(1): 185 –191.

Kawato, M. (1996). A kendama learning robot based on bi-directional theory, *Neural*

locomotion strategy for humanoid robots: Modulation of sinusoidal patterns by a

Ljung, L. & Söderström, T. (1986). *Theory and Practice of Recursive Identification*, MIT Press. Mataric, M. (1998). Behavior-based robotics as a tool for synthesis of artificial behavior and

analysis of natural behavior, *Trends in Cognitive Science* 2(3): 82–87.

adaptation, *Biological Cybernetics* 52: 367–376. 10.1007/BF00449593.

robot using a neural oscillator, *ICNC (2)*, pp. 274–282.

*International Congress Series* 1301: 218 – 221.

oscillators, *SIAM J. Appl. Math* 50: 1645–1662.

*2004 IEEE International Conference on*, Vol. 5, pp. 5064 – 5069 Vol.5.

planning of yo-yo playing, *Trans. Rob.* 25(2): 438–445.

*Neural Networks, IEEE Transactions on* 14(2): 317 – 325.

*IEEE/ASME international conference on*, pp. 1 –6.

*Conference on*, pp. 3802 – 3807.

*Conference on*, Vol. 1, pp. 240 –245 vol.1.

10.1023/A:1008924521542.

*'09. Chinese*, pp. 3677 –3682.

*Networks* 9(8): 1281–1302.

124(3): 390–397.

locomotion of snake-like robots, *Robotics and Automation, 2004. Proceedings. ICRA '04.*

robotic and biorobotic adaptive periodic movement, *Robotics and Automation, 1997.*

by a restitution effect, *Journal of Dynamic Systems, Measurement, and Control*

a handshake approaching motion model, *Advanced intelligent mechatronics, 2007*

*Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International*

running of the quadruped using neural oscillator, *Autonomous Robots* 7: 247–258.

neural networks, *Robotics and Automation, 1996. Proceedings., 1996 IEEE International*

generators for quadruped locomotion, *Control and Decision Conference, 2009. CCDC*


Yuki Funabora1, Yoshikazu Yano2, Shinji Doki1 and Shigeru Okuma1

**Structure Changes Without Model Identification** 

**Autonomous Motion Adaptation Against** 

It is expected that humanoid robots provide various services to help human daily life such as household works, home security, medical care, welfare and so on(Dominey et al., 2007; Okada et al., 2003; 2005). In order to provide various services, humanoids have multi degree-of-freedom(DOF), sophisticated and complicated structure. These humanoid robots will work under human living environments which are not definable beforehand. So humanoids have to provide their given services under not only the designed environments but also unknown environments. Under unknown environments, robots cannot perform as planned, and they may fall or collide with obstacles. These impacts will wreak several unexpected structure changes such as gear cracks, joint locking, frame distortions and so on. Because of the designed motions are optimized to the robot structure, if the robot structure has changed, the services from robots cannot be provided. Because general users have no expertise knowledge of robots, thus, quick repairs under human living environments cannot be expected. Even in that case, it is expected that the robots should provide services to help human daily life as possible. In the case the humanoid robots cannot get rapid repair service, they have to provide the desired services with their broken body. In addition, using tools to provide some services can be considered as one of the structure changes. Therefore, it is necessary for future humanoids to obtain new motions which can provide the required

We propose an autonomous motion adaptation method which can be applied to sophisticated and complicated robots represented by humanoids. As a first step, we deal with the simple services based on trajectory control; services can be provided by following the correct path designed by experts. When robot structure has changed, achieving the designed trajectories on changed structure is needed. As the conventional methods, there are two typical approaches. One is the method based on model identification (El-Salam et al., 2005; Groom et al., 1999). Robots locate the occurred changes, identify the changed structure, recalculate inverse kinematics, and then obtain the proper motions. If the changed structure is identified, inverse kinematics leads the proper motions for new properties of changed structure. However, it is so difficult to identify the complicated structure changes in sophisticated robots. In additions, the available solving methods of inverse kinematics for multi DOF robots is non-existent according to the reference (The Robotics Society of Japan, 2005). So model identification method cannot be applied for humanoids. Another approach is

**1. Introduction**

services with changed structure.

<sup>1</sup>*Nagoya University*

*Japan*

**2**

<sup>2</sup>*Aichi Institute of Technology*

