**5. Evaluation of GenVirtual in music theraphy rehabilitation**

Experiments with GenVirtual were performed in music therapy intervention. Five markers formed by symbols and colors of respective musical notes (C, D, E, F and G). Figure 10 shows the music therapist interacting with GenVirtual.

A video monitor was used to visualize the animation of virtual cubes. The webcam was placed on top of the monitor to capture images of cards on the table. As can be seen in Figure 8, the music therapist obstructed cards with her hands without using adapters.

Fig. 10. Evaluation of GenVirtual by music therapist

After testing, the music therapist noted that besides the possibility of using GenVirtual in music therapy, it can bring benefits to individuals with disabilities, since it is a facilitator and motivational tool in the rehabilitation process. As one example:

• Arm extension exercises: the goal is to reach a virtual object. With cards' flexibility, specific challenges can be created for each patient according to their motor limitations.

Contributions of an Augmented Reality Musical System

Fig. 11. Use GenVirtual by patients of AACD

**5.2 Results** 

objectives for the intervention.

for the Stimulation of Motor Skills in Music Therapy Sessions 285

The activity was held in a few moments as an expression of patient musicality (through motor activity). Also, the music therapist used an electronic keyboard to accompany the patient's musical activity. The music therapist process is a sequence of interventions and not simply a single act or therapeutic maneuver alone. A therapist-patient relationship was

The type of therapeutic process involved in this clinical trial, where the therapist used the virtual tool, depended on the type of patient and the nature of his/her problem. According to Benezon (1998), there are different types of proceedings, as follows: developmental, educational, interpersonal, artistic, creative and scientific process which can be applied as such individual processes and are not mutually exclusive, but instead overlap, with greater

According to the results collected in the Quiz and to the patient and family feedback, all patients participated actively in the therapy with the strategies suggested by the therapist showing interest in the activities. The musical cards triggered curiosity and demanded concentration and reasoning during the activity. All patients achieved the proposed

GenVirtual fostered the development of therapeutic activities using virtual reality with the computer such as playing percussion instruments with open hands, for example, triangle plate and without the need of a stick, which cannot be used by children with severe upper limb motor problems. As of the Benezon (1998), music therapy analysis approaches, the four

created, even for those cases where the therapy period was relatively brief.

or less emphasis on the process, as presented in these cases.


#### **5.1 Methodology**

This research was approved by the responsible Scientific Review Committee: the Committee for Scientific Affairs of the Association of Assistance Children with Disabilities (AACD) under Protocol 089/10. Several experiments were performed in the AACD music therapy field in order to test the impact of GenVirtual when treating two patients with cerebral palsy and three patients with neuromuscular diseases.

The patients' ages were established between five and six years old, with individuals of both sexes, being 01 (one) girl and 04 (four) boys. Patients underwent a trial test with the specific goal of rehabilitation for upper limb by motivation stimulation of musical function through interaction with music. The process began with the implementation of the User Interaction Satisfaction Questionnaire (QUIZ), repeated at the end of the interaction. The objectives of the experiments were:


During the experiments with patients, we used a computer, a sound box, a webcam with a tripod and sixteen cards: seven cards representing seven musical notes (C, D, E, F, G, A and B), three cards representing instruments (piano, violin and guitar), a marker representing a wind instrument (flute) and four markers representing the percussion instruments (cash, plate, drum and tambourine). The experiments were recorded as illustrated in Figure 11.

Initially, the music therapist prepared a linear design with cards on the table. The goal of linear motor planning for these patients was to stimulate the active pendulum motion to the shoulders, push your arm back and forth so as to touch the virtual objects.

After the demonstration of active music therapist's tool operation, the patient spontaneously was motivated and interacted with the system, making the shoulder and arm extension to reach the cards on the table and thus play the virtual objects and enjoy the unique and individual musical composition. The music therapist encouraged the activity by asking the patient to interact with the cards, to switch the left arm with the right, to exercise and active pendulum motion with left and right shoulders.

• Wrist flexion exercises: support the forearm on the table and bend wrist up and down

• Repetitive motor training: a sequence of motor movements facilitated by the musical

• Visual perception exercises: identifying musical elements from the representation of

• Auditory perception exercises: identify high and low tones (listen to sound) and

• Reproduction and development of more sophisticated music pieces, experience the "music making" to amplify their cognitive functions like attention, concentration and

This research was approved by the responsible Scientific Review Committee: the Committee for Scientific Affairs of the Association of Assistance Children with Disabilities (AACD) under Protocol 089/10. Several experiments were performed in the AACD music therapy field in order to test the impact of GenVirtual when treating two patients with cerebral palsy

The patients' ages were established between five and six years old, with individuals of both sexes, being 01 (one) girl and 04 (four) boys. Patients underwent a trial test with the specific goal of rehabilitation for upper limb by motivation stimulation of musical function through interaction with music. The process began with the implementation of the User Interaction Satisfaction Questionnaire (QUIZ), repeated at the end of the interaction. The objectives of

• Assessing motivation, satisfaction and enthusiasm of patient to interact with GenVirtual. • Observe the motor effort motivated by using virtual objects that allow music

During the experiments with patients, we used a computer, a sound box, a webcam with a tripod and sixteen cards: seven cards representing seven musical notes (C, D, E, F, G, A and B), three cards representing instruments (piano, violin and guitar), a marker representing a wind instrument (flute) and four markers representing the percussion instruments (cash, plate, drum and tambourine). The experiments were recorded as illustrated in Figure 11.

Initially, the music therapist prepared a linear design with cards on the table. The goal of linear motor planning for these patients was to stimulate the active pendulum motion to the

After the demonstration of active music therapist's tool operation, the patient spontaneously was motivated and interacted with the system, making the shoulder and arm extension to reach the cards on the table and thus play the virtual objects and enjoy the unique and individual musical composition. The music therapist encouraged the activity by asking the patient to interact with the cards, to switch the left arm with the right, to exercise and active

shoulders, push your arm back and forth so as to touch the virtual objects.

"touching" virtual objects to execute the sounds.

these elements by colourful virtual objects.

identify familiar melodies (listen to music);

and three patients with neuromuscular diseases.

memory game.

memory.

**5.1 Methodology** 

the experiments were:

expression.

• Observe the ability to concentrate.

• Observe family interaction during the intervention.

pendulum motion with left and right shoulders.

Fig. 11. Use GenVirtual by patients of AACD

The activity was held in a few moments as an expression of patient musicality (through motor activity). Also, the music therapist used an electronic keyboard to accompany the patient's musical activity. The music therapist process is a sequence of interventions and not simply a single act or therapeutic maneuver alone. A therapist-patient relationship was created, even for those cases where the therapy period was relatively brief.

The type of therapeutic process involved in this clinical trial, where the therapist used the virtual tool, depended on the type of patient and the nature of his/her problem. According to Benezon (1998), there are different types of proceedings, as follows: developmental, educational, interpersonal, artistic, creative and scientific process which can be applied as such individual processes and are not mutually exclusive, but instead overlap, with greater or less emphasis on the process, as presented in these cases.

#### **5.2 Results**

According to the results collected in the Quiz and to the patient and family feedback, all patients participated actively in the therapy with the strategies suggested by the therapist showing interest in the activities. The musical cards triggered curiosity and demanded concentration and reasoning during the activity. All patients achieved the proposed objectives for the intervention.

GenVirtual fostered the development of therapeutic activities using virtual reality with the computer such as playing percussion instruments with open hands, for example, triangle plate and without the need of a stick, which cannot be used by children with severe upper limb motor problems. As of the Benezon (1998), music therapy analysis approaches, the four

Contributions of an Augmented Reality Musical System

and Personalization, pp. 375-380.

*Interaction*, Vol. 15, No. 1, pp. 283-300.

*Applications*, Vol. 978, pp. 29-33.

*Neurology Report*, Vol. 26, No. 2, pp. 62-7.

*Cybernetics*, Taipei – Taiwan, pp. 1636 - 1641.

*Reality,* pp. 205-208.

1, pp. 1-8.

82-101.

9, No. 3, pp. 152-169.

for the Stimulation of Motor Skills in Music Therapy Sessions 287

Botella, C. ; Lopez, B.J. ; Quero, S. ; Baños, R.; Garcia-Palacios, A. (2010). Treating cockroach phobia with augmented reality. *Behavior Therapy*, Vol. 41. No. 3, p. 401-413. CETIC (2010). Centro de Estudos sobre Tecnologias da Informação e Comunicação. Pesquisa

Chen, L.; Pu, P. (2010). Eye-Tracking Study of User Behavior in Recommender Interfaces. In:

Fischer, J; Bartz, D.; Strasser, W. (2006). Enhanced Visual Realism by Incorporating Camera

Garbin, T. R.; Dainese, C. A.; Kirner, C. (2006). Sistema de Realidade Aumentada para

Garbin, T. R.; (2009). AmCARA - Ambiente e Comunicação Alternativo com Realidade

Golomb, M. McDonald B.C.; Warden S.J.; Yonkman J.; Saykin A.J.; Shirley B.; Huber

Gorman, M.; Lahav, A.; Saltzman, E.; Betke, M. (2007). A camera-Based Music-Making Tool for Physical Rehabilitation. *Computer Music Journal*, Vol. 31, No. 2, pp. 39-53. Hoffman, G. H.; Carlin, A.; Furness, T. A. (2003). Interfaces that Hal: Coupling real and

Holden M. K.; Dyar, T. (2002). Virtual Environment Training: a new tool for rehabilitation.

Hua, Z.; Lieh-ng, W. (2010) Speech Recognition Interface Design for In-Vehicle System. *In:* 

Ikuta, C. (2009). *Métodos de Intervenção Musicoterapêutica e suas Aplicações*. In: Nascimento, M.

Jacob, R. (1991). The use of eye movements in Human-Computer Interaction Techniques:

Kirner, C.; Zorzal, E.R.; Kirner, T.G. (2006). Case Studies on the Development of Games

Leme R.J.S.A. (2009). Neurofisiologia da Música. In: Nascimento. M. *Musicoterapia e a* 

*Reabilitação do Paciente Neurológico*, pp. 30-42, São Paulo: Memmon.

Pré-Simpósio VIII Simposium on Virtual Relaity, pp. 289-297.

XX Simpósio Brasileiro de Informática em Educação, pp. 1-10.

*Digital Audio Effects (DAFX-03),* London-UK, pp.1-5.

sobre o Uso das Tecnologias da Informação e Comunicação no Brasil entre 2005 e 2009. Núcleo de Informação e Coordenação do Ponto BR: São Paulo, pp. 12-203. Constanza, E.; Shelley, S.B.; Robinson, J. (2003). Introducing Audio-Touch Tangible User

Interface for Music Composition and Performance*. In: VI International Conference on* 

User Modeling, Adaptation, and Personalization In User Modeling, Adaptation,

Image Effects. *IEEE and ACM International Symposium on Mixed and Augmented* 

Trabalho com Crianças com Necessidades Especiais. In: Tori, R.; Kirner, C.; Siscoutto, R. *In: Fundamentos e Tecnologias de Realidade Virtual e Aumentada*. Livro do

Aumentada: O acesso do deficiente motor severo a softwares e Web. In: Anais do

M.; Rabin B.; Abdelbaky M.; Nwosu M.E.; Barkat-Masih M.; Burdea G.C. (2010). In-Home Virtual Reality Videogame Telerehabilitation in Adolescents with Hemiplegic Cerebral Palsy. *Archives in Physical Medicine Rehabilitation*, Vol. 91, N0.

virtual objects to cure spider phobia. *International Journal of Human-Computer* 

*ACM International Conference on Automotive User Interfaces and Interactive Vehicular* 

Musicoterapia e a Reabilitação do Paciente Neurológico. São Paulo: Memmon, pp.

What You Look At is What You Get. *ACM Transactions on Information Systems*, Vol.

Using Augmented Reality. I*n: IEEE International Conference on Systems, Man, and* 

main methods of music therapy: improvisation, re-creation, composition and listening, can be applied through using GenVirtual. For the clinical cases presented, the model re-creation where the patient performs, reproduces, transforms or interprets the piece of music or musical model as a whole was the most commonly used method
