**2. Characteristics, behavior and benefits of augmented reality**

Augmented reality is a technology that allows mixing user real environments with virtual environments. Azuma (1997) defines augmented reality as the overlay of virtual objects, computer generated, in a real environment, generating a mixed environment that can be perceived by some technological device in real time. Figure 1 shows two examples of applications of augmented reality: a sandwich and a butterfly overlayed on the real environment.

Fig. 1. Examples of augmented reality environments (Fischer et al, 2006)

The creation of an augmented reality environment requires a computer, a webcam and paper cards (Azuma, 2001). First, a webcam (connected to the computer) captures images of the real environment in search of cards. These cards have printed symbols that are cataloged

Contributions of an Augmented Reality Musical System

Fig. 3. Possibilities of interaction with augmented reality cards

for the Stimulation of Motor Skills in Music Therapy Sessions 279

The presence of cards in the webcam view field triggers the overlaying of the virtual object associated with the card and the real world captured by the webcam. The manipulation of the card in the real environment also moves the virtual object. If the individual drags the card, the virtual object associated with it moves along. This augmented reality feature facilitates the process of communication of children with learning difficulties or people with physical disabilities in many different difficulty levels, allowing their intellectual and motor development. For physically disabled with a moderate or severe physical capacity degree, the use of conventional musical instruments is restricted (Louro et al, 2005). Interacting with virtual musical instruments in the real world can facilitate interaction in music therapy especially when the therapy takes place in a specialized center for motor rehabilitation.

The next section presents GenVirtual, indications for its use in music therapy sessions and

We developed an augmented reality musical system (GenVirtual) that allows creation, improvisation and music reproduction such as composition and tunes playing. Also the

GenVirtual adds virtual musical instruments to the real world. Colored three-dimensional cubes represent the instruments. Cubes are able to simulate the sounds of wind, string and percussion instruments. Interaction occurs through GenVirtual cards with musical symbols. Cards replace the keyboard for music composition and are low cost since they can be

GenVirtual processes the images captured by a webcam to identify the cards on the table. The webcam is connected to a computer and strategically positioned on top of the table (Figure 4). Symbols on the cards are detected and associated with a sound. To execute a sound, the user just hides a card with his/her hand, without necessarily touching them with

preliminary results of using GenVirtual with patients with motor disabilities.

system enables the user to hear sounds and music, and musical memory games.

printed on plain paper in various colors and sizes (Figure 4).

• Stringed musical instruments (piano, violin and guitar).

• Percussion musical instruments (plate, box, tambourine and triangle);

**3. GenVirtual musical system** 

Cards to interaction of GenVirtual:

• Wind musical instrument (flute);

• Musical notes (Dó, Ré, Mi, Fá, Sol, Lá e Si).

in the computer software database. Symbols found on the cards are analyzed and interpreted by a software application. Symbols interpretation produces 3D virtual objects that are combined with the user's actual environment. The mixture of two environments is displayed either on a video monitor or on special glasses or a helmet display.

Possibility of transporting virtual objects to the user's physical space (for overlapping) facilitates interaction, which no longer occurs through a single specific component (conventional devices such as mouse and keyboard), but with the whole environment that surrounds the user. Thus, the virtual objects can be manipulated through actions involving tangible and multimodal touch, gestures and voice, making the process easier and more intuitive without using special equipment. Figure 2 shows the basic components of an augmented reality environment.

Fig. 2. Basic components of an augmented reality environment

Ability to carry virtual objects into the real world has fostered new forms of humancomputer interaction, making the process more natural and intuitive. This feature of augmented reality can facilitate the interaction of individuals with severe motor problems. The handling of virtual objects can occur naturally, dragging or touching an object with fingers (Figure 3.a), hands (Figure 3b) or feet (Figure 3c), without necessarily using peripheral or adapters to interact.

Often, computer adaptations are created to attend specific needs of individuals with disabilities. However, depending on the disability specificity, some individuals may require specific resources hindering the therapeutic process, promoting even higher cost for design and use. Augmented reality allows environments to build economically viable systems compared to other environments that use expensive electronic devices such as helmets and electronic gloves. With a computer (for software processing), a webcam (to capture the real environment) and a video monitor (to display mixed environments) a simple augmented reality system can be created. This system has a variety of potential applications to work with individuals with various disabilities (Richard et al, 2007), (Assis and Lopes, 2008).

in the computer software database. Symbols found on the cards are analyzed and interpreted by a software application. Symbols interpretation produces 3D virtual objects that are combined with the user's actual environment. The mixture of two environments is

Possibility of transporting virtual objects to the user's physical space (for overlapping) facilitates interaction, which no longer occurs through a single specific component (conventional devices such as mouse and keyboard), but with the whole environment that surrounds the user. Thus, the virtual objects can be manipulated through actions involving tangible and multimodal touch, gestures and voice, making the process easier and more intuitive without using special equipment. Figure 2 shows the basic components of an

Ability to carry virtual objects into the real world has fostered new forms of humancomputer interaction, making the process more natural and intuitive. This feature of augmented reality can facilitate the interaction of individuals with severe motor problems. The handling of virtual objects can occur naturally, dragging or touching an object with fingers (Figure 3.a), hands (Figure 3b) or feet (Figure 3c), without necessarily using

Often, computer adaptations are created to attend specific needs of individuals with disabilities. However, depending on the disability specificity, some individuals may require specific resources hindering the therapeutic process, promoting even higher cost for design and use. Augmented reality allows environments to build economically viable systems compared to other environments that use expensive electronic devices such as helmets and electronic gloves. With a computer (for software processing), a webcam (to capture the real environment) and a video monitor (to display mixed environments) a simple augmented reality system can be created. This system has a variety of potential applications to work with individuals with various disabilities (Richard et al, 2007), (Assis and Lopes, 2008).

displayed either on a video monitor or on special glasses or a helmet display.

Fig. 2. Basic components of an augmented reality environment

augmented reality environment.

peripheral or adapters to interact.

Fig. 3. Possibilities of interaction with augmented reality cards

The presence of cards in the webcam view field triggers the overlaying of the virtual object associated with the card and the real world captured by the webcam. The manipulation of the card in the real environment also moves the virtual object. If the individual drags the card, the virtual object associated with it moves along. This augmented reality feature facilitates the process of communication of children with learning difficulties or people with physical disabilities in many different difficulty levels, allowing their intellectual and motor development. For physically disabled with a moderate or severe physical capacity degree, the use of conventional musical instruments is restricted (Louro et al, 2005). Interacting with virtual musical instruments in the real world can facilitate interaction in music therapy especially when the therapy takes place in a specialized center for motor rehabilitation.

The next section presents GenVirtual, indications for its use in music therapy sessions and preliminary results of using GenVirtual with patients with motor disabilities.
