**2.1. Controlling strategies**

Computational Intelligence in Electromyography Analysis – 410 A Perspective on Current Applications and Future Challenges

some of the lost functionality by means of artificial limbs.

*with sensory as well as motor functions"* 

development of artificial limbs for amputees [10].

**2. Myoelectric prostheses** 

real one. Also, the research team believes that, since real upper limb prostheses are relatively heavy and can become uncomfortable and cumbersome, especially during the first stages of fitting, the use of a virtually weightless and fully controllable device can help reducing the

The human body has been considered a perfect machine, in which all parts work in harmony one with each other. Most of us can control this "machine" without much effort, until some disturbance caused by disease or injury results in loss of some of its functionality. The absence of limbs caused by trauma or congenital disorders, can affect our lives profoundly. Simple tasks such as walking or dressing can become extremely difficult or even impossible to execute. There is no doubt that the best solution for the loss of a limb might be the development of some kind of genetic manipulation that stimulates the regeneration of tissue. However, while this is not possible, the best we can do is to restore

For centuries, mankind seek ways to replace lost limbs by mechanical devices. Several ancient designs can be found in museums and libraries. However, the first artifact to be formally named artificial limb was a Roman prosthesis, made of wood and bronze, which appeared around 300 BC [9]. During the Middle Ages, while the poor wore "wooden legs", which were simple, inexpensive and stable, the rich nobles used devices made of iron, which were more decorative than functional. In 1818 Peter Ballif designed the first prosthesis actuated by movements of healthy parts of the body. Before this, the upper limb prostheses were heavy and depended on an able hand for operation [10]. Thereafter, a number of experiments have been carried out seeking the "perfect prosthesis", a device that could be similar to what Wolfe had visualized in 1952 in his book "*New York: Random House"* [11]:

*"They had perfected an artificial limb superior in many ways to the real thing, integrated into the nerves and muscles of the stump, powered by a built-in atomic energy plant, equipped* 

As we know, to date, this prediction has yet to be completely materialized, but much has been done since then. Due to the great number of casualties of World War II, the government of the United States created in 1945 a program of research and development from which scientists and engineers were deeply involved in projects aimed at the

Another fact that lead to the acceleration of the researches in the area, was the large number of congenital defects caused by the use of a drug called Thalidomide. As describe by Soares [10], it was synthesized by the German laboratory Chemie Grünenthal in 1957 and marketed worldwide between 1958 and 1962. This drug was prescribed to minimize sickness during pregnancy. The Thalidomide consumers were not warned that the drug could exceed the placenta affecting the fetus. This oversight had a catastrophic effect: drug abuse, especially

great physical and metal effort usually necessary, especially in the first trials.

The control of prostheses can be considered one of the most interesting challenges related to prosthetics. Ideally, a prosthetic limb should be controlled without any effort from the user, similar to the subconscious control of a natural limb.

Currently, there are two main strategies for controlling artificial limbs: biomechanical and bioelectrical. In the first, the motion of parts of the body results in the activation of the limb, whereas, in the latter, biosignals, generated from the electrical activity of muscles, are detected and interpreted in order to generate commands for controlling the prosthesis. Nevertheless, there is ongoing research seeking other forms of control based on more natural strategies, such as those that employ brain or neuronal activity together with sensory feedback [5-7, 12-14].

As described earlier, the first prostheses were generally passive devices that relied on intact parts of the body for their positioning and controlling. This extremely successful design allowed the user to control the device so that the movement of a part of the body was reflected in movements of parts of the prosthesis. Despite some modifications, this design remains basically the same nowadays, being the most popular control mechanism among users [10]. The reasons for such success are not well established, but according to Doeringer and Hogan [15] some of the key factors are: it results in a relatively inexpensive prosthesis; the final prosthesis is not too heavy; after training, the user begins to use the prosthesis as a natural extension of his body, having, for example, the notion of weight and size of the prosthetic limb. Kruit and Cool [16] described the main drawbacks: the mechanism of harnesses used to propagate the movements of the body is usually uncomfortable; the movement of the prosthesis requires significantly large forces; the number of control inputs is limited and thus the number of degrees of freedom of the prosthesis is also limited.

An alternative to the Body-Powered control is to employ the myoelectric control, which uses the electrical activity of muscle contraction (electromyographic signal) as a primary source of control. The prostheses that use this type of control typically do not require cables and, in some situations, there is no need for suspension straps. The operation of a myoelectric device can be summarized as follows: the brain sends commands, i.e., neuronal impulses that travel through nerves and reach the endplate of a given muscle, which, in turn, causes muscle contraction; The electrical muscle activity is then captured by electrodes (normally in a socket attached to the stump), interpreted by customized programs in a microcontroller, and used to activate the actuator of the prosthesis.

Virtual and Augmented Reality: A New Approach to Aid Users of Myoelectric Prostheses 413

electrodes positioned on the skin surface, and needle electrodes inserted in relevant positions of the muscle. In both cases the electrodes produce a difference of potential relative to a reference (typically another electrode located elsewhere on the body). This voltage is the result of an asynchronous activation of hundreds of muscle fibers. The signal is similar to a random noise whose amplitude is modulated by a voluntary input. Its shape depends on variables such as strength and speed of activation, positioning and types of electrodes used in its detection, electronic circuits used for acquisition, amplification and processing [20]. These factors make the translation of myoelectric activity into commands for a prosthetic limb a challenge. Moreover, the generation of myoelectric patterns must be learned by the user, and this is a task which requires concentration, regular training and a

A common way of training an individual to generate myoelectric patterns is by using feedback software, which provides the user with visual feedback about the relation between the forces associated to the contraction with the amplitude of the generated myoelectric signal. The main drawback of this strategy is that it does not give the user feedback information or sense of proprioception. Recent studies have suggested that virtual and augmented reality [21] may be a relevant tool to address the limitations of conventional training techniques. The main advantage of this technique is that it can simulate the physical presence of the artificial limb in the real world, as well as in imaginary worlds. Moreover, some simulations may include additional sensory information, such as sound through speakers or headphones. It is also possible to include tactile information, generally known as

Virtual Reality (VR) can be defined as an advanced computer interface where the user can, in real time, navigate within a tridimensional environment interacting with its virtual objects. Such interaction is achieved in a very intuitive and natural way. To do so,

In order to illustrate this concept, consider Figure 2, in which a user is shown standing inside a research laboratory. However, since she is equipped with multisensory devices (Head Mounted Display – HMD and hand (glove) sensors), a computer-based system

The system, known as BioSimMER© (from Sandia National Laboratories) [23], is used to train rescue personnel to respond to terrorist attacks. The screen on the top shows the working environment displayed only for the eyes of the health professional and the virtual patient exhibits realistic symptoms. Such facilities are not supported by traditional computer

Therefore, to achieve the high level of natural interface required by VR systems, it is important to provide the user with the feeling of immersion and the ability for interaction. To reach such requirements, VR developers must guarantee: 1) Real life 3D object images

provides her the feeling of being steeped into a different environment.

great amount of physical effort.

force feedback, for the individual.

**3. Virtual and augmented reality** 

multisensory devices are supplied [22].

interfaces.

Many myoelectric prostheses employed a type of control called "two-site two-states", from which a pair of electrodes is placed on two distinct muscles. The contraction of one of these muscles produces the opening of the hand. The antagonist muscle is used in the same way to control the closing of the hand. As pointed out by Scott and Parker [8], this approach works in a manner analogous to the human body, i.e., two antagonistic muscles (or group of muscles) control the movement of a joint. However, as patients must learn to generate independent contractions of the muscles, which requires a high degree of concentration, the training can be lengthy and demand a lot of mental effort. There are also some situations in which it is not possible to find two available groups of muscles, or there might be more than one joint to be controlled. For these situations other controlling approaches have been developed [17]. For instance, the "one-site three-states", from which a little contraction of a muscle produces the closing of the hand, a strong contraction open it, and the lack muscle activity stops the hand. Figure 1 shows an example of a hand prosthesis controlled by electromyographic signals captured by four pairs of dome electrodes, distributed around the residual limb ([18]).

**Figure 1.** An experimental setup for a myoelectric prosthesis, developed at the University of New Bruswick, Canada (extracted from [18]).

Currently there are a number of methods using proportional control based on the electrical activity of muscles to control the speed, torque and position of prosthetic joints. However, due to the nature of the myoelectric signal, errors and inaccuracies may occur [19]. Myoelectric signals can be detected using basically two types of electrodes: surface electrodes positioned on the skin surface, and needle electrodes inserted in relevant positions of the muscle. In both cases the electrodes produce a difference of potential relative to a reference (typically another electrode located elsewhere on the body). This voltage is the result of an asynchronous activation of hundreds of muscle fibers. The signal is similar to a random noise whose amplitude is modulated by a voluntary input. Its shape depends on variables such as strength and speed of activation, positioning and types of electrodes used in its detection, electronic circuits used for acquisition, amplification and processing [20]. These factors make the translation of myoelectric activity into commands for a prosthetic limb a challenge. Moreover, the generation of myoelectric patterns must be learned by the user, and this is a task which requires concentration, regular training and a great amount of physical effort.

A common way of training an individual to generate myoelectric patterns is by using feedback software, which provides the user with visual feedback about the relation between the forces associated to the contraction with the amplitude of the generated myoelectric signal. The main drawback of this strategy is that it does not give the user feedback information or sense of proprioception. Recent studies have suggested that virtual and augmented reality [21] may be a relevant tool to address the limitations of conventional training techniques. The main advantage of this technique is that it can simulate the physical presence of the artificial limb in the real world, as well as in imaginary worlds. Moreover, some simulations may include additional sensory information, such as sound through speakers or headphones. It is also possible to include tactile information, generally known as force feedback, for the individual.
