2.2. Technology of the work of the virtual reality and game devices with the use of the electromyographic bracelet

The control of the virtual reality happens as follows: the myoelectric device of reading in the form of a bracelet (or a similar design the device) is put on a forearm or an arm, or other parts of an extremity, as the example, legs, occurs calibration and control of a bracelet, it can be made on the computer, the phone (Figure 4), as also without the computer or the phone.

Points of virtual reality consist at least of different types of lenses, the display, the details of the case, the computing system, the accelerometer, the gyroscope, and the sensor of the wireless communication (Bluetooth) (Figure 5).

Communication between the bracelet and points of virtual reality is been organized by means of any wireless communication, in our case are used by Bluetooth at what by Bluetooth of a bracelet it is ready as conducted, and Bluetooth of points of virtual reality as the master. The myoelectric device for reading, in the form of the bracelet (or the similar design of the device), carries out registration and filtration of the electromyogram (EMG), defines position of the hand, depending on it sends the corresponding command to points of virtual reality, or obtains information on feedback. It is important that the myoelectric device of reading besides the gripper given according to numbers sends to points information from the gyroscope and the accelerometer that allows to define position of the hand in space better. The arrangement of the myoelectric device for reading not only on hands but also on legs, the back, and the neck

Figure 4. The virtual reality and bracelet.

neural network, it is possible to wear not only on a forearm and to take off data from a forearm, but also for management of the hand to take off data from a brachium that considerably dilated opportunities at a prosthetic repair by bionic prostheses of arms (of course, there is a restriction on number of gripper and features of management, at some disabled people

different grippers are defined variously in connection with features of a stump).

Figure 3. The scheme of realization of the data acquisition in the form of the myoelectric system.

Figure 2. Example of the arrangement of sensors on the hand artificial limb.

96 Biofeedback

Figure 5. Example of the scheme of the device of points of the virtual reality.

is possible allowing to define better an arrangement of other parts of the body in space. In addition, points of virtual reality are capable to transfer information of feedback and to start the vibromotor on the myoelectric device for reading that allows to create feedback; also, additional arrangement and other sensors of feedback and sensors is possible.

installation of communication between devices. The myoelectric device of reading carries out registration and filtration of the electromyogram (EMG), defines position of a hand and, depending on him, sends to model the machine the corresponding team. List of tasks: F—the machine begins advance will not receive the next task, B—model the machine begins the movement back will not receive the next task yet, L—model the machine begins the movement on the left so far this task comes, after its termination continues action which executed before receiving commands (advance, back, inaction), R—model the machine begins the movement to the right so far this task comes, after its termination continues action which executed before

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games…

http://dx.doi.org/10.5772/intechopen.74794

99

Model of the machine receives the task, and the chip processes it depending on the task and carries out manipulations over motors, to begin rotation, to change the direction of rotation, and to stop. After implementation of the current command, the device is ready to perform the following. Also, from model, the machine can come to the bracelet information of feedback

Also it is possible a realization chance of the systems of rehabilitation with points of virtual reality, and without them. Namely, when there is a removal of data, for example, on gripper from a forearm or an arm of a hand, and transfer on the computer or phone for performance of certain gripper or three-dimensional motions, movements, on the computer or phone is started the program which obtains information and signals about successful or unsuccessful perfor-

receiving commands (advance, back, inaction), S—the model stop the machine.

Figure 6. Example of the scheme of setup of the machine or other executive mechanisms.

and start the vibromotor on the bracelet or other types of sensors.

mance of a task, giving of a signal of feedback on a bracelet is also possible.

Management of radio-controlled model of the machine happens as follows: the myoelectric device of reading in the form of a bracelet (or a similar design the device) is put on the forearm or the arm, there is the calibration and setup of the device of the bracelet, it can be made on the computer, the phone (Figure 6), as also without the computer or the phone.

The radio-controlled model consists at least of a chip, sensor of the wireless communication (Bluetooth), and engines of motors (Figure 7).

Communication between a bracelet and radio-controlled model the machine is organized by means of any wireless communication, in our case are used by Bluetooth at what by Bluetooth of a bracelet it is ready as the master, and machine model Bluetooth as conducted. After start of both devices, with connection without intermediaries, the device of a wireless communication of a bracelet is connected to the communication device of the executive mechanism; in our case communication requires knowledge by the master of the name and password of the communication device of the executive mechanism. In this case devices are connected through the intermediary device (the computer, phone), and then the intermediary device participates in

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games… http://dx.doi.org/10.5772/intechopen.74794 99

Figure 6. Example of the scheme of setup of the machine or other executive mechanisms.

is possible allowing to define better an arrangement of other parts of the body in space. In addition, points of virtual reality are capable to transfer information of feedback and to start the vibromotor on the myoelectric device for reading that allows to create feedback; also,

Management of radio-controlled model of the machine happens as follows: the myoelectric device of reading in the form of a bracelet (or a similar design the device) is put on the forearm or the arm, there is the calibration and setup of the device of the bracelet, it can be made on the

The radio-controlled model consists at least of a chip, sensor of the wireless communication

Communication between a bracelet and radio-controlled model the machine is organized by means of any wireless communication, in our case are used by Bluetooth at what by Bluetooth of a bracelet it is ready as the master, and machine model Bluetooth as conducted. After start of both devices, with connection without intermediaries, the device of a wireless communication of a bracelet is connected to the communication device of the executive mechanism; in our case communication requires knowledge by the master of the name and password of the communication device of the executive mechanism. In this case devices are connected through the intermediary device (the computer, phone), and then the intermediary device participates in

additional arrangement and other sensors of feedback and sensors is possible.

Figure 5. Example of the scheme of the device of points of the virtual reality.

98 Biofeedback

computer, the phone (Figure 6), as also without the computer or the phone.

(Bluetooth), and engines of motors (Figure 7).

installation of communication between devices. The myoelectric device of reading carries out registration and filtration of the electromyogram (EMG), defines position of a hand and, depending on him, sends to model the machine the corresponding team. List of tasks: F—the machine begins advance will not receive the next task, B—model the machine begins the movement back will not receive the next task yet, L—model the machine begins the movement on the left so far this task comes, after its termination continues action which executed before receiving commands (advance, back, inaction), R—model the machine begins the movement to the right so far this task comes, after its termination continues action which executed before receiving commands (advance, back, inaction), S—the model stop the machine.

Model of the machine receives the task, and the chip processes it depending on the task and carries out manipulations over motors, to begin rotation, to change the direction of rotation, and to stop. After implementation of the current command, the device is ready to perform the following. Also, from model, the machine can come to the bracelet information of feedback and start the vibromotor on the bracelet or other types of sensors.

Also it is possible a realization chance of the systems of rehabilitation with points of virtual reality, and without them. Namely, when there is a removal of data, for example, on gripper from a forearm or an arm of a hand, and transfer on the computer or phone for performance of certain gripper or three-dimensional motions, movements, on the computer or phone is started the program which obtains information and signals about successful or unsuccessful performance of a task, giving of a signal of feedback on a bracelet is also possible.

channel. Whether from the channel it was possible to select only information on that reduction of the muscle. Further on what muscles have been reduced and what are not present to distinguish concrete gripper. The algorithm of preprocessing allows, quite precisely defining

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games…

http://dx.doi.org/10.5772/intechopen.74794

101

When algorithms were tested, two main variants were developed; the first one used simple preprocessing of the signal. Moreover, the second neural network hash function as additional

All gestures were recognized with the probability greater than 70%, of these, two gestures were recognized with a probability of less than 80%, six with recognized with probability <90%, but >80%, eight gestures >90% and two with the probability of a business to 100%.

This method has been tested for real-time amputations. Its accuracy is averaged to about 90%.

In the first stage, a feature space has been generated from the signal. After that, a hash function has been used based on neural networks. After, the code classifier with the hash function

Figure 11 shows the algorithm for recognizing gestures with neural network hash function.

Without the neural network hash function, the results were shown in Figure 9.

Results of tests without the neural network hash function are shown in Figure 10.

whether muscles, on this channel (Figure 8) have been reduced.

preprocessing.

recognizes gestures.

Figure 8. Recognition of the signal of EMG.

Figure 9. The process without the neural network hash function.

Figure 7. Scheme of the device of the executive mechanism of the car.

#### 2.3. Algorithms of the recognition of EMG: approaches on the work with EMG

In this work we recognize the electromyogram removed from the skin (i.e., it is noninvasive), the recognition purpose—to understand what the gripper has been made by the hand.

Working with data, which has been obtained from muscular activity, it has been shown that for optimum work of algorithms on capture recognition, it is necessary to execute the following main stages of processing:


Assessment of the efficiency of algorithms is carried out in two main parameters—the accuracy of work of the algorithm and volume of calculations—as the most important for use in real time. Signs for an algorithm of classification have been distinguished from the initial signal; thus, the compactness hypothesis was carried out, where each gripper is the class. To achieve performance of this hypothesis, then practically any qualifier, including the simplest, such as can be suitable for classification: "classification by a minimum of Euclidean distance." Each certain canal is not of a particular interest since it is not possible to differentiate gesture on one channel. Whether from the channel it was possible to select only information on that reduction of the muscle. Further on what muscles have been reduced and what are not present to distinguish concrete gripper. The algorithm of preprocessing allows, quite precisely defining whether muscles, on this channel (Figure 8) have been reduced.

When algorithms were tested, two main variants were developed; the first one used simple preprocessing of the signal. Moreover, the second neural network hash function as additional preprocessing.

Without the neural network hash function, the results were shown in Figure 9.

Results of tests without the neural network hash function are shown in Figure 10.

All gestures were recognized with the probability greater than 70%, of these, two gestures were recognized with a probability of less than 80%, six with recognized with probability <90%, but >80%, eight gestures >90% and two with the probability of a business to 100%.

This method has been tested for real-time amputations. Its accuracy is averaged to about 90%.

Figure 11 shows the algorithm for recognizing gestures with neural network hash function.

In the first stage, a feature space has been generated from the signal. After that, a hash function has been used based on neural networks. After, the code classifier with the hash function recognizes gestures.

Figure 8. Recognition of the signal of EMG.

2.3. Algorithms of the recognition of EMG: approaches on the work with EMG

Figure 7. Scheme of the device of the executive mechanism of the car.

main stages of processing:

• Filtration

100 Biofeedback

• Preprocessing

the recognition purpose—to understand what the gripper has been made by the hand.

In this work we recognize the electromyogram removed from the skin (i.e., it is noninvasive),

Working with data, which has been obtained from muscular activity, it has been shown that for optimum work of algorithms on capture recognition, it is necessary to execute the following

• Then, there is already a submission of data on neural network or an algorithm similar to it

Assessment of the efficiency of algorithms is carried out in two main parameters—the accuracy of work of the algorithm and volume of calculations—as the most important for use in real time. Signs for an algorithm of classification have been distinguished from the initial signal; thus, the compactness hypothesis was carried out, where each gripper is the class. To achieve performance of this hypothesis, then practically any qualifier, including the simplest, such as can be suitable for classification: "classification by a minimum of Euclidean distance." Each certain canal is not of a particular interest since it is not possible to differentiate gesture on one

• Post-data processing and additional training of neural or similar network

Figure 9. The process without the neural network hash function.

Figure 10. Results of tests without the neural network hash function.

Figure 11. The process with the neural network hash function.

However, in this test, the signals have been recorded beforehand. Real-time mode had been simulated.

Natali" company and on this base had been created unique product of recognition of EMG

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games…

http://dx.doi.org/10.5772/intechopen.74794

103

According to the previous researches described in a source of information [3] for a condition of implementation of requirements of work in real time, the time of recognition of a signal has to occupy no more than 250 ms. For comfortable work of the user productivity or recognition accuracy (percentage of right cases of classification to all considered cases), it has not been lower than 95%, as shown in a source [4]. Most details about recognition can be found in a

• For recognition at the first stage from a signal, various signs then the vector of these signs

signals of the hand.

source of information [5].

Methods of recognition of EMG:

moves on the system of recognition are been taken.

Figure 12. (a) Results of the test on seven gestures. (b) Results of the test on 11 gestures.

This algorithm has been tested on various amputees; the algorithm gives a high accuracy of determining the gesture. Below is the test of the algorithm on a person with shoulder amputation; seven gestures were recognized.

The picture shows the results of the test on seven gestures (Figure 12a), and the picture shows the results of the tests on 11 gestures (Figure 12b).

Neural network hash function consists not from one algorithm of neural net; it is complex of transformations, exactly five levels of different algorithms. On the entrance on the example, there are eight EMG signals, which had been modified with different neural networks and number of neurons. The details cannot be open because it is trade secret of the LLC "Bionic

However, in this test, the signals have been recorded beforehand. Real-time mode had been

This algorithm has been tested on various amputees; the algorithm gives a high accuracy of determining the gesture. Below is the test of the algorithm on a person with shoulder amputa-

The picture shows the results of the test on seven gestures (Figure 12a), and the picture shows

Neural network hash function consists not from one algorithm of neural net; it is complex of transformations, exactly five levels of different algorithms. On the entrance on the example, there are eight EMG signals, which had been modified with different neural networks and number of neurons. The details cannot be open because it is trade secret of the LLC "Bionic

simulated.

102 Biofeedback

tion; seven gestures were recognized.

the results of the tests on 11 gestures (Figure 12b).

Figure 11. The process with the neural network hash function.

Figure 10. Results of tests without the neural network hash function.

Figure 12. (a) Results of the test on seven gestures. (b) Results of the test on 11 gestures.

Natali" company and on this base had been created unique product of recognition of EMG signals of the hand.

According to the previous researches described in a source of information [3] for a condition of implementation of requirements of work in real time, the time of recognition of a signal has to occupy no more than 250 ms. For comfortable work of the user productivity or recognition accuracy (percentage of right cases of classification to all considered cases), it has not been lower than 95%, as shown in a source [4]. Most details about recognition can be found in a source of information [5].

Methods of recognition of EMG:

• For recognition at the first stage from a signal, various signs then the vector of these signs moves on the system of recognition are been taken.

• Most often as signs, counting bending around from different zones at the moment of time is used. As it is been made in work [6]. In addition, the system of recognition represents a set of rules. But approach at which the value bending around at the moment is chosen has one essential shortcoming. Mix-ups of gripper are possible in an area (Figure 13).

It is possible to fight against it in several ways: to pass the function which increases (falls down) quicker, than bending around, to use search of the maximum value bending around on an interval, to complicate the qualifier (e.g., to use recurrent or convolutional neural networks) that he considered not only the current values, but also some history or to use counters of operations for each class.

In our work, it has been used as signs of window dispersion. Properties of window dispersion of EMG are considered in work [7]. In addition, contrasting and scaling of value of dispersion have been applied. Then, at reduction of a muscle above which there is a sensor, the value is established in 1, and at relaxation it is in 0 (Figure 14).

After the previous processing, the signal arrives on the autoencoder to reduce entrance space of signs. Reduction of dimension of space of signs positively influences quality of the classification, in case of the use of metric qualifiers, because of a so-called "damnation of dimension" [8]. Later, there is a metric qualifier. Further, there is the counter of operations with the comparison block. After distinguished gripper goes to the operated device. The scheme of an algorithm has been submitted in Figure 15.

More detailed description is provided in the article [9, 10].

Also, alternatives for algorithms of the recognition exist; details can be found in articles [11–16].

2.4. Realization of developments with projects of LLC "Bionic Natali" and LLC "Bi-oN

Despite the fact that huge amount of works has been made, there is still a big area for activity concerning the choice and improvement of an algorithm on recognition of gripper, improvement of mechatronics, and the skin for an artificial hand. Similar work can be compared to art as we will compare the choice of an algorithm, the skin, and the solution of other technical problems with creativity. The current results of the LLC "Bioniс Natali" company in this sphere—it is recognition with probability of 98% on 14 grippers on 8 sensors with amplifiers from a forearm. The concerning removal of data and recognition of capture in disabled people then in practice were difficulties at movements of muscles and pain at a spasm in long muscular tension. In this regard, a decision together with many medical centers to develop a method of restoration of muscles and to create the tool for their training has been made—the electromyographic bracelet from the LLC "Bi-oN EMG" company has been made. An important component is the mathematical analysis of these artifacts and their elimination for the possibility of practical application of bionic artificial limbs based on neural network and other

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games…

http://dx.doi.org/10.5772/intechopen.74794

105

The same methods of control had been analyzed for movement of the knee, the first results just showed amazing implementation, and muscular activity of stump of leg can be used for control of movements of the knee. The control of sole does not need such instruments, because people usually use running artificial limbs for legs. Theoretical and preliminary practical results have shown big prospects in this direction, namely, the use of muscular activity and recognition of movements on the basis of neural network; in the process of completion of

EMG"

algorithms in practice.

works in this sphere, they will be published.

Figure 14. The algorithm of the previous processing.

Figure 13. The bending-around signal EMG.

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games… http://dx.doi.org/10.5772/intechopen.74794 105

Figure 14. The algorithm of the previous processing.

• Most often as signs, counting bending around from different zones at the moment of time is used. As it is been made in work [6]. In addition, the system of recognition represents a set of rules. But approach at which the value bending around at the moment is chosen has

It is possible to fight against it in several ways: to pass the function which increases (falls down) quicker, than bending around, to use search of the maximum value bending around on an interval, to complicate the qualifier (e.g., to use recurrent or convolutional neural networks) that he considered not only the current values, but also some history or to use counters of

In our work, it has been used as signs of window dispersion. Properties of window dispersion of EMG are considered in work [7]. In addition, contrasting and scaling of value of dispersion have been applied. Then, at reduction of a muscle above which there is a sensor, the value is

After the previous processing, the signal arrives on the autoencoder to reduce entrance space of signs. Reduction of dimension of space of signs positively influences quality of the classification, in case of the use of metric qualifiers, because of a so-called "damnation of dimension" [8]. Later, there is a metric qualifier. Further, there is the counter of operations with the comparison block. After distinguished gripper goes to the operated device. The scheme of an

Also, alternatives for algorithms of the recognition exist; details can be found in articles [11–16].

one essential shortcoming. Mix-ups of gripper are possible in an area (Figure 13).

operations for each class.

104 Biofeedback

established in 1, and at relaxation it is in 0 (Figure 14).

More detailed description is provided in the article [9, 10].

algorithm has been submitted in Figure 15.

Figure 13. The bending-around signal EMG.

### 2.4. Realization of developments with projects of LLC "Bionic Natali" and LLC "Bi-oN EMG"

Despite the fact that huge amount of works has been made, there is still a big area for activity concerning the choice and improvement of an algorithm on recognition of gripper, improvement of mechatronics, and the skin for an artificial hand. Similar work can be compared to art as we will compare the choice of an algorithm, the skin, and the solution of other technical problems with creativity. The current results of the LLC "Bioniс Natali" company in this sphere—it is recognition with probability of 98% on 14 grippers on 8 sensors with amplifiers from a forearm. The concerning removal of data and recognition of capture in disabled people then in practice were difficulties at movements of muscles and pain at a spasm in long muscular tension. In this regard, a decision together with many medical centers to develop a method of restoration of muscles and to create the tool for their training has been made—the electromyographic bracelet from the LLC "Bi-oN EMG" company has been made. An important component is the mathematical analysis of these artifacts and their elimination for the possibility of practical application of bionic artificial limbs based on neural network and other algorithms in practice.

The same methods of control had been analyzed for movement of the knee, the first results just showed amazing implementation, and muscular activity of stump of leg can be used for control of movements of the knee. The control of sole does not need such instruments, because people usually use running artificial limbs for legs. Theoretical and preliminary practical results have shown big prospects in this direction, namely, the use of muscular activity and recognition of movements on the basis of neural network; in the process of completion of works in this sphere, they will be published.

The electromyographic bracelet has found practical application in rehabilitation and game devices as it has been told earlier. And, researches and testing of bionic artificial limbs of hands on disabled people and testing of a bracelet on game devices for the last year have yielded new results which have allowed to draw a number of the main conclusions regarding formation of feedback at users, on the principles of receiving in general feedback on the basis of EMG.

Control Systems of Bionic Limbs of the New Generation and Control Systems with EMG Signals of VR and Games…

http://dx.doi.org/10.5772/intechopen.74794

107

It is important to note that because of these developments, four patents for the invention are

3. Results of the practical application of developments of the recognition

management of bionic artificial limbs of hands and for game devices for

As already it has been noted above, thanks to the created technology researches of innovative feedback based on EMG for people without loss of limbs and also with loss of limbs regarding management of the systems of recognition of EMG based on the neural network have been

• Without loss of an extremity (18 people): from 12 to 65 years with different functional

• People with loss of an extremity (16 people): from 22 to 60 years with different psycho-

Results of the analysis of an experiment have not included people who took only single part in a research as to reveal certain regularities, and there are needs to hold regular testing and checks; single participation is not natural and does not give understanding about feedback which is formed, and also in connection with complex psychological structure of people, the

The research objective is to reveal regularities using the control system of recognition of EMG from the shoulder and the forearm based on the neural network for different groups of people. If to carry out the comparative analysis at people without amputation and with amputation, then results showed that physical training and a training of muscles plays a significant role

In spite of the fact that the signal in itself at people with an amputation of a hand is much more weak than people without loss have arms, the muscular training sometimes at people with amputation of an amputation allows to receive more accurate signal than a signal at the person of the same age group without loss of the limbs. Regular researches of people with amputation of limbs showed that more often there is a training of muscles and the muscle tone and also a comprehension and "representation" of gripper which are carried out by the person with amputation, and as a result, the subsequent already management of a bionic limb comes back quicker.

of EMG based on the neural network for disabled people at the

conducted. Categories of people who participated in the research are the following:

adaptabilities to new devices and psychological outlook

logical views and speed of reaction

probability of obtaining wrong data is high.

regarding recognition of an EMG of a signal.

created [17, 18].

people without disability

Figure 15. Flowchart of the algorithm.

The electromyographic bracelet has found practical application in rehabilitation and game devices as it has been told earlier. And, researches and testing of bionic artificial limbs of hands on disabled people and testing of a bracelet on game devices for the last year have yielded new results which have allowed to draw a number of the main conclusions regarding formation of feedback at users, on the principles of receiving in general feedback on the basis of EMG.

It is important to note that because of these developments, four patents for the invention are created [17, 18].
