**1. Introduction**

Brain-computer interface (BCI) system provides one of the most important aspects, which is an alternative way of communication through brain signals. It is just to translate electroencephalogram

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

(EEG) signals from a reflection of brain activity into user action through system's hardware and software. A BCI system provides a communication channel not based on nerves and muscles that allow users to communicate by electrodes contacting on scalp. It has attracted increasing attention of a variety of research fields including neuroscience, machine learning, pattern recognition, rehabilitation medicine, and so on.

network untrainable owing to weight diffusion, while large initial values of the weights could result in poor local minima [10]. In order to resolve this problem and construct high descriptive-ability neural networks, a new model of strategies and algorithms, called deep learning

A Motor-Imagery BCI System Based on Deep Learning Networks and Its Applications

http://dx.doi.org/10.5772/intechopen.75009

17

There are many ways in machine learning for data classification. The most popular and proven method in recent decades is "Artificial Neural Network (ANN)." We know how artificial neural networks adjust weights so that the error between output and input becomes smaller. But even so, this is far from the "artificial intelligence" that we want. If the computer can analyze the data to find the features, then it is closer to the artificial intelligence we want, that is to say, the created computer can think. DL allows computers to analyze their own data to find "features," rather than decided by human beings with features, just as computers can have deep thinking to learn. DL uses not only a multilayer neural network but also an auto-

Recurrent neural networks (RNN), one of the models in DL, have proved promising results in many field [12–15] recently, especially when input and/or output are of variable length. In the application of EEG signals classification, Petrosian et al. [16] first applied RNN and wavelet transform to classify EEG signals. RNN is not satisfied in scalp EEG owing to the scalp EEG containing interference resulted from external noises. Besides, the input of the RNN does not have a special signal preprocessing, the RNN network has some problems such as gradient explosion and gradient vanish. Fully using characteristics in time-frequency features of signals, RNN with LSTM [17], have recently emerged as an effective deep learning model in a wide variety of applications that involve sequential data. The LSTM-based RNN can not only solve the problems in RNN but also store the long time information. In 2016, Li et al. [18] proposed an LSTM-based RNN integrated with DWT to classify the EEG signals. The LSTM is designed to fight against vanishing gradients through a gating mechanism. Gated recurrent neural network (GRNN), proposed by Cho et al. [19] in 2014, makes each recurrent unit to capture variable-length sequences adaptively. Similar scheme of the LSTM unit, GRNN has gating units that modulate the flow of information inside the unit, but without having a separate memory cell. In GRNN, the parameters at each level are shared through the whole network. In this chapter, LSTM and GRNN combined with the DWT to classify the EEG signals were proposed. The average power spectrum of MI-EEG signals was calculated and the effective time segment was also determined. Then, DWT is applied to each channel of MI-EEG to extract the effective time-frequency characteristics. Finally, LSTM and GRNN were used as classifiers to recognize the MI-EEG signals. The experimental results showed that GRNN and LSTM methods can make full use of the time-frequency information of MI-EEG, as well as

time sequence information, and can get better recognition performance.

The rest of this chapter is organized as follows: Section 2 describes the system architecture; wavelet transform is described in Section 3; Section 4 presents the LSTM-based recurrent network; the GRNN is discussed in Section 5; Section 6 shows the experimental results; the application to control an electric wheelchair is shown in Section 7; and finally, the discussions are

(DL), has been successfully developed and becomes prevailing in several fields [11].

encoder for unsupervised learning.

given in Section 8.

Motor imagery (MI) is an important research topic in the field of BCI that mentally simulates a given action, e.g., imaging the motions of the limbs [1]. It refers to visualization of a limbic activity, or any other movement, without the actual execution of the motion imagined. It leads to various changes in the connectivity between the neurons present in the cortex. This results in either an event-related desynchronization (ERD) or event-related synchronization (ERS) of mu rhythms. These effects are due to the changes in the chemical synapses of the neurons, the change in strength between the interconnections or the change of intrinsic membrane properties of local neurons. Since extracted from scalp EEG, MI-EEG has the characteristics of nonlinear, nonstationary, and time-varying.

In the research field of MI-EEG-based BCI, several researchers have proposed different strategies. Tomida et al. [2] presented an active data selection method for MI-EEG classification in 2015. Rejecting or selecting data from multiple trials of EEG recordings is crucial in the selection method. To aim at brain machine interfaces (BMIs), they proposed a sparsity-aware method to select data from a set of multiple EEG recordings during MI tasks. An extraction approach with transform-based feature for MI tasks classification was proposed by Baali et al. [3]. A signal-dependent orthogonal transform was used, referred to as linear prediction singular value decomposition (LP-SVD), for feature extraction. They used a logistic tree-based model classifier to classify the extracted features into one of four motor imagery movements. In 2016, Wu et al. [4] used the fuzzy integral with particle swarm optimization (PSO), which can regulate subject-specific parameters for the assignment of optimal confidence levels for classifiers. Lin and Lo [5] constructed a MI-based BCI system to control an electric wheelchair. They used discrete wavelet transform (DWT) to transform EEG signals into frequency domain and applied SVM to classify them into different commands. Chatterjee and Bandyopadhyay [6] used SVM and multilayered perceptron (MLP) for MI-EEG classification in 2016. They showed that both SVM and MLP were suitable for such MI classifications with the accuracy of 85 and 85.71%, respectively. The symmetric positive-definite (SPD) covariance matrices of EEG signals carry important discriminative information proposed by Xie et al. [7] for MI BCI system in 2016. Chatterjeel et al. [8] examined the quality of feature sets obtained from wavelet-based energy entropy with variation of scale and wavelet type for MI classification in 2016. They have verified their study with three classifiers—Naive Bayes, MLP and SVM. Jois et al. [9] compared several classification techniques for motor imagery-based BCI in 2015. They indicated that common features, e.g., band power values, present that the single EEG trials can be extracted by suitable methods for classification using SVM, neural networks, or ensemble classifiers. The classifiers yield different efficiencies and are compared to find the optimal technique for same number of features. They believed the neural net techniques were proved to be the most efficient. One obstacle of the traditional neural networks for their broader application is the initial weights need to be chosen carefully. Generally, small values could make the multilayer network untrainable owing to weight diffusion, while large initial values of the weights could result in poor local minima [10]. In order to resolve this problem and construct high descriptive-ability neural networks, a new model of strategies and algorithms, called deep learning (DL), has been successfully developed and becomes prevailing in several fields [11].

(EEG) signals from a reflection of brain activity into user action through system's hardware and software. A BCI system provides a communication channel not based on nerves and muscles that allow users to communicate by electrodes contacting on scalp. It has attracted increasing attention of a variety of research fields including neuroscience, machine learning, pattern recognition, reha-

Motor imagery (MI) is an important research topic in the field of BCI that mentally simulates a given action, e.g., imaging the motions of the limbs [1]. It refers to visualization of a limbic activity, or any other movement, without the actual execution of the motion imagined. It leads to various changes in the connectivity between the neurons present in the cortex. This results in either an event-related desynchronization (ERD) or event-related synchronization (ERS) of mu rhythms. These effects are due to the changes in the chemical synapses of the neurons, the change in strength between the interconnections or the change of intrinsic membrane properties of local neurons. Since extracted from scalp EEG, MI-EEG has the characteristics of

In the research field of MI-EEG-based BCI, several researchers have proposed different strategies. Tomida et al. [2] presented an active data selection method for MI-EEG classification in 2015. Rejecting or selecting data from multiple trials of EEG recordings is crucial in the selection method. To aim at brain machine interfaces (BMIs), they proposed a sparsity-aware method to select data from a set of multiple EEG recordings during MI tasks. An extraction approach with transform-based feature for MI tasks classification was proposed by Baali et al. [3]. A signal-dependent orthogonal transform was used, referred to as linear prediction singular value decomposition (LP-SVD), for feature extraction. They used a logistic tree-based model classifier to classify the extracted features into one of four motor imagery movements. In 2016, Wu et al. [4] used the fuzzy integral with particle swarm optimization (PSO), which can regulate subject-specific parameters for the assignment of optimal confidence levels for classifiers. Lin and Lo [5] constructed a MI-based BCI system to control an electric wheelchair. They used discrete wavelet transform (DWT) to transform EEG signals into frequency domain and applied SVM to classify them into different commands. Chatterjee and Bandyopadhyay [6] used SVM and multilayered perceptron (MLP) for MI-EEG classification in 2016. They showed that both SVM and MLP were suitable for such MI classifications with the accuracy of 85 and 85.71%, respectively. The symmetric positive-definite (SPD) covariance matrices of EEG signals carry important discriminative information proposed by Xie et al. [7] for MI BCI system in 2016. Chatterjeel et al. [8] examined the quality of feature sets obtained from wavelet-based energy entropy with variation of scale and wavelet type for MI classification in 2016. They have verified their study with three classifiers—Naive Bayes, MLP and SVM. Jois et al. [9] compared several classification techniques for motor imagery-based BCI in 2015. They indicated that common features, e.g., band power values, present that the single EEG trials can be extracted by suitable methods for classification using SVM, neural networks, or ensemble classifiers. The classifiers yield different efficiencies and are compared to find the optimal technique for same number of features. They believed the neural net techniques were proved to be the most efficient. One obstacle of the traditional neural networks for their broader application is the initial weights need to be chosen carefully. Generally, small values could make the multilayer

bilitation medicine, and so on.

16 Evolving BCI Therapy - Engaging Brain State Dynamics

nonlinear, nonstationary, and time-varying.

There are many ways in machine learning for data classification. The most popular and proven method in recent decades is "Artificial Neural Network (ANN)." We know how artificial neural networks adjust weights so that the error between output and input becomes smaller. But even so, this is far from the "artificial intelligence" that we want. If the computer can analyze the data to find the features, then it is closer to the artificial intelligence we want, that is to say, the created computer can think. DL allows computers to analyze their own data to find "features," rather than decided by human beings with features, just as computers can have deep thinking to learn. DL uses not only a multilayer neural network but also an autoencoder for unsupervised learning.

Recurrent neural networks (RNN), one of the models in DL, have proved promising results in many field [12–15] recently, especially when input and/or output are of variable length. In the application of EEG signals classification, Petrosian et al. [16] first applied RNN and wavelet transform to classify EEG signals. RNN is not satisfied in scalp EEG owing to the scalp EEG containing interference resulted from external noises. Besides, the input of the RNN does not have a special signal preprocessing, the RNN network has some problems such as gradient explosion and gradient vanish. Fully using characteristics in time-frequency features of signals, RNN with LSTM [17], have recently emerged as an effective deep learning model in a wide variety of applications that involve sequential data. The LSTM-based RNN can not only solve the problems in RNN but also store the long time information. In 2016, Li et al. [18] proposed an LSTM-based RNN integrated with DWT to classify the EEG signals. The LSTM is designed to fight against vanishing gradients through a gating mechanism. Gated recurrent neural network (GRNN), proposed by Cho et al. [19] in 2014, makes each recurrent unit to capture variable-length sequences adaptively. Similar scheme of the LSTM unit, GRNN has gating units that modulate the flow of information inside the unit, but without having a separate memory cell. In GRNN, the parameters at each level are shared through the whole network.

In this chapter, LSTM and GRNN combined with the DWT to classify the EEG signals were proposed. The average power spectrum of MI-EEG signals was calculated and the effective time segment was also determined. Then, DWT is applied to each channel of MI-EEG to extract the effective time-frequency characteristics. Finally, LSTM and GRNN were used as classifiers to recognize the MI-EEG signals. The experimental results showed that GRNN and LSTM methods can make full use of the time-frequency information of MI-EEG, as well as time sequence information, and can get better recognition performance.

The rest of this chapter is organized as follows: Section 2 describes the system architecture; wavelet transform is described in Section 3; Section 4 presents the LSTM-based recurrent network; the GRNN is discussed in Section 5; Section 6 shows the experimental results; the application to control an electric wheelchair is shown in Section 7; and finally, the discussions are given in Section 8.
