**2. Motor imagery and action observation**

However, despite their popularity and potentials, BCIs still remain mostly used inside laboratories and barely commercialized for real-world applications. The main reason behind this slow progress is the lack of reliability and poor performance of the BCI systems [6]. Even the finest BCI classifiers developed to date are not yet able to extract the relevant features from brain activity with high accuracy and robustness, particularly if the activity is recorded with electroencephalography (EEG) and contains noise. Many BCI researchers have made it a quest of their life to develop systems and algorithms that can decode EEG activity with high accuracy [7]. However, beside the algorithms, there is another element in the BCI loop that often gets neglected and that is the human user who is the source of the input signals [6, 8]. Although it has been previously shown that not every user is capable of controlling a BCI, the so-called BCI illiteracy [9], most users can obtain a decent level of "skill" with a few sessions

Motor imagery-based BCIs demand particularly longer training time compared to ERP-based BCIs (such as P300 speller) or BCIs that use steady-state visual-evoked potentials (SSVEPs). This is due to the fact that motor imagery task, the mental rehearsal of a movement without actually performing it, is a counterintuitive task for the majority of individuals. Most users cannot visualize a vivid picture of the movement and its kinesthetic experience. Hwang et al. refer to this as the unknown "feel of motor imagery" [10]. An imaginary action can range from the visualization of a self-performed movement from a first-person perspective, to a thirdperson view of the self-body movement, to the manipulation of an external object that is either physical or imaginary [11]. Although these types of motor imagery all involve voluntary actions, they may not involve similar cognitive processes. For novice BCI users, the instruction about a motor imagery task is normally given verbally by an experimenter, and it is up to the user to find the optimum image, by trial and error, that leads to a high performance.

On the other hand, similar to any other interface, BCI users should receive feedback of their performance in order to close the control loop between them and the interface. Over years, various feedback paradigms for motor imagery training have been proposed, most of which are based on visual and auditory feedback [12, 13]. One of the main issues in the design of visual feedback in most of motor imagery-based BCIs is that the feedback presentation is not congruent with the subject's image of a bodily movement. For example, in the training paradigm introduced by Pfurtscheller and Neuper, subjects imagined either a right- or a left-hand movement and watched a horizontal feedback bar on a computer screen that was extended to the right or to the left based on the classifier output [12]. Blankertz et al. presented a falling ball on the screen that could be horizontally displaced either to the left or right side if the user's left- or right-hand imagery was successfully detected by the classifier [13]. In another study, Nijboer et al. employed two feedback designs: a visual feedback with a cursor on a screen that moved up and down based on the subject's sensorimotor rhythm and an auditory feedback that presented different types of sound in existence or the absence of motor imagery activation [14]. In all of the given examples, the feedback design that was employed had no congruity with the type of image that the subjects held (image of a bodily hand or a foot). Not only the disparity between the visual feedback and the type of image can confuse the subjects during the task, but it also prevents them from obtaining "the feel of motor imagery" and

of training.

74 Evolving BCI Therapy - Engaging Brain State Dynamics

correcting their imagery strategy.

It has been shown that mental imagery of a motor action can produce cortical activation similar to that of the same action executed [23, 24]. For instance, the execution of a hand movement results in the suppression of mu rhythm (8–12 Hz) in sensorimotor regions and so does the motor imagery of the corresponding hand [25]. By monitoring single-trial EEG signals and measuring event-related desynchronization (ERD), it is even possible to detect whether the imagined hand was the right or the left one [26]. However, previous studies suggest that the detection of hand imagery can only achieve a high rate when the user has employed a kinesthetic motor imagery strategy (first-person process) [27]. In the same study, Neuper et al. report that the observation of a left- or a right-hand movement could also lead to high classification accuracies at parieto-occipital regions [27]. Many neuroimaging studies have found empirical evidences that combining motor imagery with action observation could induce a stronger cortical activation compared to either condition alone [28]. This has been associated with the firing of mirror neurons [29] that become active during the observation of a motion and represent high-level information about goals and intentions [28]. It also indicates a shared neurocognitive process between motor imagery and action observation that could be utilized in BCI training and control. Particularly, if the action is congruent with the motor imagery, the observed image is a simulation of one's own action, the combination of the two conditions can lead to a "sense of effort," a sense of agency, and imagined kinesthetic sensations that would arise during one's own motor execution [30].

feedback with a more detailed appearance and compatible action to one's real hand can extend

Brain-Computer Interface and Motor Imagery Training: The Role of Visual Feedback and Embodiment

http://dx.doi.org/10.5772/intechopen.78695

77

It is worth noting that BCI training along with visual feedback of a body movement (action observation) has been employed in neurorehabilitation studies and with stroke patients as well [37–42]. It is suggested that providing anthropomorphic feedback during motor imagery works in a similar way as does mirror therapy for phantom limb patients [39]. That is, providing feedback of a bodily movement can activate neural networks associated with action observation system and induce a "motor resonance" [40]. Thereby by directly matching the observed or imagined action onto the internal simulation of that action, motor resonance can further facilitate the relearning of the impaired motor functions [41]. For instance, Foldes et al. trained spinal cord injury patients with hand paralysis on a motor imagery-based BCI combined with a virtual hand feedback. Results showed that all patients could successfully modulate their brain activity in order to grasp the virtual hand and two of three participants could improve their sensorimotor rhythms in only one session of feedback training [39]. Kim et al. also combined an action observation training with a motor imagery-based BCI and found promising results in terms of actual functional improvements in the upper arm movement of stroke patients [42]. The above review shows that a neurofeedback paradigm that merges motor imagery with the observation of a bodily action has the potential to promote plastic changes in somatosensory activation, the recovery of motor functions, and the improvement of motor performance [43]. In a very similar way, such combination can bring significant benefits to BCI training, by helping the user to activate motor-related cortical areas and generate brain signals that are easily

To control a BCI, the user has to perform a mental imagery task and generate distinguishable brain activity for signal-processing algorithms. Modulation of one's own brain signals is not an intuitive task, and therefore the user needs to practice and learn the BCI "skill." However, an efficient learning of a skill requires optimized training protocols that consider the user's psychological states (such as motivation, attention, confidence, and satisfaction) in order to ensure more effort and better performance from the user's side [44]. Kleih et al. have shown that in the control of a P300 BCI, the level of P300 amplitude was significantly correlated with the level of self-rated motivation, that is, highly motivated subjects were able to communicate through BCI faster than less motivated subjects [45]. In another BCI study with ALS patients, Nijboer et al. reported that motivational factors, specifically challenge and confidence, were positively correlated with BCI performance, whereas fear had a negative influence [46]. It is suggested that even with highly motivated subjects, users can experience a low level of

In order to overcome such issues, many researchers have explored alternative BCI training protocols. Leeb et al. suggested employment of VR environments in designing attractive BCI

satisfaction if they do not succeed in accomplishing the BCI-control task [47].

larger effect on neural plasticity and reinforcement of motor imagery learning.

detectable by the BCI classifier.

**3. Human factors and BCI learning**

Ono et al. examined the effect of visual feedback on ERD during a motor imagery task [31]. In a series of training sessions, they hired different groups of subjects and trained them on different types of visual feedback, including a conventional feedback bar, a human hand that was shown on a screen in front of the subject and a human hand that was shown on a screen as the extension of one's own arm. They found that by the end of the training, the group that was presented anatomically congruent feedback produced the highest ERD value and classification accuracy. Neuper et al. have also investigated the impact of a visual feedback presentation on sensorimotor EEG rhythms and BCI performance [32]. They trained two groups of subjects on a motor imagery-based BCI using two feedback designs: a realistic feedback (a video of a moving hand that grasped a glass) and an abstract feedback (a moving bar that extended horizontally). Their results, however, showed no difference between the two feedback groups in terms of motor imagery learning and ERD changes. An explanation for this, as authors have indicated in their discussion, could be the short training period and few number of feedback sessions.

With recent advancement in videogames and VR technology, a more rich, realistic, and engaging visual presentation of the BCI output has become possible. Pineda et al. designed a threedimensional first-person shooter game that enabled BCI users to make navigational movements by left and right motor imageries [33]. Their results indicated that subjects could learn to control levels of mu rhythm very quickly, within approximately 3–10 hours of training that was scheduled over a course of five weeks. Leeb et al. also reported a case study with a tetraplegic patient who was able to navigate through VR by imagination of his feet movements that was translated into movements of an avatar [34]. The most obvious benefits of VR in the construction of visual feedback are the richness of details that could be incorporated, the sense of embodiment it induces (see Section 4), and a relatively low cost. Particularly, in terms of detailed feedback, it can involve different types of movement and inclusion of goal-oriented tasks. Past studies have shown that motor cortex is sensitive to different forms of observed motor behavior [35] and subjective perspective [30, 36]. Muthukumaraswamy et al. have shown that the observation of an object-directed precision grip produces more mu suppression than the observation of a nonobject-directed grip [35]. In our previous study, we compared motor imagery learning between two groups of BCI users who operated either a pair of robotic gripper or a pair of humanlike robotic hands [8]. We found a more robust learning of the BCI task for the second group who were trained with a pair of humanlike robotic hands. This result provides evidence that visual feedback with a more detailed appearance and compatible action to one's real hand can extend larger effect on neural plasticity and reinforcement of motor imagery learning.

It is worth noting that BCI training along with visual feedback of a body movement (action observation) has been employed in neurorehabilitation studies and with stroke patients as well [37–42]. It is suggested that providing anthropomorphic feedback during motor imagery works in a similar way as does mirror therapy for phantom limb patients [39]. That is, providing feedback of a bodily movement can activate neural networks associated with action observation system and induce a "motor resonance" [40]. Thereby by directly matching the observed or imagined action onto the internal simulation of that action, motor resonance can further facilitate the relearning of the impaired motor functions [41]. For instance, Foldes et al. trained spinal cord injury patients with hand paralysis on a motor imagery-based BCI combined with a virtual hand feedback. Results showed that all patients could successfully modulate their brain activity in order to grasp the virtual hand and two of three participants could improve their sensorimotor rhythms in only one session of feedback training [39]. Kim et al. also combined an action observation training with a motor imagery-based BCI and found promising results in terms of actual functional improvements in the upper arm movement of stroke patients [42].

The above review shows that a neurofeedback paradigm that merges motor imagery with the observation of a bodily action has the potential to promote plastic changes in somatosensory activation, the recovery of motor functions, and the improvement of motor performance [43]. In a very similar way, such combination can bring significant benefits to BCI training, by helping the user to activate motor-related cortical areas and generate brain signals that are easily detectable by the BCI classifier.
