*2.2.3. SSVEP based virtual gaming application*

The panel, placed at 30 cm from subject's head, was a 7 × 7 cm<sup>2</sup>

**Figure 5.** Wheelchair prototype for SSVEP-based BCI control.

100 Evolving BCI Therapy - Engaging Brain State Dynamics

red squares (represented by grey squares here) flash at 14 Hz.

red and yellow 1 × 1 cm<sup>2</sup>

**Figure 6.** Electronic visual stimulation unit.

the middle (**Figure 6**).

"interlaced square" made of

light emitting diode (LED) - squares with a white fixation cross in

The yellow squares (represented by white squares here) flicker at the frequency of 10 Hz. The

The interlaced square pattern showed a 10% improvement in accuracy in comparison with a "line" pattern [49]. The control unit was designed to precisely control the red and yellow flickering frequencies independently between 1 and 99 Hz by microcontroller based circuit.

Martišius and Damaševičius in 2016 [55] proposed an SSVEP based BCI gaming system. The researchers developed a 3-class BCI system based on SSVEP and emotive EPOC Headset. The game involved target shooting developed in the OpenVIBE environment which provided the user feedback. Emotive EPOC, a 16 electrode based gaming headset, was used in combination with the SSVEP paradigm. Raw EEG data from the head set was acquired with internal sampling of 2048 Hz. Signals from the O1, O2, P7, and P8 were taken.

At first, data was split into three groups, according to their corresponding class labels, LEFT, RIGHT, and CENTER. Each group of signals was subjected to band-pass filter centered on the target frequency of interest: for the LEFT class, 29.5–30.5 Hz; CENTER, 19.5–20.5 Hz; RIGHT, 11.5–12.5 Hz.

**Figure 7.** Overt block pattern.


**Table 2.** Mean and standard deviation of classification accuracy (in percent) obtained with the Thomson multitaper method (PMTM) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel selection algorithm.

There have been studies [46] that analyzed how different colors of the targets influence classification quality. The user was presented with an LCD display, containing three blinking targets on a black background and a yellow arrow. On cue, the targets start blinking at different frequencies as shown in **Figure 8(a)**.

*2.2.4. SSVEP based communicator/speller enhancement*

algorithm.

1 Hz) and four phases (0, 90, 180, and 270°).

s(f, φ, i) = square[2πf(

Nakanishi et al. in [56] designed a high speed speller based on SSVEP stimulus. The study was aimed at exploring the feasibility of mixed frequency and phase coding to form a high speed speller using a TFT monitor. A frequency and phase approximation approach was deployed to remove the limitation of the number of targets caused by the monitor refresh rate, resulting in a speller comprising 32 flickers specified by eight frequencies (8–15 Hz at an interval of

**Table 3.** Mean and standard deviation of classification accuracy (in per cent) obtained with the lock-in analyzer system (LAS) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel selection

**Subject (Group A) Nharm = 1 Nharm = 2 Nharm = 3**

Subject 1 76.7 ± 3.1 85.0 ± 1.9 73.2 ± 4.9 93.9 ± 1.8 — 94.7 ± 1.2 Subject 2 86.5 ± 2.1 93.5 ± 0.8 74.8 ± 5.2 95.4 ± 0.9 — 95.0 ± 1.7 Subject 3 76.7 ± 3.1 82.5 ± 2.7 73.2 ± 4.9 79.4 ± 2.7 — 74.6 ± 2.6 Subject 4 61.0 ± 4.2 73.6 ± 3.5 58.3 ± 4.5 73.6 ± 2.2 — 77.6 ± 1.9 Subject 5 69.7 ± 2.9 80.2 ± 2.4 54.2 ± 4.6 76.1 ± 2.8 — 72.8 ± 1.9 Subject 6 61.4 ± 3.6 76.2 ± 2.0 60.8 ± 4.7 79.0 ± 2.2 — 77.7 ± 3.3 Subject 7 85.2 ± 2.8 90.0 ± 2.2 76.5 ± 4.7 91.2 ± 2.4 — 94.0 ± 2.4 Subject 8 83.0 ± 2.9 87.4 ± 2.2 75.9 ± 4.4 94.3 ± 2.2 — 91.5 ± 2.6 Subject 9 90.5 ± 2.4 92.9 ± 1.7 75.9 ± 4.2 90.9 ± 2.1 — 91.5 ± 1.8 Subject 10 53.7 ± 3.8 70.1 ± 2.4 61.7 ± 4.7 70.9 ± 3.0 — 71.4 ± 2.9 Subject 11 57.8 ± 4.3 — 48.5 ± 5.1 — — — Subject 12 49.7 ± 4.3 — 54.0 ± 5.0 — — — Total 74.4 ± 3.2 83.1 ± 2.8 68.4 ± 4.7 84.5 ± 2.3 — 84.1 ± 2.3

**AC ACSA AC ACSA AC ACSA**

SSVEP-Based BCIs

103

http://dx.doi.org/10.5772/intechopen.75693

Wang et al. [57] proposed an approach that generates visual flickers at a flexible frequency by approximating the frequency with variable number of frames in a stimulation cycle. For instance, a flicker at 11 Hz under a 60 Hz refresh rate can be realized by bridging five and six frames in a stimulation cycle as "1110001110011100011100111…" Based on this technique we can generate flicker frequencies up to 50% of the screen refresh rate, hence increasing the number of stimuli that can be presented. Generally, to render a visual flicker at frequency f

where the function square [] generates a square wave of 50% duty cycle with levels 0 and 1, and i indicates the frame index. Nakanishi et al. used quad-phase coded flickering signals at

\_\_\_\_\_\_\_\_\_ i

refesh rate) <sup>+</sup> <sup>φ</sup>] (5)

with an initial phase φ, a stimulus sequence s(f, φ,i) can be generated by:

After classifier training, subjects were invited to participate in the game experiment. During the game, the subjects were presented with an interface from **Figure 8(b)**. The "spaceship" with two "engines," represented by two rectangles, and a "cannon," represented by the triangle. The subject could rotate the spaceship by focusing his/her attention on one of the rectangular targets.

By focusing attention on the middle triangle, the user was able to fire the spaceship cannon. The aim of the game is to rotate the spaceship and fire its canon to hit the red target. Once the target was hit, it disappeared to reappear in another position.

An evaluation of the system was performed using two subjects, named S1 and S2, unfamiliar with the BCI technology. The first algorithm used was wave atom transform (WAT) coefficients and the second algorithm used the band power (BP) in the stimulation frequency bands.

The accuracy was measured for each subject, while performing classification with 4 different classifiers (LDA, sparse LDA (sLDA), SVM with linear kernel, and SVM with RBF kernel (with parameter values, gamma = 10)). The results are depicted in **Table 4**.


**Table 3.** Mean and standard deviation of classification accuracy (in per cent) obtained with the lock-in analyzer system (LAS) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel selection algorithm.

#### *2.2.4. SSVEP based communicator/speller enhancement*

There have been studies [46] that analyzed how different colors of the targets influence classification quality. The user was presented with an LCD display, containing three blinking targets on a black background and a yellow arrow. On cue, the targets start blinking at different

**Table 2.** Mean and standard deviation of classification accuracy (in percent) obtained with the Thomson multitaper method (PMTM) for different numbers of harmonics with (ACSA) and without (AC) the use of automatic channel

**Subject (Group A) Nharm = 1 Nharm = 2 Nharm = 3**

Subject 1 78.4 ± 3.4 85.7 ± 1.8 73.9 ± 4.7 94.3 ± 1.8 — 94.4 ± 2.2 Subject 2 92.3 ± 2.4 94.8 ± 1.0 76.5 ± 4.9 91.3 ± 1.6 — 92.6 ± 1.1 Subject 3 78.4 ± 3.4 84.4 ± 2.1 73.9 ± 4.7 84.9 ± 2.6 — 80.6 ± 3.0 Subject 4 67.8 ± 3.5 76.0 ± 2.3 51.7 ± 4.7 76.7 ± 3.7 — 78.6 ± 3.2 Subject 5 62.4 ± 3.9 78.9 ± 2.3 51.6 ± 4.8 77.4 ± 2.4 — 70.5 ± 3.2 Subject 6 71.6 ± 3.8 85.8 ± 2.6 62.6 ± 5.0 81.9 ± 3.3 — 85.6 ± 2.9 Subject 7 89.5 ± 2.4 94.5 ± 1.4 76.0 ± 3.9 92.8 ± 2.4 — 92.6 ± 1.9 Subject 8 82.7 ± 3.3 89.2 ± 2.3 77.3 ± 4.3 94.4 ± 1.9 — 87.2 ± 1.7 Subject 9 91.1 ± 2.3 94.6 ± 1.6 84.0 ± 4.3 91.7 ± 1.1 — 91.8 ± 1.3 Subject 10 56.6 ± 4.4 63.5 ± 1.8 58.3 ± 4.7 67.5 ± 3.0 — 68.4 ± 2.4 Subject 11 57.0 ± 3.8 — 43.1 ± 5.0 — — — Subject 12 44.5 ± 3.8 — 43.2 ± 5.8 — — — Total 77.0 ± 3.4 84.7 ± 2.0 68.6 ± 4.6 85.3 ± 2.5 — 84.2 ± 2.4

**AC ACSA AC ACSA AC ACSA**

After classifier training, subjects were invited to participate in the game experiment. During the game, the subjects were presented with an interface from **Figure 8(b)**. The "spaceship" with two "engines," represented by two rectangles, and a "cannon," represented by the triangle. The subject could rotate the spaceship by focusing his/her attention on one of the rect-

By focusing attention on the middle triangle, the user was able to fire the spaceship cannon. The aim of the game is to rotate the spaceship and fire its canon to hit the red target. Once the

An evaluation of the system was performed using two subjects, named S1 and S2, unfamiliar with the BCI technology. The first algorithm used was wave atom transform (WAT) coefficients and the second algorithm used the band power (BP) in the stimulation frequency

The accuracy was measured for each subject, while performing classification with 4 different classifiers (LDA, sparse LDA (sLDA), SVM with linear kernel, and SVM with RBF kernel (with

target was hit, it disappeared to reappear in another position.

parameter values, gamma = 10)). The results are depicted in **Table 4**.

frequencies as shown in **Figure 8(a)**.

102 Evolving BCI Therapy - Engaging Brain State Dynamics

angular targets.

selection algorithm.

bands.

Nakanishi et al. in [56] designed a high speed speller based on SSVEP stimulus. The study was aimed at exploring the feasibility of mixed frequency and phase coding to form a high speed speller using a TFT monitor. A frequency and phase approximation approach was deployed to remove the limitation of the number of targets caused by the monitor refresh rate, resulting in a speller comprising 32 flickers specified by eight frequencies (8–15 Hz at an interval of 1 Hz) and four phases (0, 90, 180, and 270°).

Wang et al. [57] proposed an approach that generates visual flickers at a flexible frequency by approximating the frequency with variable number of frames in a stimulation cycle. For instance, a flicker at 11 Hz under a 60 Hz refresh rate can be realized by bridging five and six frames in a stimulation cycle as "1110001110011100011100111…" Based on this technique we can generate flicker frequencies up to 50% of the screen refresh rate, hence increasing the number of stimuli that can be presented. Generally, to render a visual flicker at frequency f with an initial phase φ, a stimulus sequence s(f, φ,i) can be generated by:

$$\mathbf{s}(\mathbf{f}, \phi, \mathbf{i}) = \text{square}\left[2\pi \mathbf{f}\left(\frac{\mathbf{i}}{\text{refresh rate}}\right) + \phi\right] \tag{5}$$

where the function square [] generates a square wave of 50% duty cycle with levels 0 and 1, and i indicates the frame index. Nakanishi et al. used quad-phase coded flickering signals at

**Figure 8.** SSVEP BCI game interface: (a) training and (b) playing.


parietal and occipital areas (FPz, F3, F4, Fz, Cz, P1, P2, Pz, PO3, PO4, PO7, PO8, POz, O1,

The entire data epochs were correlated using common average reference (CAR) and then subjected to a band-pass filter with cut off frequencies 7–50 Hz with an infinite impulse response

Canonical correlation analysis (CCA) was used for target identification which used the refer-

ρ(XT *WXX*̂

ρ(XT *WX*̂

To validate the efficiency of the combined method, this study compared classification performance of the following five methods: (M1) a standard CCA-based method; (M2) a correlation analysis using a spatial filter derived from test set and training reference signals; (M3) a correlation analysis using a spatial filter derived from test set and since-cosine reference signals; (M4) a correlation analysis using a spatial filter derived from training reference signals and sine-cosine reference signals; and (M5) the combined method using the ensemble classifier described in Eq. (6). **Figure 10** shows the averaged accuracy (**Figure 10(a)**) and ITR (**Figure 10(b)**) across all subjects for the offline experiments. Results for different CCA-based methods were calculated with different data lengths from 1 to 4 s. It is evident that the four methods (M2, M3, M4, and

ρ(XT *WXY*, *X*̂

ρ

, *X*̂ *T WXX*̂)

*<sup>Y</sup>*, *X*̂ *T WX*̂ *Y*)

*T WXY*)

<sup>k</sup>) to identify the user's intention. The study developed

⎤

⎥

⎦

<sup>k</sup>) set signals with sine-cosine

SSVEP-Based BCIs

105

http://dx.doi.org/10.5772/intechopen.75693

(6)

(IIR) filter. Zero-phase forward and reverse IIR filtering were implemented.

**Figure 9.** Presentation of the 32-target visual stimuli using mixed frequency and phase coding.

⎡

ρ1 ρ2 ρ3 ρ4

⎤

⎡

⎢

⎣

⎥ ⎦ =

⎢ ⎣

M5) outperformed M1 under all conditions with different data lengths.

an ensemble classifier that correlates the test (X) and training (X̂

reference signals Y. A correlation vector ρ is defined as follows:

O2, and Oz).

ence from the SSVEP training data (X̂

=

S1: subject number 1, S2: subject number 2, LDA: linear discriminant analysis, sLDA: sparse LDA, SVM: support vector machine, RBF: radial basis function, WAT: wave atom transform, and BP: band power.

**Table 4.** Comparison of classification accuracy.

phases or 0, 90, 180, and 270, for frequencies 8–15 Hz with an interval of 1 Hz, hence providing 32 unique targets instead of just 8 as indicated in **Figure 8**.

The subjects were instructed to gaze at one out of the 32 visual stimuli (a target stimulus) for 4 s, and the other 31 targets were indicated in a random order in a run. At the beginning of each trial, a red rectangle marker (**Figure 9**) appeared for half a second highlighting the target stimulus. Subjects were asked to shift their gaze to the target within the same duration. After which, all the stimuli started to flicker simultaneously for 4 seconds. Seven runs were carried out for each subject EEG data were recorded by 16 electrodes over the

**Figure 9.** Presentation of the 32-target visual stimuli using mixed frequency and phase coding.

parietal and occipital areas (FPz, F3, F4, Fz, Cz, P1, P2, Pz, PO3, PO4, PO7, PO8, POz, O1, O2, and Oz).

The entire data epochs were correlated using common average reference (CAR) and then subjected to a band-pass filter with cut off frequencies 7–50 Hz with an infinite impulse response (IIR) filter. Zero-phase forward and reverse IIR filtering were implemented.

Canonical correlation analysis (CCA) was used for target identification which used the reference from the SSVEP training data (X̂ <sup>k</sup>) to identify the user's intention. The study developed an ensemble classifier that correlates the test (X) and training (X̂ <sup>k</sup>) set signals with sine-cosine reference signals Y. A correlation vector ρ is defined as follows:

$$\begin{aligned} \text{Parameters} \ \text{eigenvalues} \ \text{....} \ \text{Convexment} \ \text{row} \ \text{row} \ \text{row} \ \text{row} \ \text{row} \ \text{row} \ \text{row} \ \text{row} \\ \rho = \begin{bmatrix} \rho\_1\\ \rho\_2\\ \rho\_3\\ \rho\_4 \end{bmatrix} = \begin{bmatrix} \rho\\ \rho \left(\mathbf{X}^T \mathbf{W}\_{\text{XY}} \hat{\mathbf{X}}^T \mathbf{W}\_{\text{XY}}\right)\\ \rho \left(\mathbf{X}^T \mathbf{W}\_{\text{XY}} \hat{\mathbf{X}}^T \mathbf{W}\_{\text{XY}}\right)\\ \rho \left(\mathbf{X}^T \mathbf{W}\_{\text{XY}} \hat{\mathbf{X}}^T \mathbf{W}\_{\text{XY}}\right) \end{bmatrix} \tag{6}$$

To validate the efficiency of the combined method, this study compared classification performance of the following five methods: (M1) a standard CCA-based method; (M2) a correlation analysis using a spatial filter derived from test set and training reference signals; (M3) a correlation analysis using a spatial filter derived from test set and since-cosine reference signals; (M4) a correlation analysis using a spatial filter derived from training reference signals and sine-cosine reference signals; and (M5) the combined method using the ensemble classifier described in Eq. (6).

phases or 0, 90, 180, and 270, for frequencies 8–15 Hz with an interval of 1 Hz, hence providing

S1: subject number 1, S2: subject number 2, LDA: linear discriminant analysis, sLDA: sparse LDA, SVM: support vector

The subjects were instructed to gaze at one out of the 32 visual stimuli (a target stimulus) for 4 s, and the other 31 targets were indicated in a random order in a run. At the beginning of each trial, a red rectangle marker (**Figure 9**) appeared for half a second highlighting the target stimulus. Subjects were asked to shift their gaze to the target within the same duration. After which, all the stimuli started to flicker simultaneously for 4 seconds. Seven runs were carried out for each subject EEG data were recorded by 16 electrodes over the

32 unique targets instead of just 8 as indicated in **Figure 8**.

**Table 4.** Comparison of classification accuracy.

machine, RBF: radial basis function, WAT: wave atom transform, and BP: band power.

**Figure 8.** SSVEP BCI game interface: (a) training and (b) playing.

104 Evolving BCI Therapy - Engaging Brain State Dynamics

**Classifier Features Accuracy, % F1 Score**

LDA WAT 71.5 78.2 0.64 0.67

sLDA WAT 70.6 77.4 0.64 0.68

SVM, linear kernel WAT 75.5 79.3 0.64 0.68

SVM, RBF kernel WAT 78.7 82.2 0.68 0.71

**S1 S2 S1 S2**

BP 66.2 73.2 0.56 0.62

BP 68.4 73.5 0.59 0.61

BP 74.3 75.1 0.64 071

BP 74.0 77.4 0.63 0.67

**Figure 10** shows the averaged accuracy (**Figure 10(a)**) and ITR (**Figure 10(b)**) across all subjects for the offline experiments. Results for different CCA-based methods were calculated with different data lengths from 1 to 4 s. It is evident that the four methods (M2, M3, M4, and M5) outperformed M1 under all conditions with different data lengths.

The accuracy of SSVEP based BCI's is fairly high for most subjects with substantial amount of visual capabilities. However some subjects were not able to produce a significant change in the EEG with respect to the SSVEP stimuli. This condition is termed as BCI illiteracy [59]. This phenomena cause the failure of BCI for such subjects as the task is not performed due to minimal EEG activity. To counter this problem a novel approach of hybrid brain computer interfacing (hBCI) was proposed [60, 61]. The hBCI combines a standard BCI paradigm (SSVEP, P300, slow cortical potential (SCP) or event related synchronisation/de-synchronisation (ERS/ERD)), with another BCI signal or some other physiological signal. hBCI's are an emerging area of research where all possible combinations are being explored to increase system accuracy as well as eliminate the phenomena of BCI illiteracy. The hBCI's also address the problem of subject fatigue due to fixing of gaze at flickering targets for a longer duration, this fatigue is known to reduce the accuracy of the BCI due increase in the number of False Positive (FP) outcomes.

SSVEP-Based BCIs

107

http://dx.doi.org/10.5772/intechopen.75693

[1] Kolodziej M, Majkowski A, Rak R. A new method of EEG classification for BCI with feature extraction based on higher order statistics of wavelet components and selection with generic algorithms. In: Adaptive and Natural Computing Algorithms. Lecture Notes in Computer Science. Warsaw, Poland: Warsaw University of Technology;

[2] Donchin E, Ritter W, McCallu C. Cognitive Psychophysiology: The Endogenous Components of the ERP. Brain-Event Related Potentials in Man. New York: Academic; 1978 [3] Regan D. Human Brain Electrophysiology: Evoked Potentials and Evoked Magnetic

[4] Yijun W, Ruiping W, Xiaorong G, Bo H, Shangkai G. A practical VEP-based brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5] Jinghai Y, Derong J, Jianfeng H. Design and application of brain-computer InterfaceWeb browser based on VEP. In: Proceedings of the International Conference on Future Biomedical Information Engineering (FBIE'09); Sanya, China; 13-14 December, 2009. pp. 77-80

[6] Xiaorong G, Dingfeng X, Ming C, Shangkai G. A BCI-based environment controller for the motion-disabled. IEEE Transactions on Neural Systems and Rehabilitation

Fields in Science and Medicine. New York, NY, USA: Elsevier; 1989

**Author details**

Address all correspondence to: singlar@nitj.ac.in

Associate Professor, Dr. BR Ambedkar NIT, Jalandhar, Punjab, India

Rajesh Singla

**References**

2011;**6593**:280-289

2006;**14**:234-240

Engineering. 2003;**11**:137-140

**Figure 10.** (a) Target identification accuracy, (b) ITRs as functions of data length (from 1 to 4 s) calculated by different methods.

#### **2.3. Information transfer rate (ITR)**

The dynamic performance of any BCI can be evaluated by the information transfer rate (ITR) as introduced in [58]. This parameter depends upon three factors regarding the BCI


ITR (B) is defined as

$$\mathbf{B} = \mathbf{V} \left[ \log\_2 \mathbf{N} + \mathbf{P} \log\_2 \mathbf{P} + (1 - \mathbf{P}) \log\_2 \left( \frac{1 - \mathbf{P}}{\mathbf{N} - 1} \right) \right] \tag{7}$$

where, V = application speed in trials per second.


#### **3. Conclusion**

SSVEP proves to be the most widely used paradigm for BCI used for various different application for healthy as well as locked in patients due to the fact that SSVEP-BCI's require minimum user training. This is because the subject does not have to regulate his/her own brain activity to provide controlling input for the task at hand. The subjects have to merely focus their attention towards the flickering targets which is then converted to machine command by a trained classifier.

The accuracy of SSVEP based BCI's is fairly high for most subjects with substantial amount of visual capabilities. However some subjects were not able to produce a significant change in the EEG with respect to the SSVEP stimuli. This condition is termed as BCI illiteracy [59]. This phenomena cause the failure of BCI for such subjects as the task is not performed due to minimal EEG activity. To counter this problem a novel approach of hybrid brain computer interfacing (hBCI) was proposed [60, 61]. The hBCI combines a standard BCI paradigm (SSVEP, P300, slow cortical potential (SCP) or event related synchronisation/de-synchronisation (ERS/ERD)), with another BCI signal or some other physiological signal. hBCI's are an emerging area of research where all possible combinations are being explored to increase system accuracy as well as eliminate the phenomena of BCI illiteracy. The hBCI's also address the problem of subject fatigue due to fixing of gaze at flickering targets for a longer duration, this fatigue is known to reduce the accuracy of the BCI due increase in the number of False Positive (FP) outcomes.
