**Brain-Computer Interface Using Near-Infrared Spectroscopy for Rehabilitation**

Kazuki Yanagisawa, Hitoshi Tsunashima and Kaoru Sakatani *Nihon University Japan* 

## **1. Introduction**

338 Infrared Spectroscopy – Life and Biomedical Sciences

Paschalis, E. P., Verdelis, K., Doty, S. B., Boskey, A. L., Mendelsohn, R., & Yamauchi, M.

Peterson, B., Whang, P. G., Iglesias, R., Wang, J. C., & Lieberman, R. J. (2004).

Pietrzak, W., Perns, S. V., Keyes, J., Woodell-May, J., & McDonald, N. (2005). Demineralized

Pietrzak, W. S., Ali, S. N., Chitturi, D., Jacob, M., & Woodell-May, J. E. (2009). BMP depletion

Roeges, N. P. G. (1994). *A guide to the complete interpretation of infrared spectra of organic* 

Rogers, K. D., & Daniels, P. (2002). An X-ray diffraction study of the effects of heat treatment

Saito, M., & Marumo, K. (2009). Collagen cross-links as a determinant of bone quality: a

Shemesh, A. (1990). Crystallinity and diagenesis of sedimentary apatites. *Geochimica Et* 

Silva, L. d. M. S., Ebacher, V., Liu, D., McKay, H., Oxland, T. R., & Wang, R. (2005). Elasticity

Tzaphlidou, M. (2005). The role of collagen in bone structure: An image processing

Verdelis, K. (2005). *Study of mineral and matrix maturation in dentin.* Ph.D. Dissertation,

Verdelis, K., Lukashova, L., Wright, J. T., Mendelsohn, R., Peterson, M. G. E., Doty, S., et al. (2007). Maturational changes in dentin mineral properties. *Bone, 40*(5), 1399-1407. Viguet-Carrin, S., Follet, H., Gineyts, E., Roux, J. P., Munoz, F., Chapurlat, R., et al. (2010).

Viguet-Carrin, S., Garnero, P., & Delmas, P. D. (2005). The role of collagen in bone strength.

Association between collagen cross-links and trabecular microarchitecture

*Materials Research Society Symposium Proceedings, 874*, L 5.8.1-L 5.8.6. Torroni, A. (2009). Engineered Bone Grafts and Bone Flaps for Maxillofacial Defects: State of

the Art. *Journal of Oral and Maxillofacial Surgery, 67*(5), 1121-1127.

Urist, M. R. (1965). Bone formation by autoinduction. *Science, 150*, 893–899.

properties of human vertebral bone. *Bone, 46*(2), 342-347.

*structures*. Chichester, England: Wiley & Sons.

*Osteoporosis International, 21*(2), 195-214.

*Cosmochimica Acta, 54*(9), 2433-2438.

approach. *Micron, 36*(7-8), 593-601.

Faculty of North Carolina, Chapel Hill, USA.

*Osteoporosis International, 17*(3), 319-336.

on bone mineral microstructure. *Biomaterials, 23*, 2577–2585.

implications for graft preparation. *Cell and Tissue Banking, 12*(2), 81-88. Rho, J.-Y., Kuhn-Spearing, L., & Zioupos, P. (1998). Mechanical properties and the hierarchical structure of bone. *Medical Engineering & Physics 20*, 92–102. Robling, A. G., Castillo, A. B., & Turner, C. H. (2006). Biomechanical and Molecular

*and Mineral Research, 16*(10).

*Acta Physica Polonica A, 115*(2).

*Ankle Surgery, 44*, 345-353.

498.

(2001). Spectroscopic characterization of collagen cross-links in bone. *Journal of Bone* 

Osteoinductivity of commercially available demineralized bone matrix. Preparations in a spine fusion model. *Journal of Bone Joint Surgery, 86*, 2243–2250. Petibois, C., Wehbe, K., Belbachir, K., Noreen, R., & Déléris, G. (2009). Current Trends in the

Development of FTIR Imaging for the Quantitative Analysis of Biological Samples.

bone matrix graft: a scientific and clinical case study assessment. *Journal of Foot and* 

occurs during prolonged acid demineralization of bone: characterization and

Regulation of Bone Remodelling. *Annual Review of Biomedical Engineering, 8*, 455–

possible explanation for bone fragility in aging, osteoporosis, and diabetes mellitus.

and Viscoelasticity of Human Tibial Cortical Bone Measured by Nanoindentation.

Currently, the Brain Computer Interface (BCI) is being studied vigorously. BCI extracts thoughts in the human brain as cranial nerve information and uses the information as inputs to control machinery and equipment. Fig. 1 describes schematic BCI system. If this system enables operating machinery and equipment directly from cranial nerve information without the subject moving his or her hands and feet, it can be applied to care-taking robots and rehabilitation for physically handicapped individuals.

#### Fig. 1. Schematic of BCI system

BCI systems can be divided into two forms. The invasive form reads cranial nerve information using electrodes embedded directly into the brain. The non-invasive form reads cranial nerve activity from the surface of the head using near infrared spectroscopy (NIRS) or electroencephalography (EEG). as an example of invasive form, Donoghue LR. et al. extracted nerve activity of primary motor area and controlled a robot hand and mouse cursor (Hochberg LR et. Al, 2006). Though the invasive form has high signal accuracy, it imposes a heavy load on the user (e.g., surgery and infections after surgery).

Brain-Computer Interface Using Near-Infrared Spectroscopy for Rehabilitation 341

Furthermore, if the change in concentration ( *C* ) is assumed to be proportional to the changes in oxygenated hemoglobin ( *Xoxy* ) and deoxygenated hemoglobin ( *Xdeoxy* ), the

() [ () () ] *Abs l X*

The absorption coefficients of oxygenated hemoglobin and deoxygenated hemoglobin at each

obtained by performing measurements with near-infrared rays of two different wavelengths and solving simultaneous equations for Equation 3. However, the physical quantity obtained here is the product of the change in concentration and the averaged path length. In general, this averaged path length *l* varies greatly from one individual to another, and from one part

In general, changes in oxygenated hemoglobin and deoxygenated hemoglobin when the brain is activated and restored to the original state exhibit the trend illustrated in Figure 2

of the brain to another. Caution must therefore be exercised in evaluating the results.

Fig. 2. Schematic of changes in hemoglobin concentration due to neural activity


0

Relativconerai

Relative concentration

1

2

3

Figure 3 depicts a robot control system that uses NIRS-BCI. This system is composed of a cerebral function measurement section, a feature extraction and recognition section, and a device control section. In the cerebral function measurement section, the subject's oxy-Hb is measured using a multichannel NIRS instrument, OMM-3000, made by Shimadzu Corporation, Japan. The system operated at three different wavelengths of 780,805,830nm. In the feature extraction and recognition section, the threshold is obtained by analyzing the original signal of oxy-Hb that was measured. When oxy-Hb after analysis exceeds the threshold obtained in the feature extraction and recognition section, the on signal is sent to the device control section in order to enable rotation of the joint of the robot arm (MR-999;

0 2 4 6 8 10 12

Time [s]

**3. Brain-computer interface using NIRS** 

 

 *i oxy i oxy deoxy i deox X <sup>y</sup>* . (3)

> Oxygenated hemoglobin

> > Deoxygenated hemoglobin

, are known. As a result, *oxy l X* and *deoxy l X* can be

following relational expression can be obtained:

wavelength, ( ) *oxy <sup>i</sup>* 

(Huettel, 2004).

Elekit, Japan).

 and ( ) *deoxy <sup>i</sup>* 

 

Therefore, the non-invasive form has a wider applicable range. In a study on the noninvasive form, Pfurtscheller have developed BCI that physically handicapped individuals can control a character of the virtual reality (Pfurtscheller G. et. Al, 2006), electricaldriven hand (Pfurtscheller G. et. al, 2000) and functional electrical stimulation (FES) ( Pfurtscheller G., 2003). Vaughan TM et al. have developed in prototype systems for everyday use in people's homes for locked-in patients (Vaughan TM, 2006). Many other studies of BCI have used EEG. However, EEG has low spatial resolution and it is vulnerable to electrical noise. In contrast, NIRS imposes fewer restrictions on body movement than EEG does and is more resistant to electric noise, so the load imposed on the user is less and electronic devices have no influence. And, NIRS has high space resolution. Therefore, the present study focuses on BCI that uses NIRS.

In an earlier study, Nagaoka et al. developed a NIRS-BCI rehabilitation system (Nagaoka T, 2010). In their study, electric stimuli corresponding to cranial nerve information are applied to the user's biceps brachii muscle by setting a threshold on signals measured from NIRS, in order to cause the elbow joint to move. But, Signals measured by NIRS are unstable because they include signals components that are irrelevant to the subject (e.g., noise of measuring instruments, heartbeat, and respiration). Moreover, a processing method for NIRS signals has not yet been established. For these reasons, it is difficult obtain a high identification rate by merely setting a simple threshold on NIRS signals. And this BCI system lacks versatility, because it is large system using multi-channel NIRS equipment.

In this study, we propose a new detection method that uses oxy-Hb and its differential as indexes for application to the NIRS-BCI rehabilitation system, detecting brain activity from the data measured using NIRS. First, we develop a BCI system to control robot arm using NIRS, and confirm that NIRS-BCI system can control machine and device. Next, we apply NIRS-BCI system developed for rehabilitation.

#### **2. Near-infrared spectroscopy (NIRS)**

Using near-infrared rays, NIRS non-invasively measures changes in cerebral blood flow. The principle of measurement was developed by Jöbsis (1977) , based on the measurement of hemoglobin oxygenation in the cerebral blood flow.

In uniformly distributed tissue, incident light is attenuated by absorption and scattering. The following expression, a modified Lambert-Beer law, was therefore used:

$$Abs = -\log(I\_{out} \; / \; I\_{in}) = \varepsilon \overline{\text{lC}} + S \; . \tag{1}$$

Here, *Iin* is the irradiated quantity of light; *Iout* is the detected quantity of light; is the absorption coefficient; *C* is the concentration; *l* is the averaged path length; and *S* is the scattering term.

If it is assumed that no scattering changes in brain tissue occur during activation of the brain, the change in absorption across the activation can be expressed by the following expression:

$$
\Delta A \text{bs} = -\log(\Delta I\_{out} \; / \, \Delta I\_{in}) = \varepsilon \overline{l} \, \Delta \mathbb{C} (\Delta X\_{oxy} \, \_t \Delta X\_{deoxy}) \, . \tag{2}
$$

Therefore, the non-invasive form has a wider applicable range. In a study on the noninvasive form, Pfurtscheller have developed BCI that physically handicapped individuals can control a character of the virtual reality (Pfurtscheller G. et. Al, 2006), electricaldriven hand (Pfurtscheller G. et. al, 2000) and functional electrical stimulation (FES) ( Pfurtscheller G., 2003). Vaughan TM et al. have developed in prototype systems for everyday use in people's homes for locked-in patients (Vaughan TM, 2006). Many other studies of BCI have used EEG. However, EEG has low spatial resolution and it is vulnerable to electrical noise. In contrast, NIRS imposes fewer restrictions on body movement than EEG does and is more resistant to electric noise, so the load imposed on the user is less and electronic devices have no influence. And, NIRS has high space resolution. Therefore, the present study focuses on

In an earlier study, Nagaoka et al. developed a NIRS-BCI rehabilitation system (Nagaoka T, 2010). In their study, electric stimuli corresponding to cranial nerve information are applied to the user's biceps brachii muscle by setting a threshold on signals measured from NIRS, in order to cause the elbow joint to move. But, Signals measured by NIRS are unstable because they include signals components that are irrelevant to the subject (e.g., noise of measuring instruments, heartbeat, and respiration). Moreover, a processing method for NIRS signals has not yet been established. For these reasons, it is difficult obtain a high identification rate by merely setting a simple threshold on NIRS signals. And this BCI system lacks versatility,

In this study, we propose a new detection method that uses oxy-Hb and its differential as indexes for application to the NIRS-BCI rehabilitation system, detecting brain activity from the data measured using NIRS. First, we develop a BCI system to control robot arm using NIRS, and confirm that NIRS-BCI system can control machine and device. Next, we apply

Using near-infrared rays, NIRS non-invasively measures changes in cerebral blood flow. The principle of measurement was developed by Jöbsis (1977) , based on the measurement

In uniformly distributed tissue, incident light is attenuated by absorption and scattering.

log( / ) *A out in bs I I lC S*

absorption coefficient; *C* is the concentration; *l* is the averaged path length; and *S* is the

If it is assumed that no scattering changes in brain tissue occur during activation of the brain, the change in absorption across the activation can be expressed by the following

> log( /) ( , ) *Abs I I l C X X out in oxy deoxy*

Here, *Iin* is the irradiated quantity of light; *Iout* is the detected quantity of light;

. (1)

. (2)

is the

The following expression, a modified Lambert-Beer law, was therefore used:

because it is large system using multi-channel NIRS equipment.

NIRS-BCI system developed for rehabilitation.

**2. Near-infrared spectroscopy (NIRS)** 

of hemoglobin oxygenation in the cerebral blood flow.

BCI that uses NIRS.

scattering term.

expression:

Furthermore, if the change in concentration ( *C* ) is assumed to be proportional to the changes in oxygenated hemoglobin ( *Xoxy* ) and deoxygenated hemoglobin ( *Xdeoxy* ), the following relational expression can be obtained:

$$
\Delta A \text{bs} (\mathcal{X}\_{\text{i}}) = \overline{I} \left[ \mathcal{E}\_{\text{oxy}} (\mathcal{X}\_{\text{i}}) \Delta X\_{\text{oxy}} + \mathcal{E}\_{\text{deoxy}} (\mathcal{X}\_{\text{i}}) \Delta X\_{\text{deoxy}} \right]. \tag{3}
$$

The absorption coefficients of oxygenated hemoglobin and deoxygenated hemoglobin at each wavelength, ( ) *oxy <sup>i</sup>* and ( ) *deoxy <sup>i</sup>* , are known. As a result, *oxy l X* and *deoxy l X* can be obtained by performing measurements with near-infrared rays of two different wavelengths and solving simultaneous equations for Equation 3. However, the physical quantity obtained here is the product of the change in concentration and the averaged path length. In general, this averaged path length *l* varies greatly from one individual to another, and from one part of the brain to another. Caution must therefore be exercised in evaluating the results.

In general, changes in oxygenated hemoglobin and deoxygenated hemoglobin when the brain is activated and restored to the original state exhibit the trend illustrated in Figure 2 (Huettel, 2004).

Fig. 2. Schematic of changes in hemoglobin concentration due to neural activity

#### **3. Brain-computer interface using NIRS**

Figure 3 depicts a robot control system that uses NIRS-BCI. This system is composed of a cerebral function measurement section, a feature extraction and recognition section, and a device control section. In the cerebral function measurement section, the subject's oxy-Hb is measured using a multichannel NIRS instrument, OMM-3000, made by Shimadzu Corporation, Japan. The system operated at three different wavelengths of 780,805,830nm. In the feature extraction and recognition section, the threshold is obtained by analyzing the original signal of oxy-Hb that was measured. When oxy-Hb after analysis exceeds the threshold obtained in the feature extraction and recognition section, the on signal is sent to the device control section in order to enable rotation of the joint of the robot arm (MR-999; Elekit, Japan).

Brain-Computer Interface Using Near-Infrared Spectroscopy for Rehabilitation 343

●Illuminator Detector Channel

●

●

● ●

● ● ●

28 29 30 31 32 33 34 35 36 37 38 39 40 41

25 26 27

● ●

● ●

● ●

●

● ●

● ● 42 43 44 45 46 47 48

oxy-Hb ON/OFF

OFF

ON

● ●

●

4 5 6 7 8 9 10 11 12 13 14 15 16 17

1 2 3

● ●

●

●

●

● ●

●

● ● 18 19 20 21 22 23 24

●

●

Fig. 4. Position of optical fibers and channels for recording NIRS signals (grasping tasks:

R T R RR T T

NIRS signals also include signals that are not related to brain activity (e.g., measuring device noise, influences of respiration, and fluctuations of blood pressure). Therefore, to evaluate brain activity in detail, it is necessary to remove these extraneous signals. For this purpose, we selected the channels that have a good S/N ratio and exhibit remarkable task-related fluctuations (44 channels (contralateral motor cortex) in this case), and subjected the NIRS signals obtained from them to magnetic resonance angiogram analysis (MRA) (Kojima et. al, 2006, Yanagisawa et. al, 2009, Tsunashima et. al, 2009) through discrete wavelet

Time (s)

100 150 200 250

Fig. 5. Result of ON/OFF decision using conventional method

transformation to decompose and reconfigure the signals.

**4. Signal processing method for NIRS-BCI** 

**4.1 Detection method of brain activity** 

matrix, 48 channels)

Hb -0.04 -0.02 0 0.02 0.04 0.06 0.08

conc. (mM

 cm)

Fig. 3. BCI robot control system

Subjects were conducted using the robot arm control system depicted in Fig. 1. Five trials were carried out, with each trial consisting of 10sec of pre-task rest, 30sec of task, and 10sec of post-task rest. The first two trials were defined as the learning stage, where the feature extraction and recognition section learned the fluctuation pattern of the subject's oxy-Hb without moving the robot. In the third and subsequent trials, the robot's arm was rotated according to the learned pattern corresponding with oxy-Hb. The subject was instructed to perform grasping during tasks and instructed to rest during the rest time. The motor area was selected as the measurement site.

Optical fibers were arranged in 4×4 matrices on the right and left sides to perform measurement, with a total of 48 channels (Fig.4). The subjects were two healthy male

volunteers in their twenties. Prior informed consent was obtained from all subjects, in an effort to ensure full consideration of their safety and protection of their human rights.

In the detection method that used a simple threshold, the moving average was obtained. The threshold was set to 20% of the maximum oxy-Hb during the first two trials; detection was made in the third and subsequent trials, and an on state was judged when oxy-Hb exceeded the threshold.

Changes in concentration of oxy-Hb and the detection results for grasping tasks are depicted in Fig.5. ON detections were made in tasks from the first to second trials. OFF decisions in rest were wrong. ON and OFF detections alternated within a short time, indicating instability.

Subjects were conducted using the robot arm control system depicted in Fig. 1. Five trials were carried out, with each trial consisting of 10sec of pre-task rest, 30sec of task, and 10sec of post-task rest. The first two trials were defined as the learning stage, where the feature extraction and recognition section learned the fluctuation pattern of the subject's oxy-Hb without moving the robot. In the third and subsequent trials, the robot's arm was rotated according to the learned pattern corresponding with oxy-Hb. The subject was instructed to perform grasping during tasks and instructed to rest during the rest time. The motor area

Optical fibers were arranged in 4×4 matrices on the right and left sides to perform

volunteers in their twenties. Prior informed consent was obtained from all subjects, in an effort to ensure full consideration of their safety and protection of their human rights.

In the detection method that used a simple threshold, the moving average was obtained. The threshold was set to 20% of the maximum oxy-Hb during the first two trials; detection was made in the third and subsequent trials, and an on state was judged when oxy-Hb

Changes in concentration of oxy-Hb and the detection results for grasping tasks are depicted in Fig.5. ON detections were made in tasks from the first to second trials. OFF decisions in rest were wrong. ON and OFF detections alternated within a short time, indicating

measurement, with a total of 48 channels (Fig.4). The subjects were two healthy male

Fig. 3. BCI robot control system

was selected as the measurement site.

exceeded the threshold.

instability.

Fig. 4. Position of optical fibers and channels for recording NIRS signals (grasping tasks: matrix, 48 channels)

Fig. 5. Result of ON/OFF decision using conventional method
