**4. Hybrid approach of human computer interaction**

To increase HCI efficiency, hybrid approaches offer good results. Three physiological signals: blood volume pulse, galvanic skin response and pupil diameter, to automatically monitor stress were used, successfully (Zhai et al., 2005). A vision-based multimodal human computer interface system using eye and hand motion tracking was developed. This vision-based virtual interface integrates the function of the motion tracking of eye blinking and hand gestures with the function of their recognition as a virtual interface (Shin & Chun, 2007). EEG error-related signals present a hybrid approach for HCI. This approach uses human gestures to send commands to a computer and exploits brain activity to provide implicit feedback about the recognition of such commands (Chavarriaga et al., 2010).

### **4.1 Progressing study for hybrid multi-modal human computer interaction**

In order to contribute to the HCI field, a novel multi-modal integrated design is completed and establishing an efficient communication and control channel for individuals with motor neuron diseases such as ALS has been continuing. The preliminary results are promising. Using experience in relevant fields such as EEG (Usakli & Gencer, 2007a; Usakli & Gencer, 2007b), EOG (Usakli & Gurkan, 2010; Usakli et al., 2009), BCI prototyping (Erdogan et al, 2009), feature extraction (Usakli, 2010), this novel system attempts to increase usefulness and performance. HR and GSR signals are to be processed with the bio-signals mentioned above, simultaneously. Evaluation of these signals is whether individually or combined, depends on the user. The situation of the severity of the disease also determines the mode of the hybrid multi-mode operation. Through focusing on one of the tasks and sending correct or wrong messages, the bio signals of the subjects will be acquired and processed. Detecting these changes can be used to send data without speaking or control the device.

Human Computer Interactions for Amyotrophic Lateral Sclerosis Patients 697

Akan, B. & Argunsah, A.O. (2007). A Human-Computer Interface (HCI) Based on

Allison, R.S.; Eizenman, M. & Cheung, B. (1996). Combined Head and Eye Tracking System

Astolfi, L.; Gonzalez Andino, S.; De Vico Fallani, F. & Babiloni, F. (2010). Processing of Brain

Ayaz, H.; Izzetoglu, M.; Bunce, S.; Heiman-Patterson & T. Onaral, B. (2007). Detecting

Barea, R.; Boquete, L.; Mazo, M. & Lopez, E. (2002). System for Assisted Mobility Using

*Engineering, IEEE Transactions on*, Vol. 10 4, pp. 209 – 218, ISSN: 1534-4320 Bunce, S.; İzzetoglu, M.; İzzetoglu, K.; Onaral, B. & Pourrezaei, K. (2006). Functional Near

Buttfield, A.; Ferrez, P.W. & Millan, Jd.R. (2006). Towards a Robust BCI: Error Potentials

Chance, B.; Anday, E.; Nioka, S.; Zhou, S.; Hong, L.; Worden, .K.; Li, C.; Murray, T.;

Chavarriaga, R.; Biasiucci, A.; Forster, K.; Roggen, D.; Troster, G. & Millan Jdel, R. (2010).

Chen, S.; Tsai T. & Luo, C. (1999). Portable Clinical EOG Instrument System*, Proc. of the First Joint BMES/EMBS Conference*, pp.858-859, Atlanta, USA, 13-16 Oct. 1999. Chung-Hsien, K.; Yi-Chang, C.; Hung-Chyun, C. & Jia-Wun S. (2009). Eyeglasses based

Daud, W.M.B.W. & Sudirman, R. (2010). A Wavelet Approach on Energy Distribution of Eye

Deng, L.Y.; Chun-Liang Hsu Tzu-Ching Lin Jui-Sen Tuan Yung-Hui Chen, (2009). EOG-

*2010 IEEE Symposium on*, Penang, Malaysia, 3-5 Oct. 2010.

Brain Function, Non-invasively, With Light. *Optics Express*, 2, 10.

*Applications SIU 2007 IEEE 15th*, Eskisehir, Turkey, 11-13 June 2007.

*Computational Intelligence and Neuroscience*, 2010, ID 934180, pp.1-2.

*Engineering*, vol.43, no.11, pp.1073-1082.

*Biomedical Engineering*, vol.30, no.12, pp.781-786.

*Transactions on*, Vol. 14 2, pp. 164 - 168ISSN: 1534-4320

Argentina, Aug. 31-Sept. 4 2010.

4751, ISSN: 1062-922X, 11-14 Oct. 2009.

China, ISBN: 978-1-4244-3702-3, 2009

Electrooculogram (EOG) for Handicapped, *Signal Processing and Communications* 

for Dynamic Testing of the Vestibular System, *IEEE Trans. on Biomedical* 

Signals by Using Hemodynamic and Neuroelectromagnetic Modalities,

Cognitive Activity Related Hemodynamic Signal for Brain Computer Interface Using Functional Near Infrared Spectroscopy, *Neural Engineering, 2007. CNE'07. 3rd International IEEE/EMBS Conference on,* 342, Kohala Coast, HI, USA, 2-5 May 2007. Bahill, A.T. (1982). Prediction Final Eye Position Halfway Through a Saccade, *IEEE Trans. on* 

Eye Movements Based on Electrooculography, *Neural Systems and Rehabilitation* 

Infrared Spectroscopy: An Emerging Neuroimaging Modality, *IEEE Engineering in Medicine and Biology Magazine, Special Issue on Clinical Neuroengineering*, 25(4): pp.

and Online Learning, *Neural Systems and Rehabilitation Engineering, IEEE* 

Ovetsky, Y.; Pidikiti, D. & Thomas, R. (1998). A Novel Method for Fast Imaging of

Adaptation of Hybrid Human-computer Interaction Systems Using EEG Errorrelated Potentials, *Conference Proceedings IEEE Eng Med Biol Soc*. pp. 4226-4229,

electrooculography human-wheelchair interface Systems, *Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on*, San Antonio, TX, USA, pp. 4746 –

Movement Potential Towards Direction, *Industrial Electronics & Applications (ISIEA),* 

Based Signal Detection and Verification for HCI, *Machine Learning and Cybernetics, 2009 International Conference on*, 12-15 July 2009, Vol. 6, 3342, Baoding, Hebei,

**6. References** 

54-62.

### **4.2 Design rationale**

The novel system is microcontroller based and battery powered. Data is transferred via optic fiber. To remove the dc level and 50-Hz power line noise, differentiating approach is used. This approach is more successful and practical than the classical methods in the application. After signal conditioning; including filtering and amplification, the analog signal is digitized (at least 12 effective bits) at variable sampling rates and then transferred to the PC via optic fiber. To classify bio-signals, feature extraction with the fractal approach (Usakli, 2010) and wavelet transformed data is applied to artificial neural networks for classification. By using a user-friendly interface, the virtual keyboard and controlling pad allows messages to be given, and some other needs such as cleanup or medical assistance can be selected.

The integrated HCI system provides with high 1) efficiency, 2) usefulness, 3) robustness 4) accurate and reliable output, 5) fast response, 6) user friendly, 7) flexibility, 8) cheap, and 9) designed available components.
