**4. Analysis and implementation**

Possible strategies to locate the missing person in the Adera interphase is indicated by the predicted fixations. Rather than following the original eye movement (pink circles), the player can retrace the steps and identify the missing person by following the predictions. Four areas indicating stress mood were detected. One of these is on a commercial widget at the right upper edge of the interphase. The neural point is detected inside the game interphase. The predicted and original (natural) response lying in the same cartesian coordinate of the Adera game interphase (**Figure 3**) shows possibility of a high performance of the control model. The stress and neutral mood are indicated of the three affect states generated. One interesting aspect is the stress point at the area where a question mark is located on the interphase just close to the position where we have a pointing black arrow. This icon (question mark) is there to provide suggestions on which direction/strategy to take in locating the missing person. The user was seen to be undecided whether to make a move on and make use of the lifeline

15:

16: *loop*:

20: end.

17: **if** size(emotion2) = [0, 1] ; **then** 18: *emotion*2 ← 1. else 19: *emotion*2 ← *emotion*2.

30 Human-Robot Interaction - Theory and Application

23: **goto** *loop*. 24: **close**;

29: **goto** *top*.

**3.6. Operator querying**

21: emotion1 = strmatch('stressed', PP12.Affectate(locs(m))); 22: emotion2 = strmatch('neutral', PP12.Affectate(locs(m)));

The mechanism involved is a direct synchronization of the two-sensor port while there is a querying model for the system output. This helps to identify optimal responses in the physiological measure that correlates to visual attention on the part of the user. The next exposition

Possible strategies to locate the missing person in the Adera interphase is indicated by the predicted fixations. Rather than following the original eye movement (pink circles), the player can retrace the steps and identify the missing person by following the predictions. Four areas

25: *X* ← *Response*.*MappedFixationPointX*(*locs*(*emotion*1)) ;. 26: *Y* ← *Response*.*MappedFixationPointY*(*locs*(*emotion*1)) ;. 27: *XX* ← *Response*.*MappedFixationPointX*(*locs*(*emotion*2)) ;. 28: *YY* ← *Response*.*MappedFixationPointY*(*locs*(*emotion*2)) ;.

discusses analysis and findings from the control system.

**4. Analysis and implementation**

**Figure 2.** Control system feedback configuration.

**Figure 3.** Detected affect state and correlating physiological reaction to Adera episode 1.

or simply ignore this icon, hence, the appearance of a stress indicator (neutral mood) on that area. The pattern or arrangement of fixations toward that direction indicates it might be the right strategy to take; these predicted points are indicated close to the question mark content. The user also experienced a stress mood, while looking at the advert section on the right side of the interphase. The participants' physiological reaction toward that phase indicates more phasic changes, hence, there is a higher emotion indicator (stress and neutral moods) toward the interphase with an average baseline response of 2.72*s*. At the point where a stress and a relaxed indicator intersects, a neutral indicator is produced (purple indicator).

On the yahoo interphase (**Figure 4**), there are different dynamic and static contents that can distract the user and induce both positive and negative emotions. The user feels a neutral mood toward the dynamic picture content having the headline "Silicon Vas reacts to Trump inauguration" which is indicated by the neutral points close to and on the picture content. The user's physiological reaction toward the interphase suggests an increase in amplitude between 20 and 40 s in the interaction. This interval correlates to the convoluted fixation points close to the dynamic picture content. The pattern for the predicted response lies in the same convoluted pattern as the original response.

The reaction of the user to the book interphase stimulus suggests confusion and an undecided strategy to take when locating an interesting peace of phrase he might find interesting. This is with an average baseline on 3.1*μ* which is quite high for this interphase. The natural law of attention is that the gaze is directed to the center of the book, as seen by the convoluted arrangement of fixations of both the predicted and natural eye movement on one point at the center of the book interphase stimuli. The pattern of the predicted eye movement suggests the possible strategy he could take on that area to locate an interesting piece of phrase. Three affect indicators were located on this interphase, two of which are neutral and the other a stress mood. The emotion of

**Figure 5.** Detected affect state on Adera episode 1. (a)Textbook with original fixation and prediction fixation,(b)Detected

A Control System for Detecting Emotions on Visual Interphase Stimulus

http://dx.doi.org/10.5772/intechopen.75873

33

emotion on Textbook,(c)Detected affect state and correlating physiological reaction to Textbook.

the user can be seen and is indicated by the three-emotion detector on the spot (**Figure 5**).

The state variables used to estimate the coefficient of the control systems were defined by the input parameters which the signal provided. The sensors were used to generate the primary data that

**5. Results**

**Figure 4.** Detected affect state and correlating physiological reaction to yahoo page.

A Control System for Detecting Emotions on Visual Interphase Stimulus http://dx.doi.org/10.5772/intechopen.75873 33

**Figure 5.** Detected affect state on Adera episode 1. (a)Textbook with original fixation and prediction fixation,(b)Detected emotion on Textbook,(c)Detected affect state and correlating physiological reaction to Textbook.

The reaction of the user to the book interphase stimulus suggests confusion and an undecided strategy to take when locating an interesting peace of phrase he might find interesting. This is with an average baseline on 3.1*μ* which is quite high for this interphase. The natural law of attention is that the gaze is directed to the center of the book, as seen by the convoluted arrangement of fixations of both the predicted and natural eye movement on one point at the center of the book interphase stimuli. The pattern of the predicted eye movement suggests the possible strategy he could take on that area to locate an interesting piece of phrase. Three affect indicators were located on this interphase, two of which are neutral and the other a stress mood. The emotion of the user can be seen and is indicated by the three-emotion detector on the spot (**Figure 5**).

#### **5. Results**

**Figure 4.** Detected affect state and correlating physiological reaction to yahoo page.

or simply ignore this icon, hence, the appearance of a stress indicator (neutral mood) on that area. The pattern or arrangement of fixations toward that direction indicates it might be the right strategy to take; these predicted points are indicated close to the question mark content. The user also experienced a stress mood, while looking at the advert section on the right side of the interphase. The participants' physiological reaction toward that phase indicates more phasic changes, hence, there is a higher emotion indicator (stress and neutral moods) toward the interphase with an average baseline response of 2.72*s*. At the point where a stress and a

On the yahoo interphase (**Figure 4**), there are different dynamic and static contents that can distract the user and induce both positive and negative emotions. The user feels a neutral mood toward the dynamic picture content having the headline "Silicon Vas reacts to Trump inauguration" which is indicated by the neutral points close to and on the picture content. The user's physiological reaction toward the interphase suggests an increase in amplitude between 20 and 40 s in the interaction. This interval correlates to the convoluted fixation points close to the dynamic picture content. The pattern for the predicted response lies in the

relaxed indicator intersects, a neutral indicator is produced (purple indicator).

same convoluted pattern as the original response.

32 Human-Robot Interaction - Theory and Application

The state variables used to estimate the coefficient of the control systems were defined by the input parameters which the signal provided. The sensors were used to generate the primary data that serves as the user attributes. These are the SCR, ST, and eye movement represented as fixations. Multiple polynomial orders were chosen to run the model. The best of these include polynomial order 1–3; this gives a precise representation of the physiological reaction to the dynamic contents on the interphase. This concept represents that which is applied to physical systems. This is very appropriate, if the goal is to predict and indicate emotion on visual interphase. We have to go beyond the normal approach to apply a multidimensional procedure to achieve the targeted objective. For the Adera Episode 1 interphase, both polynomial order one and three have the same phasic change when compared to polynomial order one. The phasic response of the MappedFX (input 3) with MappedFY (response variable) has the same but opposite reaction to the systems control; this is a positive and negative effect to the connection between the user and the Adera interphase. The SCR (input 1) has a positive effect in the connection, and this implies that it is a good indicator of the emotional response for the Adera interphase (**Figure 6**).

The magnitude of the response on all inputs has a positive impact to the systems' interphase between the webpage (**Figure 7a**) and the user. For this case, applying polynomial order one and two have the same phasic change compared to three. The input ST have the same magnitude and phasic change for all polynomial order. The different variations in phasic changes are indicated in the input 3 and input 4 to the response output. On the other hand, all inputs have the same magnitude and phasic change for the polynomial orders used in the text-user interphase interaction; using order three and two runs on the system on average signifies

**Figure 7.** Bode plot of the system on webpage interphase.

A Control System for Detecting Emotions on Visual Interphase Stimulus

http://dx.doi.org/10.5772/intechopen.75873

35

**Figure 8.** Comparing original and predicted response.

**Figure 6.** Bode plot of the system on Adera interphase.

A Control System for Detecting Emotions on Visual Interphase Stimulus http://dx.doi.org/10.5772/intechopen.75873 35

**Figure 7.** Bode plot of the system on webpage interphase.

serves as the user attributes. These are the SCR, ST, and eye movement represented as fixations. Multiple polynomial orders were chosen to run the model. The best of these include polynomial order 1–3; this gives a precise representation of the physiological reaction to the dynamic contents on the interphase. This concept represents that which is applied to physical systems. This is very appropriate, if the goal is to predict and indicate emotion on visual interphase. We have to go beyond the normal approach to apply a multidimensional procedure to achieve the targeted objective. For the Adera Episode 1 interphase, both polynomial order one and three have the same phasic change when compared to polynomial order one. The phasic response of the MappedFX (input 3) with MappedFY (response variable) has the same but opposite reaction to the systems control; this is a positive and negative effect to the connection between the user and the Adera interphase. The SCR (input 1) has a positive effect in the connection, and this implies that it is a

The magnitude of the response on all inputs has a positive impact to the systems' interphase between the webpage (**Figure 7a**) and the user. For this case, applying polynomial order one and two have the same phasic change compared to three. The input ST have the same magnitude and phasic change for all polynomial order. The different variations in phasic changes are indicated in the input 3 and input 4 to the response output. On the other hand, all inputs have the same magnitude and phasic change for the polynomial orders used in the text-user interphase interaction; using order three and two runs on the system on average signifies

good indicator of the emotional response for the Adera interphase (**Figure 6**).

34 Human-Robot Interaction - Theory and Application

**Figure 6.** Bode plot of the system on Adera interphase.

**Figure 8.** Comparing original and predicted response.

possibility of a good performance on the system model. The response output illustrates the most significant response magnitude at order 3 (**Figure 8b**).

[4] Demiral SB, Schlesewsky M, Bornkessel-Schlesewsky I. On the universality of language comprehension strategies: Evidence from turkish. Cognition. 2008;**106**(1):484-500

A Control System for Detecting Emotions on Visual Interphase Stimulus

http://dx.doi.org/10.5772/intechopen.75873

37

[5] Isiaka F, Mwitondi K, Ibrahim A.Window based model for simulation of integrative human physiological response to webpages. In: Computing and Communication (IEMCON), 2015

[6] Isiaka F, Mwitondi KS, Ibrahim AM. Detection of natural structures and classification of HCI-HPR data using robust forward search algorithm. International Journal of

[7] Kamon E, Pandolf K, Cafarelli E. The relationship between perceptual information and physiological responses to exercise in the heat. Journal of Human Ergology. 1974;**3**(1):

[8] Karnopp DC, Margolis DL, Rosenberg RC. System Dynamics: Modeling, Simulation,

[9] Kretzschmar F, Pleimling D, Hosemann J, Fussel S, Bornkessel-Schlesewsky I, Schlesewsky M. Subjective impressions do not mirror online reading effort: Concurrent eegeyetracking evidence from the reading of books and digital media. PLoS One.

[10] Liversedge SP, Blythe HI. Lexical and sublexical influences on eye movements during

[11] Mindfield. eSence temperature, mindfield biofeedback sytems. eSence Skin Temperature-

[12] Mizokawa T.Control system for controlling object using pseudo-emotions and pseudo-

[14] Paulson LD. Building rich web applications with ajax. Computer. 2005;**38**(10):14-17

[15] Petrushin VA. Detecting emotions using voice signal analysis, US Patent 7,222,075, May

[16] Ramakrishnan S. Recognition of emotion from speech: A review. In: Speech Enhance ment, Modeling and Recognition-Algorithms and Applications. Rijeka: InTech; 2012 [17] Schneider-Hufschmidt M, Malinowski U, Kuhme T. Adaptive User Interfaces: Principles

[18] Widyantoro DH, Ioerger TR, Yen J. An adaptive algorithm for learning changes in user interests. In: Proceedings of the Eighth International Conference on Information and

personality generated in the object, US Patent 6,230,111, May 8 2001

[13] Niedenthal PM, Ric F. Psychology of Emotion. Psychology Press; 2017

International Conference and Workshop on. IEEE; 2015. pp. 1-8

and Control of Mechatronic Systems. John Wiley and Sons; 2012

Intelligent Computing and Cybernetics. 2016;**9**(1):23-41

reading. Lang & Ling Compass. 2007;**1**(1-2):17-31

and Practice. Elsevier Science Inc.; 1993

Knowledge Management. ACM; 1999. pp. 405-412

45-54

2013;**8**(2):e56178

22 2007

Handbook. 2014;**1**(1):1-12
