**3.3. Textbook**

The textbook (The Designer) incentive involves locating an interesting phrase from a stimulus page that captures the reader's attention at a single glance. All the tasks are contrived, simply to induce slight stress so we can observe the amplitudes or increase in physiological response in reaction to the visual interphase.

#### **3.4. Physiological measures**

The physiological measures adopted for the experiment includes SCR, skin temperature (ST), and eye movement (saccade and fixations). These user attributes are used to measure changes in reaction to dynamic contents on the visual interphase.


#### **3.5. Hypothesis generator**

The concepts behind modeling physiological processes involve setting to get a significant accuracy and a prediction focus close to original data model; the model adopts the concepts for physical processes on dynamic systems [8], a least squares technique applied to system controls. For a user attribute saved from the sensors, the entire system is represented by the expression in Eqs. (1) and (2); the model fit to data represents prediction focus, 4 min from the detected fixations, and corresponding stress levels.

$$\frac{du}{dy} = Cu(y) + Fr(y) + e\tag{1}$$

e. The input generator is a discrete time-identified model fit representing the set of values the physiological reaction processes can take in response to dynamic visual stimuli. The primary data sources are measured in frequency (Herz and rads) and contain both categorical and numeric variables that contribute to making predictions on the multiple response output using multiple inputs (MIMO), for this case, *<sup>y</sup> and <sup>y</sup> <sup>m</sup>* are the response outputs (**Figure 2**). This represents the affects' states and fixations. The model is tested on the different estimated polynomial orders of the differential equation u(y). An identity state space object is created with C as the estimated initial state for the model, that is, the possible set of values the process can take, F is the estimated coefficients as a product of physiological parameters, G is the estimated output, and H the transformation matrix with noise e; y(m) is used to represent other response variables like the eye movement (fixations on the interphases). The threshold is set

A Control System for Detecting Emotions on Visual Interphase Stimulus

http://dx.doi.org/10.5772/intechopen.75873

29

*thresh* = 0.5(*meanamplitude* − *minimumSCR*) (3)

The alternate hypothesis is chosen if the rules does not apply to the null hypothesis, that is, H0: stress is constant if *thresh* <sup>&</sup>lt; *meanamplitude*, *Ha*:stress fluctuates if thresh <sup>&</sup>gt; *meanamplitude*. The affect states to identify are stress, relaxed, and a neutral mood on the interphase. To detect the affect state, control is directed to a logical output in a loop which has a place holder for the dimensions of

The module to identify the optimal responses correlating to user's mood is given in the following steps (Algorithm 1), the 'FINDPEAKS' function detects and finds phasic changes in physiological signals and high-level tonic phases (Appendix 1). Red indicates stress; blue and purple indicate relaxed and a neutral mood, respectively. The predicted fixations on the visual interphase indicate possible moves or positions for the user to reach

minpeakdistanc e′

,15;)

based on unique baseline such that:

the physiological response.

**Algorithm 1.** Algorithm to detect affect state on visual interphase.

6: **if** *mean*(*peaks*) > = thresh < = baseline **then return** false

(mean(amplitude) − min(eda)) ;

2: [*peaks*, *locs*] ← findpeaks(Response.eda,magenta'

their goal.

5: *top*:

9:

11: else.

13: end. 14: end.

1: **procedure** findpeaks

3: *m* ← length(locs) ; 4: *thresh* ← 0.5<sup>∗</sup>

7: disp('stressed')

10: disp('neutral')

12: disp('relaxed')

8: elseif mean(peaks) < = thresh > = baseline.

$$y(m) = Gr(\mathcal{Y}) + Wr(\mathcal{Y}) + e \tag{2}$$

where *y*(*m*) is the response variables (stress levels) that determine coefficients of physiological reactions with computed variables *r*(*y*), *C*, *F*, *G*, and *H* are the estimated coefficients with noise

**Figure 1.** Fixations and saccades of eye movement.

e. The input generator is a discrete time-identified model fit representing the set of values the physiological reaction processes can take in response to dynamic visual stimuli. The primary data sources are measured in frequency (Herz and rads) and contain both categorical and numeric variables that contribute to making predictions on the multiple response output using multiple inputs (MIMO), for this case, *<sup>y</sup> and <sup>y</sup> <sup>m</sup>* are the response outputs (**Figure 2**). This represents the affects' states and fixations. The model is tested on the different estimated polynomial orders of the differential equation u(y). An identity state space object is created with C as the estimated initial state for the model, that is, the possible set of values the process can take, F is the estimated coefficients as a product of physiological parameters, G is the estimated output, and H the transformation matrix with noise e; y(m) is used to represent other response variables like the eye movement (fixations on the interphases). The threshold is set based on unique baseline such that:

$$thresh = 0.5(mean\_{amplitude} - minimum\_{SCR})\tag{3}$$

The alternate hypothesis is chosen if the rules does not apply to the null hypothesis, that is, H0: stress is constant if *thresh* <sup>&</sup>lt; *meanamplitude*, *Ha*:stress fluctuates if thresh <sup>&</sup>gt; *meanamplitude*. The affect states to identify are stress, relaxed, and a neutral mood on the interphase. To detect the affect state, control is directed to a logical output in a loop which has a place holder for the dimensions of the physiological response.

The module to identify the optimal responses correlating to user's mood is given in the following steps (Algorithm 1), the 'FINDPEAKS' function detects and finds phasic changes in physiological signals and high-level tonic phases (Appendix 1). Red indicates stress; blue and purple indicate relaxed and a neutral mood, respectively. The predicted fixations on the visual interphase indicate possible moves or positions for the user to reach their goal.

#### **Algorithm 1.** Algorithm to detect affect state on visual interphase.

**3.4. Physiological measures**

28 Human-Robot Interaction - Theory and Application

indicated in **Figure 1**.

**3.5. Hypothesis generator**

in reaction to dynamic contents on the visual interphase.

detected fixations, and corresponding stress levels.

\_\_\_ *du*

**Figure 1.** Fixations and saccades of eye movement.

The physiological measures adopted for the experiment includes SCR, skin temperature (ST), and eye movement (saccade and fixations). These user attributes are used to measure changes

**1.** SCR/ST: The SCR measures the electrical changes of the skin; it provides a functional signal of emotional responses by measuring the electrodermal (EDA) changes of the skin, caused because of sweat [16]. The skin temperature (ST) changes according to blood circulation at the surface of the skin through body tissue. In a state of increased emotion, such as interest

**2.** Eye movement: This is the behavior of the eye during interaction; the eye-gaze pattern is a measure of behavior. The movement of a user's eyes is based on fixations (location of a user's eye gaze), saccades (rapid movements of the eye from one fixation to another) as

The concepts behind modeling physiological processes involve setting to get a significant accuracy and a prediction focus close to original data model; the model adopts the concepts for physical processes on dynamic systems [8], a least squares technique applied to system controls. For a user attribute saved from the sensors, the entire system is represented by the expression in Eqs. (1) and (2); the model fit to data represents prediction focus, 4 min from the

*y*(*m*) = *Gr*(*y*) + *Hr*(*y*) + *e* (2)

where *y*(*m*) is the response variables (stress levels) that determine coefficients of physiological reactions with computed variables *r*(*y*), *C*, *F*, *G*, and *H* are the estimated coefficients with noise

*dy* <sup>=</sup> *Cu*(*y*) <sup>+</sup> *Fr*(*y*) <sup>+</sup> *<sup>e</sup>* (1)

or stress, muscle fibers contract and cause a stenosis of the vasculature [17, 18].

```
1: procedure findpeaks
2: [peaks, locs] ← findpeaks(Response.eda,magenta'
                                              minpeakdistanc e′
                                                           ,15;)
3: m ← length(locs) ;
4: thresh ← 0.5∗
                    (mean(amplitude) − min(eda)) ;
5: top:
6: if mean(peaks) > = thresh < = baseline then return false
7: disp('stressed')
8: elseif mean(peaks) < = thresh > = baseline.
9:
10: disp('neutral')
11: else.
12: disp('relaxed')
13: end.
14: end.
```


indicating stress mood were detected. One of these is on a commercial widget at the right upper edge of the interphase. The neural point is detected inside the game interphase. The predicted and original (natural) response lying in the same cartesian coordinate of the Adera game interphase (**Figure 3**) shows possibility of a high performance of the control model. The stress and neutral mood are indicated of the three affect states generated. One interesting aspect is the stress point at the area where a question mark is located on the interphase just close to the position where we have a pointing black arrow. This icon (question mark) is there to provide suggestions on which direction/strategy to take in locating the missing person. The user was seen to be undecided whether to make a move on and make use of the lifeline

A Control System for Detecting Emotions on Visual Interphase Stimulus

http://dx.doi.org/10.5772/intechopen.75873

31

**Figure 3.** Detected affect state and correlating physiological reaction to Adera episode 1.

**Figure 2.** Control system feedback configuration.

#### **3.6. Operator querying**

The mechanism involved is a direct synchronization of the two-sensor port while there is a querying model for the system output. This helps to identify optimal responses in the physiological measure that correlates to visual attention on the part of the user. The next exposition discusses analysis and findings from the control system.
