**3. Signal properties**

### **3.1 Blinking**

4 Will-be-set-by-IN-TECH

33 34 35 36 37 38 39 40

33 34 35 36 37 38 39 40 **time [s]**

The reliable measurements should be obtained using the flat band–pass filters (e.g. the Buttherworth filter), but the phase distortions are introduced by the analog filters. The combined filtering of a signal using the analog and digital filters and higher sampling rates is

The number of quantization levels (the number of bits per sample) for EOG should be carefully set depending on the DC level processing. Even a 8–bit per sample is enough for not demanding applications with correct DC level maintenance and 50/60 Hz interference suppression before sampling. The higher resolution of quantization is used (e.g. 12–16 bits per sample) if both mentioned components are hard to control by electronics. The higher resolution of quantization allows the signal processing using the digital signal processing

The higher sampling rate and better quantization create the possibilities of precise observation

The calibration of the system is necessary. A few extreme orientations of eyes are used and the intermediate orientations are interpolated. The HCI system requires the calibration before the beginning of measurements. The non real–time applications support the additional calibration at the end and intermediate calibration if they are necessary. More than single calibration

High quality measurements are recommended, but the signal processing of the obtained biosignal is necessary for the separation of the EOG and blinking signals. The estimation

Fig. 3. Example of the EOG signal – two channels for the 3/4 configuration

of the EOG signal, which is important especially for the medical purposes.

allow the correction of the measurements and improve the acquisition results.

of parameters for both signals is necessary.

necessary.

algorithms.

The EOG signal has one important artifact and it is the blinking signal. The blinking occurs if the eyelid makes movements (vertical one). The disturbance depends on the electrode configuration and it is additive for the 3/4 configuration.

Fig. 4. Example of blinking pulses

The blinking pulse is similar to the Gaussian pulse for typical blinking (Fig. 4).

There are also atypical blinking cases when the eyelid moves very slowly or if the eyelid closing time is different than the eyelid opening time. There are much more cases when blinking is non–Gaussian, but they are not considered in typical systems. In this chapter typical blinking is assumed.

### **3.2 Saccades**

The saccade is the rapid change (Fig. 5) of eye orientation [Becker (1989); Gu et al. (2008); Mosimann et al. (2005)]. The rapid changes require the acquisition of high frequency components of a signal. This is one of the reasons why the necessary sampling rate is higher in comparison to the other systems. The low–pass filtering for the removal of the 50/60 Hz component using the cut of frequency about 30–40 Hz disturbs a saccade signal.

There are also interesting cases, for instance, when the blinking is near to the saccades. This situation is not rare in real measurements, but it is very often not considered by researchers.

### **3.3 Smooth pursuits**

During the tracking of a slowly moving object, the eyes move smoothly (Fig. 6). Such movements are named as smooth pursuits.

There are also other features of the EOG measurements, like the microsaccades that are rapid movements of eyes but in the smaller scale.

blink removal is not possible: a typical case if a blink is after a falling saccade step or before a

Real–Time Low–Latency Estimation of the Blinking and EOG Signals 319

Such techniques are useful for real–time systems. The latency is higher than the maximal blinking pulse width. The rising edge of blinking pulse is simple to detect, but without the falling edge of blinking it is not possible to recognize it correctly. The rising edge for saccadic movement is similar. The availability of both edges of blinking is necessary for correct detection. The median filter calculates a median value from available values and is used for the estimation of the EOG signal level. The expected value is obtained if more than 50% values are assigned to the signal part without blinking. The median filter size (a median

The implementation of the median filter is possible using more efficient way due to the

The estimation of signal parameters using the synthesis technique is possible (Fig. 8). Such a technique is based on fitting a signal generator model to a signal using the optimization techniques [Krupi ´nski & Mazurek (2010b;d;e; 2011)]. The additional constraints are added for improving results and for the reduction of computation time. A correct model related to the

The EOG signal and blinking one are well defined in time domain, so synthesis is performed by the comparison of the part (*S*) of selected signal (*s*) and synthesized one (*m*). The aim of optimization process is to reduce an error value using, for example, the following fitting

The estimated parameters are related to the selected part of a signal and the number of them

The advantage of this technique is the parallel signal separation, estimation, filtering and detection. The signals are separated using the model of one of them or both of them. The estimated signals are properly obtained if the model of both signals (EOG and blinking) is applied. The artifacts that are not assigned to EOG or blinking are considered as noise, but depend on the model quality and may be still interesting, e.g. for further microsaccadic movement analysis. The models are based on discreet events (blinking pulses, saccades) and additional pattern recognition techniques are not necessary. The obtained results are based on

*i*∈*S*

moving window requirements and full sorting is not necessary [Arce (2005)].

**EOG signal**

**blinking signal**

(*si* <sup>−</sup> *mi* (*signal parameters*))<sup>2</sup> . (1)

**Blink filtering algorithm**

saccade step.

**input signal**

Fig. 7. Typical separation technique

moving window) determines the latency.

**4.2 Analysis by synthesis technique**

specific domain of synthesis is necessary.

depend on the number of detected features.

*E* (*signal parameters*) = ∑

criteria:

Fig. 5. Example of saccades

Fig. 6. Example of smooth pursuit with a two saccades

#### **4. Signal separation techniques**

#### **4.1 Filtering techniques**

There are many techniques for the separation of the EOG and blinking signals. The main techniques are based on the filtering (linear or non–linear) of blinking pulses. The signal with removed blinking pulses is the EOG one. The subtraction of EOG from the original signal gives a blinking signal. Such operation is applied independently to both channels in the 3/4 system. The pattern recognition techniques are used for the estimation of the position of the saccades (e.g. using differentiation and thresholding). The blinking pulses are detected using thresholding. The independent processing of both channels for blinking signal is necessary if the asymmetric blinking is possible. The typical blinking is related to both eyelids together, but the single eyelid blinking is possible too.

The typical filter used for the separation (Fig. 7) is a median filter and derivative filters that are used for the removal of the pulses [Bankman & Thakor (1990); Juhola (1991); Krupi ´nski (2010); Krupi ´nski & Mazurek (2010a); Martinez et al. (2008); Niemenlehto (2009)]. The filtering based on a median filter is the simplest technique but not reliable. There are many cases when the blink removal is not possible: a typical case if a blink is after a falling saccade step or before a saccade step.

Fig. 7. Typical separation technique

6 Will-be-set-by-IN-TECH

21 22 23 24 25 26 **time [s]**

> 9 9.5 10 10.5 **time [s]**

There are many techniques for the separation of the EOG and blinking signals. The main techniques are based on the filtering (linear or non–linear) of blinking pulses. The signal with removed blinking pulses is the EOG one. The subtraction of EOG from the original signal gives a blinking signal. Such operation is applied independently to both channels in the 3/4 system. The pattern recognition techniques are used for the estimation of the position of the saccades (e.g. using differentiation and thresholding). The blinking pulses are detected using thresholding. The independent processing of both channels for blinking signal is necessary if the asymmetric blinking is possible. The typical blinking is related to both eyelids together,

The typical filter used for the separation (Fig. 7) is a median filter and derivative filters that are used for the removal of the pulses [Bankman & Thakor (1990); Juhola (1991); Krupi ´nski (2010); Krupi ´nski & Mazurek (2010a); Martinez et al. (2008); Niemenlehto (2009)]. The filtering based on a median filter is the simplest technique but not reliable. There are many cases when the

Fig. 6. Example of smooth pursuit with a two saccades

**4. Signal separation techniques**

but the single eyelid blinking is possible too.

**4.1 Filtering techniques**

Fig. 5. Example of saccades

Such techniques are useful for real–time systems. The latency is higher than the maximal blinking pulse width. The rising edge of blinking pulse is simple to detect, but without the falling edge of blinking it is not possible to recognize it correctly. The rising edge for saccadic movement is similar. The availability of both edges of blinking is necessary for correct detection. The median filter calculates a median value from available values and is used for the estimation of the EOG signal level. The expected value is obtained if more than 50% values are assigned to the signal part without blinking. The median filter size (a median moving window) determines the latency.

The implementation of the median filter is possible using more efficient way due to the moving window requirements and full sorting is not necessary [Arce (2005)].

#### **4.2 Analysis by synthesis technique**

The estimation of signal parameters using the synthesis technique is possible (Fig. 8). Such a technique is based on fitting a signal generator model to a signal using the optimization techniques [Krupi ´nski & Mazurek (2010b;d;e; 2011)]. The additional constraints are added for improving results and for the reduction of computation time. A correct model related to the specific domain of synthesis is necessary.

The EOG signal and blinking one are well defined in time domain, so synthesis is performed by the comparison of the part (*S*) of selected signal (*s*) and synthesized one (*m*). The aim of optimization process is to reduce an error value using, for example, the following fitting criteria:

$$E\left(\text{signal parameters}\right) = \sum\_{i \in S} \left(\mathbf{s}\_i - \boldsymbol{m}\_i \left(\text{signal parameters}\right)\right)^2. \tag{1}$$

The estimated parameters are related to the selected part of a signal and the number of them depend on the number of detected features.

The advantage of this technique is the parallel signal separation, estimation, filtering and detection. The signals are separated using the model of one of them or both of them. The estimated signals are properly obtained if the model of both signals (EOG and blinking) is applied. The artifacts that are not assigned to EOG or blinking are considered as noise, but depend on the model quality and may be still interesting, e.g. for further microsaccadic movement analysis. The models are based on discreet events (blinking pulses, saccades) and additional pattern recognition techniques are not necessary. The obtained results are based on

there is a lack of the analysis of the more complex scenarios, like a saccade near to a blink,

Real–Time Low–Latency Estimation of the Blinking and EOG Signals 321

The wavelets are useful for the processing of signals and depend on the applied wavelet so the selected properties of a signal are emphasized [Augustyniak (2003); Mallat (1999); Mallat & Zhang (1993)]. The selection of a particular wavelet defines the specific response

The non–isolated singularities need the multifractal analysis. The signals with singularities are analyzed using the singularity spectrum. The EOG signal with blinking is such a kind of signal that has the isolated singularities for most cases. The distance between events of any type is quite large, but both kinds of events may appear in short time and in such a case the limited multifractal properties exist. The analysis of the singularities is the basis of the detection and gives the possibility for real–time processing without median filtering (Fig. 9). The singularities create the large amplitude values in their cone of influence what is observed in a singularity spectrum. The analysis of singularity spectrum is possible using the detection of local maximum for every scale. The maximal values of the wavelet transform coefficients |*W f*(*u*,*s*)| are obtained by differentiation and testing the values. The zero value is obtained if

*∂*|*W f*(*u*0,*s*0)|

appear for the constant value of |*W f*(*u*,*s*)| for some cases.

The following CWT formula is used for the computations:

*C* (*a*, *b*; *f* (*t*), *ψ* (*t*)) =

**transfrom ABS Maximum**

The additional conditions are necessary for the removal of non–strict maximum points that

The detected maximum points are connected on the every scale. The parameters of a line: a length, an accumulated value over a line, and a slope are used for the detection of the even

The singularity analysis needs the tracking of the lines starting from the small scale to the largest one. The line length depends on the fitting of the wavelets to the singularity (event). A small length corresponds to the less important feature. The longest lines are taken into account. The length of a line is not only one method for the detection of features. The accumulation of the values along trajectories of accumulated singularity spectra is a technique used in this work. The accumulated value should be higher than a predefined threshold and this value is set by the previous observation of signal behavior. The wavelet shape 'gaus2' (Fig. 10) is applied for the signal processing by the continuous wavelet transform (CWT).

> ∞ −∞

where ∗ denotes the complex conjugate, *a* is a scale parameter, *b* is a position, and *ψ* is the

*<sup>f</sup>* (*t*) <sup>1</sup> √*a ψ*∗  *<sup>t</sup>* <sup>−</sup> *<sup>b</sup> a*

*dt* (3)

**detector**

*<sup>∂</sup><sup>u</sup>* <sup>=</sup> <sup>0</sup> (2)

**Line tracking** **detected events**

what appears in real measurements.

for singularities too.

a maximum point is found.

**signal Wavelets**

selected wavelet.

**input**

type and the estimation of parameters.

Fig. 9. Singularity analysis scheme

Fig. 8. Analysis by synthesis technique

the set of discreet events and the corresponding values like the height of a blink and an EOG level value.

The disadvantage of this technique is computation power requirements [Krupi ´nski & Mazurek (2010c)]. Real–time processing is very difficult and additional latency occurs. Some applications does not need the detection of saccades and the EOG signal is a signal that is used directly (e.g. in the motion capture applications or the analysis of point–of–interest).

In [Krupi ´nski & Mazurek (2011)] such a technique for the EOG and blinking signals is introduced. This algorithm uses evolutionary search with the mutation of a single child [Michalewicz (1996)]. The additional gradient optimization is used for the computation time reduction by the local improving of convergence for a blink position and height, a saccade position and the value of the EOG signal level between two saccades.
