**5. Waveletes separation technique**

### **5.1 Singularity processing**

Another method of the event detection (a saccade, a blink) is the wavelets transform [Ghandeharion & Ahmadi–Noubari (2009); Reddy et al. (2010)]. The saccade and blink are well defined in time domain and they have a limited length in time.

In [Bukhari et al. (2010a;b; 2011)] the signal analysis and filtering of eye movements using scalogram and 'db4' wavelets up to details level 10 are considered. In [Reddy et al. (2010)] a few wavelets are compared ('sym8','haar','db4','db10','coif3') for the blink detection, and the best is 'sym8'. The threshold based technique is applied for the blink detection. In [Bhandari et al. (2006)] the signal enhancement techniques using wavelets are proposed. The 'coiflet3' wavelet for denoising is applied. Blinks and saccades are enhanced using the 'haar' wavelet. Discussed wavelets techniques by the authors are not sufficient for HCI systems. Moreover, 8 Will-be-set-by-IN-TECH

**signal EOG**

**EOG signal synthesis**

**parameters**

**blinking signal synthesis**

**Optimization algorithm**

the set of discreet events and the corresponding values like the height of a blink and an EOG

The disadvantage of this technique is computation power requirements [Krupi ´nski & Mazurek (2010c)]. Real–time processing is very difficult and additional latency occurs. Some applications does not need the detection of saccades and the EOG signal is a signal that is used

In [Krupi ´nski & Mazurek (2011)] such a technique for the EOG and blinking signals is introduced. This algorithm uses evolutionary search with the mutation of a single child [Michalewicz (1996)]. The additional gradient optimization is used for the computation time reduction by the local improving of convergence for a blink position and height, a saccade

Another method of the event detection (a saccade, a blink) is the wavelets transform [Ghandeharion & Ahmadi–Noubari (2009); Reddy et al. (2010)]. The saccade and blink are

In [Bukhari et al. (2010a;b; 2011)] the signal analysis and filtering of eye movements using scalogram and 'db4' wavelets up to details level 10 are considered. In [Reddy et al. (2010)] a few wavelets are compared ('sym8','haar','db4','db10','coif3') for the blink detection, and the best is 'sym8'. The threshold based technique is applied for the blink detection. In [Bhandari et al. (2006)] the signal enhancement techniques using wavelets are proposed. The 'coiflet3' wavelet for denoising is applied. Blinks and saccades are enhanced using the 'haar' wavelet. Discussed wavelets techniques by the authors are not sufficient for HCI systems. Moreover,

directly (e.g. in the motion capture applications or the analysis of point–of–interest).

position and the value of the EOG signal level between two saccades.

well defined in time domain and they have a limited length in time.

**error**

**signal**

**blinking signal**

**input**

Fig. 8. Analysis by synthesis technique

**5. Waveletes separation technique**

**5.1 Singularity processing**

level value.

there is a lack of the analysis of the more complex scenarios, like a saccade near to a blink, what appears in real measurements.

The wavelets are useful for the processing of signals and depend on the applied wavelet so the selected properties of a signal are emphasized [Augustyniak (2003); Mallat (1999); Mallat & Zhang (1993)]. The selection of a particular wavelet defines the specific response for singularities too.

The non–isolated singularities need the multifractal analysis. The signals with singularities are analyzed using the singularity spectrum. The EOG signal with blinking is such a kind of signal that has the isolated singularities for most cases. The distance between events of any type is quite large, but both kinds of events may appear in short time and in such a case the limited multifractal properties exist. The analysis of the singularities is the basis of the detection and gives the possibility for real–time processing without median filtering (Fig. 9).

The singularities create the large amplitude values in their cone of influence what is observed in a singularity spectrum. The analysis of singularity spectrum is possible using the detection of local maximum for every scale. The maximal values of the wavelet transform coefficients |*W f*(*u*,*s*)| are obtained by differentiation and testing the values. The zero value is obtained if a maximum point is found.

$$\frac{\partial |\mathcal{W}f(u\_0, s\_0)|}{\partial u} = 0 \tag{2}$$

The additional conditions are necessary for the removal of non–strict maximum points that appear for the constant value of |*W f*(*u*,*s*)| for some cases.

The detected maximum points are connected on the every scale. The parameters of a line: a length, an accumulated value over a line, and a slope are used for the detection of the even type and the estimation of parameters.

Fig. 9. Singularity analysis scheme

The singularity analysis needs the tracking of the lines starting from the small scale to the largest one. The line length depends on the fitting of the wavelets to the singularity (event). A small length corresponds to the less important feature. The longest lines are taken into account. The length of a line is not only one method for the detection of features. The accumulation of the values along trajectories of accumulated singularity spectra is a technique used in this work. The accumulated value should be higher than a predefined threshold and this value is set by the previous observation of signal behavior. The wavelet shape 'gaus2' (Fig. 10) is applied for the signal processing by the continuous wavelet transform (CWT).

The following CWT formula is used for the computations:

$$\mathcal{C}\left(a, b; f\left(t\right), \psi\left(t\right)\right) = \int\_{-\infty}^{\infty} f\left(t\right) \frac{1}{\sqrt{a}} \psi^\*\left(\frac{t-b}{a}\right) dt\tag{3}$$

where ∗ denotes the complex conjugate, *a* is a scale parameter, *b* is a position, and *ψ* is the selected wavelet.

<sup>0</sup> <sup>100</sup> <sup>200</sup> <sup>300</sup> <sup>400</sup> <sup>500</sup> <sup>600</sup> −0.4

(a) Original signal

<sup>0</sup> <sup>100</sup> <sup>200</sup> <sup>300</sup> <sup>400</sup> <sup>500</sup> <sup>600</sup> −50

(d) Accumulated peaks of

**Time [n]**

**Time [n]**

(g) Accumulated

100 200 300 400 500 600

Fig. 11. Example 1 – two blinks and a single saccade

**6.1 Monte Carlo approach and a signal generator**

**Time [n]**

**Scale**

**Scale**

**6. Performance analysis of wavelets separation technique**

**Counted Slopes**

transform

**Time [n]**

Real–Time Low–Latency Estimation of the Blinking and EOG Signals 323

**Time [n]**

(e) Lines

<sup>0</sup> <sup>100</sup> <sup>200</sup> <sup>300</sup> <sup>400</sup> <sup>500</sup> <sup>600</sup> −6

**Time [n]**

(h) Slope Counter

The Monte Carlo approach [Metropolis & Ulam (1949)] is applied for the tests of wavelets performance depending on a few conditions. The testing of the algorithms using synthetic EOG and blinking signals generator is necessary. Such approach is very good for the testing of algorithm. The tests based on the analysis of the recorded signal are limited by the number of available samples. The representative set of the real samples is necessary with the man–made description of every example. The synthetic technique needs a good generator, but the tests are much more reliable. The samples obtained by the real measurement process are related to the small set of humans. The EOG and blinking signals generator is described in [Krupi ´nski & Mazurek (2010a)] and used in the papers [Krupi ´nski & Mazurek (2010b;d;e; 2011)] with some additional extensions (the smooth pursuit support). There are two possible techniques

100 200 300 400 500 600

(b) Continuous wavelets

100 200 300 400 500 600

**Scale**

−1.5 −1 −0.5 0 0.5 1 1.5 2

transform

**Accumulated Peaks**

**Time [n]**

<sup>0</sup> <sup>100</sup> <sup>200</sup> <sup>300</sup> <sup>400</sup> <sup>500</sup> <sup>600</sup> <sup>0</sup>

**Time [n]**

(f) Accumulated lines of

<sup>0</sup> <sup>100</sup> <sup>200</sup> <sup>300</sup> <sup>400</sup> <sup>500</sup> <sup>600</sup> −2

**Time [n]** (i) Detected signals

(c) Peaks

100 200 300 400 500 600

−0.2 0 0.2 0.4 0.6 0.8 1

transform

**Summed along Scale**

**Scale**

**Amplitude**

Fig. 10. Wavelets shape – 'gaus2'

#### **5.2 Example 1 – two blinks and a single saccade**

In this example the results of CWT for two blinks and a single saccade are shown.

First blink is slow and second one is fast (Fig. 11a). For the blinks and saccade positions the cones are visible and start from the smallest scale and they extend towards the larger scale (Fig. 11b). The local peaks are detected and multiplied by the wavelets coefficients and the results are shown in Fig. 11c. The accumulated values from the previous step are depicted in (Fig. 11d). The obtained positions are used as the starting positions for the tracking algorithm. The track–before–detect approach (TBD) is used and the simplified spatio–temporal track–before–detect algorithms are used with recurrent processing [Mazurek (2009; 2010a;b;c; 2011a;b)]. There are three motion vectors used due to the high resolution of the scale. There is not need for searching more motion vectors. The result of tracking is shown in Fig. 11e. The tracking of values along lines is important and the accumulated value corresponds to the strength of the event (a blinking height and saccade differential height). The motion trajectory vectors are accumulated too (Fig. 11f). The motion vector near zero corresponds to the blink signal, because a wavelet function is similar. The slope is oriented to the left or right direction depending on the saccade falling or rising edge. The slope counter measures the motion vector (Fig. 11h) and is used together with accumulated values (Fig. 11g) for the threshold–based detection of the event. The detection of the event is shown in (Fig. 11i) where a peak and the peak marks correspond to the event type.

#### **5.3 Example 2 – two blinks and two saccades and smooth pursuit**

The next example shows the influence of the smooth pursuit on the processing of a signal. The smooth pursuit is very low frequency component of a signal and a very large scale should be considered by the CWT algorithm. The limited scale range allows reducing the influence of the trend from smooth pursuit. The filtering behavior (Fig. 12) is similar to Example 1.

The verification of the algorithm needs the computation of many examples using the Monte Carlo approach.

10 Will-be-set-by-IN-TECH

−5 0 5

(a)

First blink is slow and second one is fast (Fig. 11a). For the blinks and saccade positions the cones are visible and start from the smallest scale and they extend towards the larger scale (Fig. 11b). The local peaks are detected and multiplied by the wavelets coefficients and the results are shown in Fig. 11c. The accumulated values from the previous step are depicted in (Fig. 11d). The obtained positions are used as the starting positions for the tracking algorithm. The track–before–detect approach (TBD) is used and the simplified spatio–temporal track–before–detect algorithms are used with recurrent processing [Mazurek (2009; 2010a;b;c; 2011a;b)]. There are three motion vectors used due to the high resolution of the scale. There is not need for searching more motion vectors. The result of tracking is shown in Fig. 11e. The tracking of values along lines is important and the accumulated value corresponds to the strength of the event (a blinking height and saccade differential height). The motion trajectory vectors are accumulated too (Fig. 11f). The motion vector near zero corresponds to the blink signal, because a wavelet function is similar. The slope is oriented to the left or right direction depending on the saccade falling or rising edge. The slope counter measures the motion vector (Fig. 11h) and is used together with accumulated values (Fig. 11g) for the threshold–based detection of the event. The detection of the event is shown in (Fig. 11i)

The next example shows the influence of the smooth pursuit on the processing of a signal. The smooth pursuit is very low frequency component of a signal and a very large scale should be considered by the CWT algorithm. The limited scale range allows reducing the influence of the trend from smooth pursuit. The filtering behavior (Fig. 12) is similar to Example 1.

The verification of the algorithm needs the computation of many examples using the Monte

In this example the results of CWT for two blinks and a single saccade are shown.

−0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 1.2

**5.2 Example 1 – two blinks and a single saccade**

where a peak and the peak marks correspond to the event type.

**5.3 Example 2 – two blinks and two saccades and smooth pursuit**

Fig. 10. Wavelets shape – 'gaus2'

Carlo approach.

Fig. 11. Example 1 – two blinks and a single saccade
