**3. Eye-gaze detection by image analysis**

Eye gaze is defined as a unit vector in a three-dimensional coordinate space. The origin of this unit vector is the center of the eyeball. Generally, the user's gaze is detected on a two-dimen‐ sional plane. It has horizontal and vertical components. The method of tracking the iris (the colored part of the eye) is the most popular method for eye-gaze detection using image analysis in natural light [4-6]. For example, if the edge between the iris and the sclera (the white part of the eye) is estimated by image processing, the appropriately approximated ellipse of the edge shows the location of the iris. However, it is difficult to distinguish the iris and the sclera by image analysis, because the edge between the iris and the sclera is not sharp. In addition, if a large part of the iris is hidden by the upper and lower eyelids, the measurement errors increase, because the obscuring of the iris by the eyelids causes estimation errors in the delineation of the iris. To resolve these issues, we propose a new image analysis method for detecting eye gaze using both the horizontal and vertical directions. This detection method is based on the limbus tracking method. Our eye-gaze detection method can obtain the coordi‐ nates of the user's gaze point.

In our eye-gaze detection method, the video camera records images of the user's eye from a distant location (the distance between the user and camera is approximately 70 cm), and then this image is enlarged. The head movements of the user induce a large error in the detected gaze. We compensated for the head movements by tracing the location of a corner within the eye.

#### **3.1. Horizontal gaze detection**

The limbus tracking method is an eye-gaze detection method using the difference in reflectance between the iris and the sclera. By this method, eye gaze can be estimated with relative ease, and therefore it has been used since the 1960s [12]. The general eye-gaze detection system using the limbus tracking method irradiates an eyeball of a subject with infrared light. The eye gaze of the subject is detected by measuring the reflected light using optical sensors such as photodiodes.

We have developed a new eye-gaze detection method that is used under natural light [8,9]. An overview of the proposed horizontal gaze detection method is shown in Figure 2. The difference in reflectance between the iris and the sclera is used as follows: the gaze is estimated from the difference between the integral values of the light intensities in Areas A and B, as shown in Figure 2. We designate this differential value as the horizontal eye-gaze value, which gives a value for the horizontal gaze component. The relation between the horizontal eye-gaze value and the angle of sight is nearly proportional. Therefore, the system can be calibrated using this relation.

dimensional projection) are calculated from these eye images as reference data. The user's vertical gaze can be detected by pattern matching based on these reference data. An overview of the method is shown in Figure 4, which illustrates the detection of each of the three gaze directions: top, center, and bottom. The wave patterns at the right of the eye illustrations show the light-intensity distributions. We confirmed that with increasing reference data the method

Eye-Gaze Input System Suitable for Use under Natural Light and Its Applications Toward a Support for ALS Patients

Distribution of light intensity

http://dx.doi.org/10.5772/56560

247

Center Top Bottom

We developed a new eye-gaze input system using the methods discussed in Section 3. This system detects the eye gaze of a user under natural light and operates the application programs for communication aids such as text input. Two interfaces to operate the application programs have been developed. One of the interfaces has indicators displayed on the PC monitor. The functions of application programs are executed by gazing at these indicators. The other interface allows eye gaze to control the mouse cursor. By means of this interface, a user can operate the general Windows software. We describe our eye-gaze input system and its input

Our eye-gaze input system comprises a PC, a home video camera, and an IEEE 1394 interface for image capture from the camera. For eye-gaze detection, the computer runs image analysis software on Windows (XP, Vista, or 7). This system (illustrated in Figure 5) does not require a device exclusively for image processing. The characteristics of eye gaze vary from one individual to another. Therefore, the eye-gaze input system requires calibration. The indicators for calibration are shown in Figure 6. Users must calibrate the system before using it for tasks. After the camera location is adjusted, the calibration begins. While the calibration is being performed, users gaze at each indicator when its color switches to red. Our eye-gaze input system has two types of indicators, which are specific to each application. In particular, the five calibration indicators shown in Figure 6(a) are used for the input interface with a work‐ space displayed at the center of the PC monitor. The workspace is used for displaying an application software window. In addition, the nine calibration indicators shown in Figure

can distinguish five to seven vertical gaze directions.

**Figure 4.** Detection of Vertical gaze (Method 2)

interface below.

**4.1. Eye-gaze input system**

**4. Input interfaces based on eye gaze**

**Figure 2.** Detection of horizontal gaze

#### **3.2. Vertical gaze detection**

An overview of the proposed vertical gaze detection method is shown in Figure 3. The vertical eye-gaze is also detected by the limbus tracking method. In other words, the light intensity of the eye image is used to detect the vertical eye gaze. Specifically, the vertical eye gaze is estimated from the integral value of the light intensity in Area C that is not hidden by the eyelids [10]. We designate this integral value as the vertical eye-gaze value, which gives a value for the vertical gaze component. The relation between the vertical eye-gaze value and the angle of sight is a characteristic function. Therefore, the system can be calibrated using this relation. Many application programs for eye-gaze input need low-accuracy measurements that involve only three directions of vertical eye gaze (top, center, and bottom). Therefore, our practical eye-gaze input system detects only three general directions of vertical eye gaze.

**Figure 3.** Detection of Vertical gaze (Method 1)

In reality, the light-intensity distribution of the eye image changes with iris movement, and vertical gaze can be detected using this change [9]. The system stores vertically aligned images of the eye gazing at the indicators. The light-intensity distributions (the results of a onedimensional projection) are calculated from these eye images as reference data. The user's vertical gaze can be detected by pattern matching based on these reference data. An overview of the method is shown in Figure 4, which illustrates the detection of each of the three gaze directions: top, center, and bottom. The wave patterns at the right of the eye illustrations show the light-intensity distributions. We confirmed that with increasing reference data the method can distinguish five to seven vertical gaze directions.

**Figure 4.** Detection of Vertical gaze (Method 2)

We have developed a new eye-gaze detection method that is used under natural light [8,9]. An overview of the proposed horizontal gaze detection method is shown in Figure 2. The difference in reflectance between the iris and the sclera is used as follows: the gaze is estimated from the difference between the integral values of the light intensities in Areas A and B, as shown in Figure 2. We designate this differential value as the horizontal eye-gaze value, which gives a value for the horizontal gaze component. The relation between the horizontal eye-gaze value and the angle of sight is nearly proportional. Therefore, the system can be calibrated

Area B

Area C

An overview of the proposed vertical gaze detection method is shown in Figure 3. The vertical eye-gaze is also detected by the limbus tracking method. In other words, the light intensity of the eye image is used to detect the vertical eye gaze. Specifically, the vertical eye gaze is estimated from the integral value of the light intensity in Area C that is not hidden by the eyelids [10]. We designate this integral value as the vertical eye-gaze value, which gives a value for the vertical gaze component. The relation between the vertical eye-gaze value and the angle of sight is a characteristic function. Therefore, the system can be calibrated using this relation. Many application programs for eye-gaze input need low-accuracy measurements that involve only three directions of vertical eye gaze (top, center, and bottom). Therefore, our practical

In reality, the light-intensity distribution of the eye image changes with iris movement, and vertical gaze can be detected using this change [9]. The system stores vertically aligned images of the eye gazing at the indicators. The light-intensity distributions (the results of a one-

eye-gaze input system detects only three general directions of vertical eye gaze.

Center of eye

Area A

using this relation.

246 Current Advances in Amyotrophic Lateral Sclerosis

**Figure 2.** Detection of horizontal gaze

**3.2. Vertical gaze detection**

**Figure 3.** Detection of Vertical gaze (Method 1)
