**Assistive Systems for the Visually Impaired Based on Image Processing** Assistive Systems for the Visually Impaired Based on

DOI: 10.5772/intechopen.70679

Hotaka Takizawa and Mayumi Aoyagi

Additional information is available at the end of the chapter Hotaka Takizawa and Mayumi Aoyagi Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.70679

#### Abstract

Image Processing

In this chapter, we proposed three assistive systems for visually impaired individuals based on image processing: Kinect cane system, Kinect goggle system, and light checking system. The Kinect cane system can detect obstacles of various sizes and also recognize objects such as seats. A visually impaired user is notified of the results of detection and recognition by means of vibration feedback. The Kinect goggle system is another type of wearable system, and can make user's hands free. The light checking system is implemented as an application for a smartphone, and can tell a visually impaired user the ON/OFF states of room lights and elevator button lights. The experimental results demonstrate that the proposed systems are effective in helping visually impaired individuals in everyday environments.

Keywords: assistive system, Kinect, cane, goggle, smartphone, camera, image processing, obstacle detection, object recognition, light checking

#### 1. Introduction

The world health organization estimated the number of visually impaired individuals to be approximately 285 million in 2014 [1]. Many of them use white canes to detect obstacles around them. However, the detectable ranges of white canes are very short. Guide dogs are also used to navigate visually impaired individuals to their destinations while avoiding obstacles. However, it is difficult to provide the sufficient number of guide dogs due to long-time periods and expenses to train them. In order to overcome these problems, extensive research has been dedicated to creating assistive systems for the visually impaired [2].

Obstacle detection is one of the representative research themes. Many research groups have proposed obstacle detection systems based on laser sensors [3–11], single charge-coupled

> © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

devices cameras [12–16], ultrasonic sensors [17–29], stereoscopic cameras [30–41], or RGB-D cameras [42–46]. These assistive systems are built on the basis of the concept of the electronic travel aid (ETA) [47, 48], which aims to assist visually impaired users in walking while avoiding obstacles. Therefore, these systems can notify the users about obstacles but cannot tell them what kind of objects they are.

2. Kinect cane system

supply, respectively.

2.1. Obstacle detection

Figure 2. Our Kinect cane system.

Figure 2 shows our Kinect cane system composed of a white cane and a backpack [68]. A Kinect sensor, a numeric keypad, and a tactile device are attached to the white cane. Kinect is an infrared-based range sensor for a consumer game machine, that is, Microsoft Xbox, and the white cane is also a commercial product for the visually impaired. (The sensor and the cane are approximately 300 and 100 USD, respectively.) The Kinect sensor is set at 75 cm from the floor. These devices are connected by wires with a portable personal computer and a UPS battery in the backpack. The computer and the UPS battery are used for device controlling and power

Assistive Systems for the Visually Impaired Based on Image Processing

http://dx.doi.org/10.5772/intechopen.70679

111

In this system, the X, Y, and Z axes of the world coordinate system are defined to be the horizontal vector oriented from left to right, the vertical vector oriented from top to bottom, and the horizontal vector extending from a Kinect sensor into the environment, respectively. The Kinect cane system can detect obstacles and recognize several objects, such as seats, by use

The Kinect cane system can detect obstacles which would prevent a visually impaired user to walk safely [45]. We provide two detection methods of small and large obstacles considering

of our special computer programs implemented for the following methods.

Here, let us consider a situation where there is a seat (bench) in front of a visually impaired user as shown in Figure 1, and the user wants to sit on the seat to take a rest. In this situation, the seat is not just any obstacle, but a useful equipment. If the user uses one of the obstacle detection systems mentioned earlier (see Figure 1(a)), he or she is required to confirm the obstacle by himself or herself. However, if the user uses an object recognition system, which can determine the object to be a seat (Figure 1(b)), the user can obtain a benefit. It is necessary to build an assistive system to recognize objects around a visually impaired user.

Several research groups have proposed object recognition systems. Drug packages [49], classroom doors [50–53], podiums [50], and pathways [54, 55] are recognized by using barcodes [49, 56], augmented reality markers [50, 52–54], radio frequency identification tags [23, 51, 57, 58], Bluetooth devices [59], wireless network devices [55], or visible light communication devices [60, 61]. These physical devices are useful, but it is difficult to deploy such devices in everyday environment.

Other research groups have also proposed assistive systems to notify visually impaired users about tables [62], color blocks [63], and staircases [64–67] by means of laser range sensors [65– 67] and Kinect sensors [62, 63]. These systems are useful, but are not sufficient yet. Other types of object should be recognized to help visually impaired individuals more.

This chapter proposes our assistive systems not only to detect obstacles of various sizes but also to recognize objects of various types by use of image processing technique.

Figure 1. Obstacle detection system (a) versus object recognition system for the visually impaired (b).
