**5. Functional complex of camera and operation assistant systems**

The robots that we developed do not only assume a straight motion but also roll around the pipe axis through the spherical wheels mounted on the head and tail. However, operators need to select the direction in which the robot orientation should be rotated from only camera images, which normally requires a practiced skill. In addition, this reduces difficulty in operating the robot as it can detect the orientation relative to the pathway direction of the bent pipe by itself.

Regardless of the design, in-pipe inspection robots need at least one illuminator and one camera to view its environment. For this matter, we proposed an anisotropic shadow-based operation assistant method using only a single LED and a camera (**Figure 12**) [26]. By displacing the position of the LED relative to that of the camera, a crescent-shaped shadow appears in the images captured in a bent pipe as illustrated in **Figure 13** [27, 28]. The size, position, and orientation of the shadow depend on the robot's orientation around the pipe axis, and it disappears in a certain robot's orientation (anisotropic shadow). Generally, as for shadow-based navigation systems, shadow disappearance should be avoided to prevent the robot from losing its way. As exclusion, AIRo-2.2 is designed so that it could adapt to a bent pipe without any control when the robot's orientation and the pathway direction of the bent pipe are aligned. By aligning those two specific orientations, the robot can select the optimal orientation to adapt to the bent sections.

Even though the shadow that appears in the bent pipe can be clearly extracted by binarization alone after the threshold is tuned, the shadow that appears in the straight pipe is also detected in straight sections. Therefore, the robot may mistakenly recognize that it is in a bent pipe even if it is in a straight pipe. If the straight pipe and the bent pipe can be distinguished beforehand, then the operation assistant system used to adjust the robot roll orientation can be executed only in bent sections.

**9**

**Figure 13.**

**Figure 12.**

high on the image histogram (**Figure 14a**).

*Principle of crescent shadow appearance in camera images.*

*Robotic Search and Rescue through In-Pipe Movement DOI: http://dx.doi.org/10.5772/intechopen.88414*

*AIRo-2.2 with a shadow-based operation assistant system [26].*

In our research, a monochrome image histogram (the relationship between the number of pixels and the brightness value of a camera image) is used to automatically distinguish a bending part and a straight pipe part (**Figure 14**). In straight pipes, the LED brightens the inner pipe wall around the robot. However, the light does not reach the far-off portion of the pipe (the center of the camera image), consequently leading to an even distribution in the luminance values from low to

In bent sections, as the LED light is brightly reflected in many areas (**Figure 14b**) in the camera images (the distance between the LED and the wall of the pipe is close), the luminance value is concentrated at the high brightness on the image histogram (**Figure 15**). This difference can be distinguished by the *Robotic Search and Rescue through In-Pipe Movement DOI: http://dx.doi.org/10.5772/intechopen.88414*

#### **Figure 12.**

*Unmanned Robotic Systems and Applications*

**5. Functional complex of camera and operation assistant systems**

*Multi-link-articulated wheeled-type in-pipe robots in the AIRo-series developed by the authors.*

orientation relative to the pathway direction of the bent pipe by itself.

optimal orientation to adapt to the bent sections.

The robots that we developed do not only assume a straight motion but also roll around the pipe axis through the spherical wheels mounted on the head and tail. However, operators need to select the direction in which the robot orientation should be rotated from only camera images, which normally requires a practiced skill. In addition, this reduces difficulty in operating the robot as it can detect the

Regardless of the design, in-pipe inspection robots need at least one illuminator and one camera to view its environment. For this matter, we proposed an anisotropic shadow-based operation assistant method using only a single LED and a camera (**Figure 12**) [26]. By displacing the position of the LED relative to that of the camera, a crescent-shaped shadow appears in the images captured in a bent pipe as illustrated in **Figure 13** [27, 28]. The size, position, and orientation of the shadow depend on the robot's orientation around the pipe axis, and it disappears in a certain robot's orientation (anisotropic shadow). Generally, as for shadow-based navigation systems, shadow disappearance should be avoided to prevent the robot from losing its way. As exclusion, AIRo-2.2 is designed so that it could adapt to a bent pipe without any control when the robot's orientation and the pathway direction of the bent pipe are aligned. By aligning those two specific orientations, the robot can select the

Even though the shadow that appears in the bent pipe can be clearly extracted by binarization alone after the threshold is tuned, the shadow that appears in the straight pipe is also detected in straight sections. Therefore, the robot may mistakenly recognize that it is in a bent pipe even if it is in a straight pipe. If the straight pipe and the bent pipe can be distinguished beforehand, then the operation assistant system used to adjust the robot roll orientation can be executed only

**8**

in bent sections.

**Figure 11.**

*AIRo-2.2 with a shadow-based operation assistant system [26].*

#### **Figure 13.**

*Principle of crescent shadow appearance in camera images.*

In our research, a monochrome image histogram (the relationship between the number of pixels and the brightness value of a camera image) is used to automatically distinguish a bending part and a straight pipe part (**Figure 14**). In straight pipes, the LED brightens the inner pipe wall around the robot. However, the light does not reach the far-off portion of the pipe (the center of the camera image), consequently leading to an even distribution in the luminance values from low to high on the image histogram (**Figure 14a**).

In bent sections, as the LED light is brightly reflected in many areas (**Figure 14b**) in the camera images (the distance between the LED and the wall of the pipe is close), the luminance value is concentrated at the high brightness on the image histogram (**Figure 15**). This difference can be distinguished by the

**Figure 14.** *Image histograms captured in a straight section (a) and a bent section (b).*

*Variance value depending on the distance between the robot and the entrance of the bent pipe.*

calculation of the variance of the image histogram. Apparently, the value of variance increases in bent pipes, and it decreases in straight pipes.

We set the threshold of variance to 1.4 million and found that the robot recognizes the bent pipe when it exceeded about D = 0.4 m. There is a portion where the variance value increases rapidly near D = 1.0 m in the graph mainly because of the influence of reflected light from the step part of the connecting pipe. Although the robot incorrectly recognizes it as a bent pipe in a straight section, traveling is not inhibited even if roll rotation is performed. At D = 0 m, the variance value decreases because the shadow image of the bent pipe is detected.

**11**

*Robotic Search and Rescue through In-Pipe Movement DOI: http://dx.doi.org/10.5772/intechopen.88414*

We confirmed from the experiments that the robot with the shadow-based operation assistant system could travel by approximately 7.5 m in length, including three vertical pipes and seven bent pipes. At this time, the operator used only one

Herein, we introduced our researched and developed in-pipe inspection robots and their shadow-based operation assistant system (orientation adjustment). The key point to designing such robot for various pipelines available is downsizing and simplification by the functional complex. Our approach for this functional complex included a differential mechanism, arrangement of multiple DoFs on a common axis, and usage of a camera not only for inspection but also for the operation assistant system. At the present stage, we are testing such approach in simulated pipelines installed in our laboratory. Nonetheless, we will continue to improve the development while collaborating with the user company and are planning to carry

This work was supported by JSPS KAKENHI Grant Number 17K14632, Gesuido Academic Incubation to Advanced Project (GAIA Project) of the Ministry of Land, Infrastructure, Transport and Tourism and many other cooperative companies as

commissioned research. We would like to thank all those who supported.

There is no conflict of interest regarding the publication of this article.

button to make the robot move forward or stop (**Figure 16**).

*Experimental result of the robot with the shadow-based operation assistant system.*

**6. Conclusions**

**Figure 16.**

out experiments on the actual site.

**Acknowledgements**

**Conflict of interest**

#### **Figure 16.**

*Unmanned Robotic Systems and Applications*

**10**

**Figure 14.**

**Figure 15.**

calculation of the variance of the image histogram. Apparently, the value of vari-

We set the threshold of variance to 1.4 million and found that the robot recognizes the bent pipe when it exceeded about D = 0.4 m. There is a portion where the variance value increases rapidly near D = 1.0 m in the graph mainly because of the influence of reflected light from the step part of the connecting pipe. Although the robot incorrectly recognizes it as a bent pipe in a straight section, traveling is not inhibited even if roll rotation is performed. At D = 0 m, the variance value decreases

ance increases in bent pipes, and it decreases in straight pipes.

*Variance value depending on the distance between the robot and the entrance of the bent pipe.*

*Image histograms captured in a straight section (a) and a bent section (b).*

because the shadow image of the bent pipe is detected.

*Experimental result of the robot with the shadow-based operation assistant system.*

We confirmed from the experiments that the robot with the shadow-based operation assistant system could travel by approximately 7.5 m in length, including three vertical pipes and seven bent pipes. At this time, the operator used only one button to make the robot move forward or stop (**Figure 16**).
