**4. Pothole metrology data parametrization**

**Figure 4** illustrates the conceptual approximation of a pothole with dimensional parameters that define the pothole metrology as: width, depth, surface area and volume. Assuming the potholes have the shape of a circular paraboloid, then in 2D they can be represented by the function *f x*ð Þ¼ *; <sup>y</sup> <sup>x</sup>*<sup>2</sup> <sup>þ</sup> *<sup>y</sup>*2.

#### **4.1 Pothole depth determination using depth image**

The depth-image plane (**Figure 4**) is one of the noise factors, whereby the plane is not necessarily parallel to the pavement surface. The noise points, which are the non-defect points between the pavement-pothole plane and the camera, have to be filtered out for the accurate depth detection and the subsequent 2D-pothole detection from the depth image. The general principle of removing the outlier points (noise), is by determining the local minimum of each column and then subtracting from the column itself in order to extract the pothole from the rest of data [47]. The minimum of each column defines the depth below which the pothole starts on the road pavement surface, and is referred to as the depth-image plane. Using this approach, the depths *di* including the maximum depth *di*max can be quantified, and the mean depth *di* for a given pothole is also computed.

**Test**

**158**

**Pavement RGB**

**2-class**

*k***-means** 

**adaptive** 

**median-filtering**

**classification**

 **and**

**Fitted edge on candidate**

**Ellipse-fitting**

 **in PSE for pothole**

**Horizontal**

 **and vertical integral**

**projection (HVIP) plots**

*Geographic Information Systems in Geospatial Intelligence*

**presence detection**

**pothole image**

**data**

#1

#2

#3

#4

**Table 3.**

*PSE and HV-integral*

 *projection search for pothole and non-pothole*

 *frames from RGB test data.*

**image frame**

### **4.2 Pothole width measurement**

The width of a pothole can be defined by the semi-major *a* and semi-minor *b* axes, on the assumption that an ellipse, based on the major path elliptic regression, is used pothole shape extraction [48]. To determine the lateral width of the pothole, it can be estimated using a circular paraboloid, which is an elliptical paraboloid. And, an elliptical paraboloid is a surface with parabolic cross-sections in 2-orthogonal directions and 1-elliptical cross-section in the other orthogonal direction. Using an edge detection algorithm, the near-true shape of the pothole is first derived using the proposed SFCM, and then an elliptical fit is used to approximate the shape, from which the axes are defined for the calculation of the surface area and volume of the pothole.

#### **4.3 Pothole surface area determination**

In order to determine the surface area of the pothole, the optimally detected edge is used to fit the shape of the pothole as either elliptic paraboloid or circular paraboloid. While the former is defined by the dimensions of semi-major axis *a* and semi-minor axis *b*, the latter is defined by the estimated radius *r*. The surface area is then computed by using the surface integrals of either of the paraboloids [49], as respectively shown in Eqs. (17) and (18) for the elliptic and circular paraboloids.

$$A = \pi ab\tag{17}$$

area within the frame [51]. Therefore the estimated volume *Vd* in terms of the pixel

*On the Use of Low-Cost RGB-D Sensors for Autonomous Pothole Detection with Spatial…*

X *x*

where *Vd* is the total pothole volume, and *Id*ð Þ *x; y* is depth of pixel *p* at location

**Figure 5** illustrates the processing steps in implementing the detection, and visualization potholes and related metrological parameters from the Kinect v2.0 RGB-D, based on the experimental iMMSS data capture system. In summary the processing system should comprise of data acquisition and geometric transformation; preprocessing for noise minimization; cascaded pothole detection approach from fused RGB-D data using dual-clustering approach comprising of *k*-means and spatial fuzzy *c*-means, and a parallel processing system for pothole area and volume

*Processing pipelines for pothole detection based on cascaded dual-clustering and pothole metrology quantification and visualization from multimodal iMMSS low-cost RGB-D sensor system.*

*Id*ð Þ� *x; y Ip*ð Þ *x; y* (22)

*Vd* ¼ *l* 2 *p* � X *y*

**4.5 Prototype implementation strategy for pothole detection**

depth is given by Eq. (22)

*DOI: http://dx.doi.org/10.5772/intechopen.88877*

**using low-cost sensor**

detection from RGB and depth imagery.

ð Þ *x; y* .

**Figure 5.**

**161**

$$A\_r = \frac{\pi}{6} \sqrt{\left(1 + 4r^2\right)^3} \tag{18}$$

If pixels counts are used, then Eq. (19) can be implemented, [8]. Whereby in Eq. (19), *l* is the pixel size and *Ip* is the binary value of pixel at coordinate position (x,y). The area *Ap* is estimated on the basis of the average of a 2 � 2 window.

$$A\_p = l^2 \cdot \sum\_{\mathbf{x}} \sum\_{\mathbf{y}} I\_p(\mathbf{x}, \mathbf{y}) \tag{19}$$

#### **4.4 Pothole volume estimation**

According to [50], if *T* is a closed region bounded by a surface *S*, and *F* is a vector field defined at each point of *T* and on its boundary surface, then ÐÐÐ *<sup>T</sup>Fdv* is the volume integral of *F* through the bounded region *T*. As in case for the surface area of a pothole, the area is either estimated by an elliptic paraboloid or a circular paraboloid. The volume of the elliptic paraboloid *V* can be estimated according to Eq. (20), and the volume *Vr* 0f the pothole is estimated using a circular paraboloid as in Eq. (21).

$$V = \frac{4}{3}\pi abd\_{\text{max}}\tag{20}$$

$$V\_r = \frac{\pi r^4}{2} \tag{21}$$

Since the depth for each pixel *di* is obtainable from the depth image, the integration of all small volumes represented by each pixel leads to the total volume of *On the Use of Low-Cost RGB-D Sensors for Autonomous Pothole Detection with Spatial… DOI: http://dx.doi.org/10.5772/intechopen.88877*

area within the frame [51]. Therefore the estimated volume *Vd* in terms of the pixel depth is given by Eq. (22)

$$V\_d = l\_p^2 \cdot \sum\_{\mathcal{y}} \sum\_{\mathbf{x}} I\_d(\mathbf{x}, \mathcal{y}) \cdot I\_p(\mathbf{x}, \mathcal{y}) \tag{22}$$

where *Vd* is the total pothole volume, and *Id*ð Þ *x; y* is depth of pixel *p* at location ð Þ *x; y* .

### **4.5 Prototype implementation strategy for pothole detection using low-cost sensor**

**Figure 5** illustrates the processing steps in implementing the detection, and visualization potholes and related metrological parameters from the Kinect v2.0 RGB-D, based on the experimental iMMSS data capture system. In summary the processing system should comprise of data acquisition and geometric transformation; preprocessing for noise minimization; cascaded pothole detection approach from fused RGB-D data using dual-clustering approach comprising of *k*-means and spatial fuzzy *c*-means, and a parallel processing system for pothole area and volume detection from RGB and depth imagery.

#### **Figure 5.**

*Processing pipelines for pothole detection based on cascaded dual-clustering and pothole metrology quantification and visualization from multimodal iMMSS low-cost RGB-D sensor system.*

**4.2 Pothole width measurement**

*Geographic Information Systems in Geospatial Intelligence*

area and volume of the pothole.

**4.4 Pothole volume estimation**

as in Eq. (21).

**160**

**4.3 Pothole surface area determination**

The width of a pothole can be defined by the semi-major *a* and semi-minor *b* axes, on the assumption that an ellipse, based on the major path elliptic regression, is used pothole shape extraction [48]. To determine the lateral width of the pothole, it can be estimated using a circular paraboloid, which is an elliptical paraboloid.

In order to determine the surface area of the pothole, the optimally detected edge is used to fit the shape of the pothole as either elliptic paraboloid or circular paraboloid. While the former is defined by the dimensions of semi-major axis *a* and semi-minor axis *b*, the latter is defined by the estimated radius *r*. The surface area is then computed by using the surface integrals of either of the paraboloids [49], as respectively shown in Eqs. (17) and (18) for the elliptic and circular paraboloids.

> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>1</sup> <sup>þ</sup> <sup>4</sup>*r*<sup>2</sup> ð Þ<sup>3</sup>

ð17Þ

(18)

*<sup>T</sup>Fdv* is

*Ip*ð Þ *x; y* (19)

<sup>3</sup> *<sup>π</sup>abd*max (20)

<sup>2</sup> (21)

And, an elliptical paraboloid is a surface with parabolic cross-sections in 2-orthogonal directions and 1-elliptical cross-section in the other orthogonal direction. Using an edge detection algorithm, the near-true shape of the pothole is first derived using the proposed SFCM, and then an elliptical fit is used to approximate the shape, from which the axes are defined for the calculation of the surface

> *Ar* <sup>¼</sup> *<sup>π</sup>* 6

*Ap* ¼ *l* 2 � X *x*

q

If pixels counts are used, then Eq. (19) can be implemented, [8]. Whereby in Eq. (19), *l* is the pixel size and *Ip* is the binary value of pixel at coordinate position (x,y). The area *Ap* is estimated on the basis of the average of a 2 � 2 window.

According to [50], if *T* is a closed region bounded by a surface *S*, and *F* is a vector field defined at each point of *T* and on its boundary surface, then ÐÐÐ

the volume integral of *F* through the bounded region *T*. As in case for the surface area of a pothole, the area is either estimated by an elliptic paraboloid or a circular paraboloid. The volume of the elliptic paraboloid *V* can be estimated according to Eq. (20), and the volume *Vr* 0f the pothole is estimated using a circular paraboloid

*<sup>V</sup>* <sup>¼</sup> <sup>4</sup>

*Vr* <sup>¼</sup> *<sup>π</sup>r*<sup>4</sup>

Since the depth for each pixel *di* is obtainable from the depth image, the integration of all small volumes represented by each pixel leads to the total volume of

X *y*
