**2. Methodology**

OpenStreetMap, are useful for approximate construction surveys in urban areas. However, online maps are often insufficient for infrastructure inspection to recognize the details of natural features. Thus, base maps and 3D data should be prepared before inspection. Massive point-cloud data can be acquired with a terrestrial laser scanner, mobile mapping systems (MMSs), handheld laser scanners, and cameras using structure from motion (SfM) methodology. SfM is a methodology for reconstructing a scene using multiple cameras simultaneously from all available relative motions through key point detection, feature matching, motion estimation, triangulation, and bundle adjustment. In an open sky environment, aerial photogrammetry and SfM using an unmanned aerial vehicle (UAV) are more effective than groundbased scanning. On the other hand, when environments include natural obstacles, such as trees, a terrestrial laser scanner is more effective than a UAV or MMSs. In indoor navigation and building information modeling (BIM), floor maps and 3D data are also required. We expected terrestrial laser scanners and indoor MMSs to be adequate for colored point-cloud

Moreover, point-cloud clustering is an essential technique for modeling massive point clouds. **Figure 1** shows an example of point-cloud clustering using a terrestrial laser scanner data

There are three clustering approaches in point-cloud clustering, namely model-based clustering [1], edge-based clustering [2], and region-based clustering [3]. Model-based clustering is a 3D model preparation approach. The model-based clustering requires 3D models such as CAD models to estimate simple objects or point clusters from the point cloud. In 3D industrial modeling, standardized objects, such as pipes, boxes, and parts, are prepared as CAD models in advance. Although the model-based clustering is suitable for modeling known objects such as the standardized objects, the model-based clustering is unsuitable for modeling unknown objects such as complex and natural objects. On the other hand, in modeling unknown objects, such as buildings and roads in geoinformatics and civil engineering

**Figure 1.** Point-cloud clustering: colored point cloud (left image) and clustered point cloud (right image).

acquisition in an indoor environment.

76 Recent Applications in Data Clustering

acquired in an indoor environment.

Our proposed processing flow for point-cloud clustering is shown in **Figure 2**. First, we register and integrate point-cloud data acquired from a viewpoint. Next, the point-cloud data are projected into the image space with translation, view angle, and resolution parameters in "Panoramic multilayered range image generation with point cloud rendering" to generate several range images. Then, normal vectors around each projected point are estimated using the 3D coordinate values of the point cloud in "Normal vector estimation in panoramic multilayered range image." Next, edges are extracted from depth images generated in the panoramic

to generate missing points in the rendered result using distance values between the viewpoint and objects. The colored point cloud is projected from a 3D space to a panorama space. This transformation simplifies viewpoint translation, filtering, and point-cloud browsing. The LIDAR VR data consist of a projection model and multilayered range image, as shown in **Figure 3**. The panorama model can be selected from a spherical, cylindrical, plane, hemispherical, or cubic model. In this chapter, the spherical model is described. First, the measured point data are projected onto a spherical surface as a range of data. Next, the measured point data are projected onto a spherical surface to manage X, Y, Z, R, G, B, and the intensity values in the multilayered panorama space. Then, azimuth and elevation angles are calculated using 3D vectors generated from the viewpoint and the measured points. The azimuth and elevation angles are converted to panorama image coordinate values with adequate spatial angle resolution in the range data. Finally, a spherical panorama image is generated from

Based on this panorama projection, the multilayered range data with a translated viewpoint are generated using the point cloud, as shown in **Figure 4**. When a panorama space is gener-

the points from *P1*

Point Cloud Clustering Using Panoramic Layered Range Image

http://dx.doi.org/10.5772/intechopen.76407

79

to *P10* are continuously

to *P10* from a viewpoint *Xo*

the measured point cloud.

ated using points from *P1*

**Figure 3.** LIDAR VR data model.

**Figure 2.** Processing flow for point-cloud clustering.

multilayered range image generation in "Depth edge detection." Finally, groups that have similar direction in the point cloud are extracted after normal vector classification in the projected image in "Normal vector classification in projected image." Generated range images are managed in a multilayered range image.

#### **2.1. Panoramic multilayered range image generation with point-cloud rendering**

An advantage of 3D point-cloud data is that they allow accurate display from an arbitrary viewpoint and 3D modeling. Additionally, point-cloud data have the potential for applications such as panoramic image geo-referencing and distance value-added panoramic image processing for 3D geographical information system (GIS) visualization [9, 10]. However, point-cloud visualization has two technical issues. The first issue is the near-far effect caused by distance differences from the viewpoint to scanned points. The second issue is the transparency effect caused by rendered hidden points. These effects degrade the quality of pointcloud visualization. Thus, we focus on methodologies to improve the quality of point-cloud visualization. The Splat-based ray tracing [11] can generate a photorealistic curved surface from point-cloud data; surface generation requires the long period in the 3D work space. Moreover, the curved-surface description is unsuitable for representing urban objects such as CAD and GIS data. Therefore, we have applied a point-based rendering and filtering, which we call a layered image-based depth arrangement refiner for versatile rendering (LIDAR VR) [12] for point-cloud rendering.

The processing flow of LIDAR VR is described as follows. First, the sensors acquire a point cloud with additional color data such as RGB data. The sensor position is defined as the origin point in a 3D work space. Second, a multilayered range image from the simulated viewpoint is generated using the point cloud. Finally, the generated multilayered range image is filtered to generate missing points in the rendered result using distance values between the viewpoint and objects. The colored point cloud is projected from a 3D space to a panorama space. This transformation simplifies viewpoint translation, filtering, and point-cloud browsing. The LIDAR VR data consist of a projection model and multilayered range image, as shown in **Figure 3**. The panorama model can be selected from a spherical, cylindrical, plane, hemispherical, or cubic model. In this chapter, the spherical model is described. First, the measured point data are projected onto a spherical surface as a range of data. Next, the measured point data are projected onto a spherical surface to manage X, Y, Z, R, G, B, and the intensity values in the multilayered panorama space. Then, azimuth and elevation angles are calculated using 3D vectors generated from the viewpoint and the measured points. The azimuth and elevation angles are converted to panorama image coordinate values with adequate spatial angle resolution in the range data. Finally, a spherical panorama image is generated from the measured point cloud.

Based on this panorama projection, the multilayered range data with a translated viewpoint are generated using the point cloud, as shown in **Figure 4**. When a panorama space is generated using points from *P1* to *P10* from a viewpoint *Xo* the points from *P1* to *P10* are continuously

**Figure 3.** LIDAR VR data model.

multilayered range image generation in "Depth edge detection." Finally, groups that have similar direction in the point cloud are extracted after normal vector classification in the projected image in "Normal vector classification in projected image." Generated range images

An advantage of 3D point-cloud data is that they allow accurate display from an arbitrary viewpoint and 3D modeling. Additionally, point-cloud data have the potential for applications such as panoramic image geo-referencing and distance value-added panoramic image processing for 3D geographical information system (GIS) visualization [9, 10]. However, point-cloud visualization has two technical issues. The first issue is the near-far effect caused by distance differences from the viewpoint to scanned points. The second issue is the transparency effect caused by rendered hidden points. These effects degrade the quality of pointcloud visualization. Thus, we focus on methodologies to improve the quality of point-cloud visualization. The Splat-based ray tracing [11] can generate a photorealistic curved surface from point-cloud data; surface generation requires the long period in the 3D work space. Moreover, the curved-surface description is unsuitable for representing urban objects such as CAD and GIS data. Therefore, we have applied a point-based rendering and filtering, which we call a layered image-based depth arrangement refiner for versatile rendering (LIDAR VR)

The processing flow of LIDAR VR is described as follows. First, the sensors acquire a point cloud with additional color data such as RGB data. The sensor position is defined as the origin point in a 3D work space. Second, a multilayered range image from the simulated viewpoint is generated using the point cloud. Finally, the generated multilayered range image is filtered

**2.1. Panoramic multilayered range image generation with point-cloud rendering**

are managed in a multilayered range image.

**Figure 2.** Processing flow for point-cloud clustering.

78 Recent Applications in Data Clustering

[12] for point-cloud rendering.

averaging filter to improve the quality of the range data generated from point-cloud data with the view-point translation. However, general image resampling techniques, such as the nearest interpolation, linear interpolation, and cubic interpolation, degrade the quality of the range data because the resampling blends various data, such as valid points, occluded points, measurement noises, and missing data. Therefore, the pixel-selectable averaging filter is applied to this technical issue. The pixel-selectable averaging filter extracts valid points around a point for the resampling, as shown in **Figure 5**. This processing consists of a detection of valid data extraction and rejection of occluded points, noises, and missing data, and

Point Cloud Clustering Using Panoramic Layered Range Image

http://dx.doi.org/10.5772/intechopen.76407

81

The detailed flow of the pixel-selectable averaging filter is described as follows. First, a threeby-three block of pixels is prepared in the range data projected from point clouds. The center point in the block is the focus point in the range data. Second, the block is checked to see whether valid points exist. When there are more than two valid pixels, the processing moves to the next step. If there is only the focus point, it is deleted as spike noise. When the focus point is a missing part, a new pixel value such as color and intensity value is given to the focus point using the other valid points around the center point in the block. Third, after these point extraction steps, an average value of valid points in the block within the search range is calculated to overwrite the focus point value. The average value is a distance from the viewpoint to the valid points. This processing is applied to each channel in the RGB and intensity image. However, when the center point in the block has a distance value within the search range, the overwriting processing is not performed. Because, when the point can be defined approximately as the nearest surface, the overwriting processing has a possibility of degrading geometrical accuracy and image quality in this case. This processing sequence is

In the valid point extraction, a range of search distances should be given. The distance from the viewpoint to the nearest point found among the extracted points is defined as the start

missing-point regeneration.

applied to all points.

**Figure 5.** Pixel-selectable averaging filter.

**Figure 4.** Point distribution calculated by viewpoint translation in range data, occlusion detection using the point cloud, and point-cloud interpolation with distance information.

arranged in the range data. An azimuth or elevation angle from a viewpoint *Xo* to a measured point *P1* is denoted as *Ro* . On the other hand, when a panorama space is generated using the points from *P1* to *P10* from a different viewpoint *Xt* , the angle from the viewpoint *Xt* to the measured point *P1* is denoted as *Rt* . Thus, the change in angle from *Ro* to *Rt* affects the arrangement of the projected points in the range data.

After the viewpoint translation, three types of filtering are applied to point-cloud rendering, as shown in **Figure 4**. The first filtering is the occluded point overwriting. **Figure 4** shows an example to overwrite the projected point *P1* by the projected point *P2* . After the viewpoint translation from *Xo* to *Xt* , the projected point *P1* becomes an occluded point behind *P*. Thus, **Figure 4** shows that *P1* is overwritten by *P2* . The second filtering is the new point generation in the no-data space. **Figure 4** also shows an example to generate a new point *Pnew1*. After the viewpoint translation from *Xo* to *Xt* , a no-data space occurs between the projected points *P3* and *P4* . Therefore, **Figure 4** shows that *Pnew1* is generated between the projected points *P3* and *P4* . The third filtering is the occluded point replacement. **Figure 4** shows an example to replace an occluded point *P8* with a new point *Pnew2*. After the viewpoint translation from *Xo* to *Xt* , the point *P8* is visible between points *P9* and *P10*. However, the point *P8* exists behind the real surface. Thus, the occluded point *P8* should be given a new distance value as *Pnew2*. The new distance value is calculated the distance values of points *P9* and *P10* through the pixel-selectable averaging filter developed in this study, which we now describe.

In general image processing, each pixel value in an image is resampled by using pixel values around it when the image is transformed. A similar technique is applied to the pixel-selectable averaging filter to improve the quality of the range data generated from point-cloud data with the view-point translation. However, general image resampling techniques, such as the nearest interpolation, linear interpolation, and cubic interpolation, degrade the quality of the range data because the resampling blends various data, such as valid points, occluded points, measurement noises, and missing data. Therefore, the pixel-selectable averaging filter is applied to this technical issue. The pixel-selectable averaging filter extracts valid points around a point for the resampling, as shown in **Figure 5**. This processing consists of a detection of valid data extraction and rejection of occluded points, noises, and missing data, and missing-point regeneration.

The detailed flow of the pixel-selectable averaging filter is described as follows. First, a threeby-three block of pixels is prepared in the range data projected from point clouds. The center point in the block is the focus point in the range data. Second, the block is checked to see whether valid points exist. When there are more than two valid pixels, the processing moves to the next step. If there is only the focus point, it is deleted as spike noise. When the focus point is a missing part, a new pixel value such as color and intensity value is given to the focus point using the other valid points around the center point in the block. Third, after these point extraction steps, an average value of valid points in the block within the search range is calculated to overwrite the focus point value. The average value is a distance from the viewpoint to the valid points. This processing is applied to each channel in the RGB and intensity image. However, when the center point in the block has a distance value within the search range, the overwriting processing is not performed. Because, when the point can be defined approximately as the nearest surface, the overwriting processing has a possibility of degrading geometrical accuracy and image quality in this case. This processing sequence is applied to all points.

In the valid point extraction, a range of search distances should be given. The distance from the viewpoint to the nearest point found among the extracted points is defined as the start

**Figure 5.** Pixel-selectable averaging filter.

arranged in the range data. An azimuth or elevation angle from a viewpoint *Xo*

, the projected point *P1*

is overwritten by *P2*

to *Xt*

. Thus, the change in angle from *Ro*

**Figure 4.** Point distribution calculated by viewpoint translation in range data, occlusion detection using the point cloud,

After the viewpoint translation, three types of filtering are applied to point-cloud rendering, as shown in **Figure 4**. The first filtering is the occluded point overwriting. **Figure 4** shows an

in the no-data space. **Figure 4** also shows an example to generate a new point *Pnew1*. After the

. Therefore, **Figure 4** shows that *Pnew1* is generated between the projected points *P3*

. The third filtering is the occluded point replacement. **Figure 4** shows an example to replace

In general image processing, each pixel value in an image is resampled by using pixel values around it when the image is transformed. A similar technique is applied to the pixel-selectable

with a new point *Pnew2*. After the viewpoint translation from *Xo*

and *P10*. However, the point *P8*

to *P10* from a different viewpoint *Xt*

. On the other hand, when a panorama space is generated using the

by the projected point *P2*

, the angle from the viewpoint *Xt*

to *Rt*

becomes an occluded point behind *P*. Thus,

. The second filtering is the new point generation

, a no-data space occurs between the projected points *P3*

should be given a new distance value as *Pnew2*. The new dis-

point *P1*

and *P4*

point *P8*

*P4*

points from *P1*

sured point *P1*

translation from *Xo*

**Figure 4** shows that *P1*

an occluded point *P8*

viewpoint translation from *Xo*

face. Thus, the occluded point *P8*

is denoted as *Ro*

80 Recent Applications in Data Clustering

is denoted as *Rt*

and point-cloud interpolation with distance information.

example to overwrite the projected point *P1*

to *Xt*

is visible between points *P9*

tance value is calculated the distance values of points *P9*

averaging filter developed in this study, which we now describe.

of the projected points in the range data.

to a measured

affects the arrangement

. After the viewpoint

to the mea-

and

, the

to *Xt*

exists behind the real sur-

and *P10* through the pixel-selectable

value of the search range. Moreover, the start value plus a defined distance parameter is assumed as the end value. The defined distance parameter is determined with the continuity of the points in the point cloud. For example, the defined parameter would be between 10 cm and 1 m from experience, when trees and building walls are measured.

Thus, the pixel-selectable averaging filter uses valid points in the range data to achieve an interpolation without reducing geometrical accuracy by a uniform smoothing effect. **Figure 6** shows an example of processing result.
