*Material Classification via Machine Learning Techniques: Construction Projects Progress… DOI: http://dx.doi.org/10.5772/intechopen.96354*

The accuracy of the allocated labels was checked by pixel-based field comparison to manual labels. Lee and Park [22] developed automatic reinforcing-bar image analysis system (ARIAS), which could separate the background for the bar area calculation. The model was also able to count the number of bars by testing various combinations between RF and super-pixel method (RFSP), otsu threshold for extracting the bars areas (OTSU), hough transforms (HT), and iterative voting for binary image (IVB). The combination RFSP+IVB gave better output results than other combinations. Ghassemi et al. [23] proposed the material classification model based on deep learning (DL) algorithm for classifying in various illumination conditions and different camera angles/positions. Eleven construction materials (sandstorms, paving, gravel, stone, cement-granular, brick, soil, wood, asphalt, clay hollow block, and concrete block) were classified in this study, and good accuracy was achieved by VGG16 algorithm, even for images that were hard to identify by the human eye. Yuan et al. [8] proposed an automatic material classification method with the help of 2D digital images using the graphical characteristics of building materials. A coloured laser scan data was generated using a terrestrial laser scanner (TLS) with a built-in camera, which contained the surface geometries, material reflectance and surface roughness of building materials. TLS data were classified using DT, Discriminant Analysis (DA), Naive Bayes (NB), SVM, and KNN. A laser scan database for ten common construction building materials (stone, pottery, mortar, concrete, ceramic, wood, plastic, plaster, metal, and painting) was used to train and validate the model. However, better results were achieved from one-class SVM (OC-SVM) and SVDD. Fernando and Marshall [13] performed a state of the art classification methodology for rock and gravel, by identifying force data (proprioceptive force data) acquired from load-haul-dump equipment with capacity of 14-tonne and adopting ANN, KNN, and k-mean algorithms. However, good results were obtained by ANN (5-NN) model with more realistic classification. Since the system only relies on proprioceptive sensing, it is feasible in harsh, dusty, and dark conditions that hinder the use of external sensor data. **Table 4** exemplifies the


colour space was transformed to non-RGB colour spaces to enhance the separability between background classes and concrete. The model was tested for three ML algorithm, i.e., GMM, ANN, and SVM. However, SVM along with hue-saturationand-intensity (HSI) colour space was found more effective. Yazdi and Sarafrazi [14] used five different classifiers, ANN, SVM, bayesian, KNN, and fisher's linear discriminate (FLD) algorithm in combinations and as separate, i.e. Bayesian, Bayesian + FLD, KNN, KNN + FLD, SVM, SVM + FLD, ANN, and ANN + FLD on the segmented concrete images. This study found the ANN model as the better option for automatic image segmentation. Dimitrov and Golparvar-Fard [9] proposed C-support vector machine (C-SVM) algorithm combined with texture and hue-saturation-value (HSV) colour features. This study tested the developed algorithm on 20 construction materials (paving, grass, gravel, stone-limestone, formwork, soil-vegetation, marble, metal-grills, soil-mulch, soil-compact, stonegranular, wood, soil-loose, asphalt, cement-granular, brick, concrete-cast, cementsmooth, foliage, and concrete-precast) with more than 150 images for each category and compared various pixel sizes (n x n), i.e., 30, 50, 75, 100, 150, and 200. Better accuracy and effective output were reported for 200 x 200 pixel-images. Han and Golparvar-Fard [3] developed a construction material library (CML) based on C-SVM classifiers with linear *<sup>x</sup>*<sup>2</sup> kernels on 100 100, 75 75, and 50 <sup>50</sup> pixel-images datasets of cement-based surfaces, paving, brick, asphalt, formwork, foliage, concrete, marble, gravel, insulation, metal, soil, wood, stone, and waterproofing paint. This study developed an appearance-based material classification technique for progress monitoring using daily photologs and BIM. Point cloud models were generated using SfM and multi-view-stereo (MVS) algorithms from the construction site images. These point cloud models were superimposed with 4D BIM models, and registered site images were back-projected for the BIM elements

*Deep Learning Applications*

for extracting related image patches. Testing of the extracted patches was performed with multi-class material classification technique that was pre-trained with the extended CML dataset. Rashidi et al. [5] conducted a comparison study on SVM, radial basis function (RBF), and multilayer perceptron (MLP) by evaluating the performance on building construction materials, i.e. OSB boards, red brick, and concrete. The feature extraction was performed from image blocks to compare the efficiency for detecting building construction materials, and SVM classifier with RBF kernel results were found more precise in perceiving the images for material textures. Yang et al. [17] performed material recognition of windows, walls, protrusions, tile, brick, stone, and coating using image-based 3D modelling. SfM was used for generating the 3D point cloud model for site images. The building facade was modelled as combined planes of protrusions, windows, and wall. Planes were

primarily detected from 3D point clouds using random sample consensus

**190**

(RANSAC) and further recognised as distinct structural components by SVM classifier. Hamledari et al. [16] adopted CV integrated with shape and colour-based modules that automatically detect the interior components using 2D digital images, i.e. three states drywall sheets (plastered, installed, and painted), insulation, studs, and electrical outlets. Cumulative distribution function (CDF) used for electrical outlet module, SVM classifier has an RBF kernel type for drywall, local binary patterns (LBP) for insulation module, and progressive probabilistic hough transform (PPHT) for stud module. The method was validated by indoor construction site images captured by unmanned aerial vehicle (UAV), smartphone and internet sources. Braun and Borrmann [21] developed an automatically labelling process via construction images with 4D BIM and 3D point cloud approach. The 3D point cloud model was integrated with the BIM model, and automated labelling for structural elements was provided with the semantic information. The convolutional neural network (CNN) model was trained on this information to generate classification tasks.


**Ref Adopted technique**

[22] RFSP, OTSUHT, & IVB

[8] DT, DA, NB, SVM, & KNN

[13] ANN, KNN, & k-mean

**Table 4.**

**Figure 2.**

**193**

*General workflow of ML-material classification models.*

**Main features Achieved outcomes Remarks/observed**

score.

*Material Classification via Machine Learning Techniques: Construction Projects Progress…*

of 97.35%.

an average

of 96.7%.

of 90%.

*Technical summary of collected articles for progress monitoring via ML of the building construction projects.*

RFSP+IVB gave better results with 0.89 as F-

VGG16 algorithm gave the maximum accuracy

OC-SVM and SVDD gave better results with

classification accuracy

ANN model was found effective with a classification accuracy

The model performs analysis on the reinforcing bars of the production plant moving along a conveyor belt by accurately calculating the bar area and its number.

model by data augmentation & prevents over-fitting of network structures for images with varying camera resolution, illumination, &

The TLS based laser scan data can provide information for building material for surface geometries such as surface roughness and material

The study followed a material classification methodology using proprioceptive force data acquired from an digging machine integrated with ML. The model is pertinent in the dusty, dark

[23] ANN The method uses the DL

*DOI: http://dx.doi.org/10.5772/intechopen.96354*

small datasets.

reflectance.

and harsh areas.

**limitations**

Raspberry Pi 3 was used with datasets taken from different construction sites with 1231 images of 11 classes for various views of materials.

In this study, only plane target surfaces were considered.

The model only covers rock & gravel, where excavated soil may consist of various other

materials.


*Material Classification via Machine Learning Techniques: Construction Projects Progress… DOI: http://dx.doi.org/10.5772/intechopen.96354*

#### **Table 4.**

**Ref Adopted technique**

*Deep Learning Applications*

[9] C-SVM (SVM)

[3] C-SVM (SVM)

[5] MLP, ANN, & RBF

[16] CDF, SVM, RBF, LBP, & PPHT

[21] CNN (ANN)

**192**

Various pixels sizes were tested, i.e., 30 30, 50 50, & 200 200. Material datasets were created for varying degrees of viewpoint, illumination, and scales.

This study adopted BIM integrated daily construction photologs for extraction relevant image patches. Study adopted materials library for images with three different patch sizes of 50 50, 75 75, & 100 100.

For feature extraction, the construction materials were divided into three groups: 1) materials encompasses a very distinct colour, 2) variable colour patterns, & 3) material without a distinctive

material detection method & image-based 3D modelling. The 3D model was generated via SfM and segmented into planar components, which were recognised as structural components via knowledge-

Input source data was tested for UAV, smartphones & internet sources. A CV technique was adopted for the detection of interior components & extrapolated the existing scenario via 2D

colour pattern.

based reasoning.

digital images.

The study follows an automated construction materials' labelling process of construction site images. The model combines the available information from the photogrammetric model & the 4D BIM. By aligning the BIM & 3D point cloud, a digital components can be projected onto the image.

[17] SVM The study performed the

**Main features Achieved outcomes Remarks/observed**

97.1% average classification rate was achieved for 200 200 pixel images.

The accuracy of 92.4% was achieved for the dataset with 100 100 pixel size images.

SVM classifier with RBF kernel was found better than other techniques.

RANSAC was adopted for detection in the point cloud. The model achieved an average accuracy of 95.55%.

Three datasets were adopted, and the following outcomes were attained: 1) for stud module, precision above 90%, & recall above 83%. 2) for insulation module, precision above 88%, & recall above 89%. 3) for electric outlet module, precision above 86%, & recall above 87%.

91% pixel-wise accuracy was validated by the sample.

**limitations**

include

This study did not cover the challenge of segmentation.

Practical limitations for adopted methodology

comprehensiveness of materials library, completeness of 3D reconstruction, and computation time.

Outcomes may be affected due to lack of adequate light, varying viewpoint angle, & image capturing distance.

The datasets consist of 463 stone samples, 637 brick samples, 504 tile samples, & 409 coating

Practical limitations for adopted methodology include the inability to detect & metallic electrical boxes, partitions with low visibility, & improperly captured scenes.

The construction & model inaccuracies, errors in post estimation during SfM, large scale deviations for real world coordinates, & occlusions may cause labelling errors.

samples.

*Technical summary of collected articles for progress monitoring via ML of the building construction projects.*

**Figure 2.** *General workflow of ML-material classification models.* technical summary of collected studies for their main features, achieved outcomes, and observed limitation if any.

dataset comprised of 50 images. As the model has been designed to train on 75% of the provided data; therefore, for each construction item, the model was trained on 38 images. The remaining 12 images were used by the model for validation or testing. Two different models were trained on varying epochs per run, i.e., 150 and 300. The first model, with 150 epochs per run, attained the accuracy of 56% as shown in **Figure 3**, where its graphical representation for the attainment of model accuracy and loss can be seen in **Figure 4**. It can be seen that 'Asphalt' in this model got '0' (zero) value for precision and F1 score. The maximum precision and F1 score have been attained for wood (precision = 0.75, F1 score = 0.80). Whereas, the lowest precision and F1 score have been achieved by concrete block (preci-

*Material Classification via Machine Learning Techniques: Construction Projects Progress…*

In the second model, the epochs per run for the model was set to 300, and 64% model accuracy was attained. The attained model accuracy, data loss along with precision, recall and F1 score can be seen in **Figure 5**, where the graphical representation of the training data for the attainment of model accuracy and loss for 300 epochs can be seen in **Figure 6**. The accuracy and F1 score of the second model,

sion = 0.35, F1 score = 0.56).

*DOI: http://dx.doi.org/10.5772/intechopen.96354*

*Training/testing model output for 150 epochs per run.*

*Graphical representation of model accuracy and loss for 150 epochs per run.*

**Figure 3.**

**Figure 4.**

**195**

The general workflow of ML material classification model, via images, is shown in **Figure 2**. In the training process, the model is usually trained with the help of datasets of construction material images. The segmentation process is performed on the collected images, as segmentation increases the performance output of the model as compared to non-segmented images [24]. GMM technique can also be utilised for image segmentation. The feature extraction is performed on the segmented images for their colour, texture, compactness, contrast. Researchers mostly have adopted HSV, RGB (red, green and blue), Image patch (IP), 48-dimensional form (LM), and 18-dimensional rotationally invariant form (rLM) for features extraction and found better results with LM + HSV combination [9]. Depending upon the type of model, clustering process can be adopted, and on this data any ML classifier can be applied for model training. While testing the model for any random material image, the segmentation and feature extraction processes are performed, which are further identified with the help trained ML model for final material recognition output.
