Industrial Fluids Components Health Management Using Deep Learning

*Vidyadevi G. Biradar, H.C. Nagaraj, S.G. Mohan and Piyush Kumar Pareek*

## **Abstract**

The fatigue state of fluid components such as valves, metal surfaces in gas or oil carrying pipelines is important to monitor on regular basis and plan for repair work to avoid risks associated with them, this becomes more crucial when the pipelines are supplying hazard prone fluids. There exist methods for detection of corroded surfaces, scratches and fractures in pipelines, valves, and regulators etcetera. The conventional methods are based on sensors and chemical analysis methods. There are challenges with conventional methods pertaining to the desired metric of scalability and disadvantages of these methods is they are contact based and destructive methods. Therefore, to overcome these limitations of existing methods there is a need for development of non-contact and nondestructive methods. The recent advancements in Artificial Intelligence technology in every domain including health care monitoring, agriculture sector, defense applications and civilian applications etc., have shown that deep learning methods can be explored in industrial applications to develop fault tolerant systems which help fluid components state of health monitoring through computer vision. In this chapter proposes various methods for analysis of health state of fluid components using deep convolutional neural networks and suggest the best models for these applications.

**Keywords:** deep learning, fluid component, convolutional neural network

## **1. Introduction**

In oil and gas transportation industry metal pipelines are the major transport means for transporting fluids such as crude oils, petrochemical products for long distance. Due to various environmental conditions and fatigue induced in pipelines as a result of fluid pressure over a period time of operation the health of pipelines will be deteriorated due to corrosion, dents, and damages and other reasons which leads to various types of pipelines defects [1]. Therefore, timely maintenance of pipelines plays vital role in avoidance of untoward incidents and economic loss etc. the early detection of pipeline defect helps in planning preventive maintenance to mitigate corrosion progression, obstacle in flow etcetera and thus reduce

maintenance costs [2, 3]. The conventional methods such as crack detection using camera, magnetic field, acoustic methods, and thermal camera are very useful methods and demonstrate satisfactory performance [4], however, these are tedious and dependable on environmental conditions. Therefore, there is a need for automatic techniques. Typical pipeline defects are caused due to metal loss, dents, stress induced cracks, gouge, and coating damage etc. The quick and reliable detection of leakages is very much essential to avoid hazards. It is required to ensure that these fluids are safely transported to the destination.

Identifying leaks at right time is essential to avoid serious problems. Methods used for leakage detection are, i) Distributed temperature monitoring approaches utilizing optical fibers to identify and localize leakages, ii) acoustic impact monitoring method, iii) artificial neural network technique and, Leakage detection techniques needs improvement to achieve greater precision in identification of defect location [3].

The objectives of books chapter are listed below.


The first objective of the chapter is discussed in introduction section, Section 2 gives insight into deep learning methods for fluid component damage detection and design of convolutional neural networks, Section 4 gives guidelines for transfer learning strategies and Section 5 presents conclusion.

This chapter provides insight into the alternate method for pipeline damage detection is deep learning paradigm. This chapter presents practical perspectives of convolutional neural network and provides guidance on transfer techniques to tune the pretrained model to solve the problem.

## **2. Deep learning models for pipeline damage detection**

In industries defect detection is performed by an expert human expert to analyze the defect patterns [5], manual analysis by experts is tedious and time-consuming job, therefore, there is a need for automated technique. The automated techniques which utilize computer vision help to solve these challenges.

The identification and classification of pipe damages such as cracks with image analysis combined with neuro fuzzy algorithms are presented in [6], here the significant features considered for characterization were features from Hough transforms, morphological operations, shape statistics, regression analysis and eigen vector analysis. The defects are classified using back propagation algorithm.

Artificial neural networks (ANNs) possess learning skills and capable of adapting themselves to alterations in the training phase. ANNs are interrelated groups of

### *Industrial Fluids Components Health Management Using Deep Learning DOI: http://dx.doi.org/10.5772/intechopen.107929*

neurons, neural networks are used in modeling complex connections between inputs and outputs, neural networks gave good results in detection of cracks in pipelines [3].

Pipe cracks detection using computer vision helps to solve the issues of conventional methods, however, with complex background content in the image makes problem solving a challenging job. A method based on computer vision is presented in [7]. The methodology of detecting cracks apply image filtering for background subtraction using tuned adaptive thresholding technique and crack contour is extracted using morphological operator. To understand the depth of the crack 3D visualization is performed using successive images [8].

Pipeline cracks are detected and classified using image processing techniques, a method which converts RGB into gray scale image, apply Sobel operator to edge points extraction and through edge linking judge the artifacts are holes, cracks etc., is presented in [5].

Deep learning models provide flexibility in the process of defect detection as the model adapts to the dataset in learning relevant features and therefore give higher success rate of defect identification and classification. The purpose of defect detection depends on the application. In some cases, just detecting the presence of a defect is sufficient and others, classification and labelling are important [3, 9].

Image processing and machine learning algorithms largely depend on the accuracy of image feature extraction which is a challenging job and which intern depends on the image quality. Therefore, automatic feature extraction using data driven methods come for rescue. The convolutional neural networks are widely used for automatic feature extraction and classification. The deep learning models have shown highest success among the state-of-the-art approaches to solve computer vision problems [10]. A deep convolutional neural network with configurations of convolutional layers, filters, batch normalization and pooling in different combination are experimented for determining optimal hyperparameters of the model and encouraging results are obtained.

## **3. Convolutional neural networks**

The convolutional neural networks (CNN) consist of various types of layers with specific functionality. The convolutional layer convolves input image with various filters whose co-efficient are determined during the run of backpropagation algorithm in number of iterations. The convolution operations are carried out using a set of filters to extract images features. The CNN may have any number of convolutional layers as shown in the **Figure 1**, in this, initial layers extract primitive level features from input image such as edges whereas later layers extract high levels features such as shape features and contour etc. The pooling layers reduces feature dimension by extracting significant pixels and down sampling images, min, max, avg. pooling are the typical available options. The drop out layer dynamically changes the pool of neurons in the network to improve the generalization of the model to avoid overfitting, fully connected layer classify the images based on the features extracted by the series of convolution layers [11]. To improve the speed of the model training and avoid vanishing gradient problem activation layer are added to CNN network, different types of activation functions such as Tanh, ReLu, Sigmoid and Softmax are used. Among these the popular one is ReLu and its variants [12].

An input layer, hidden layers, and output layer are different types of layers in CNN. The design of the neural networks is the way human brain works. Input layers

### **Figure 1.**

*Architectural components of CNN.*

collect inputs, process them, and produces the result. CNN has many hidden layers which perform convolution operations to extract features from the input image. The features are classified by the fully connected layer. There are different types of CNN models, all types have convolutional and pooling layers.

**Figure 2** depicts convolution operation, in this operation a filter or kernel is convolved with the input image. The next layer takes in the output generated by first layer and so on. Convolutions operation in image processing are applied to sharpen, blur images and edge detection etc. CNNs establish a connection pattern between neurons of adjacent layers with drop out technique.

The layer of CNN generates number of activation maps from the input image, and these are then fed to the subsequent layers, this process in shown in **Figure 3**. The primitive features such as horizontal and vertical edges are most of the times extracted in the earlier layers. The later layers extract high level features like objects, shape of the object and features which helps in making sense of features.

Pooling Layer – the dimension of feature map is reduced by pooling layers. This in turn reduces the number of parameters of the network and computation time.

**Figure 2.** *Convolution operation.*

*Industrial Fluids Components Health Management Using Deep Learning DOI: http://dx.doi.org/10.5772/intechopen.107929*

**Figure 3.** *Feature extraction in CNN.*

**Figure 4.** *Max pooling operation.*

The down sampling of image using max pooling operation is shown in the **Figure 4** it selects max pixel value from 2x2 image patch and uses this single value to represent feature in the image. In a similar manner, min pooling and average pooling works where in these choose min and average pixel values respectively.

## **4. Pretrained models for pipeline health detection**

This section gives an insight into detailed methodology which is suitable for fluid components health detection using deep convolutional neural networks and covers image augmentation techniques.

The deep learning models automatically determining features are of interest for pipelines defect classification based on the knowledge acquired during training of huge dataset. To understand the behavior of the AI model, model interpretation tools such as Heat maps and Grad CAM tools are used. A U-Net architecture-based model is presented in [13], a method based on YOLOv5 model demonstrated in [14] which uses X-Ray images for detection of pipeline weld defects, the results indicated that YOLOv5 performs better than R-CNN interns of metrics such as speed of detection and classification accuracy.

The detection of cracks which are thin, irregular in shape and with complex texture background in the image makes crack detection task challenging. The deep learning models show improved performance in cracks detection in fluid components, however, there are many bottlenecks for achieving the best performance. These include inadequate training data, imbalanced data and manually labeled data and absence of ground truth information. In addition to this, the computational requirements are very high as deep learning involve training periods [9].

The research work in [15], combines fuzzy digital twin (FDT) and support vector machine (SVM), with backstepping (BS) and good improvement in accuracy compared other methods.

Deep learning tools aid in identification of pipeline damages at the earliest time. This paper utilizes convolutional neural networks for diagnosis of pipeline threats [3].

The CNN model deployed on AUV (autonomous underwater vehicle) is used for detection of underwater pipeline detection, the challenges addressed were distortion in the focus, contrast, and color. The scarcity of the balanced dataset is another issue as number of damaged pipeline sample were less compared to the total number of pipelines. The dataset is preprocessed through augmentation techniques such as flip, scale, shift etc. and generate pipeline sample images which fall in small subset. The issue of color distortion is eliminated by converting images into gray scale, background is eliminated using segmentation. The various models that are experimented for pipeline damage detection are custom architecture CNN, VGG and MobileNet, and among these models, MobileNet outperformed the other models [16].

## **5. Image augmentation**

Additional images are generated from the original by slightly modifying it by adding noise, cropping, changing the contrast and by rotating etc.

The image can be flipped vertically and horizontally, it can be rotated left and right by any angle, the size of the image can be changed, regions of interest can be cropped to generate additional sample images, image can be translated in left, right direction and various types of noises can be added [17].

## **6. Transfer learning**

Transfer learning eliminates the dependency of deep learning models on huge dataset which are required for learning features. The process of transfer learning involves application of a pre-trained model for classification of specific domain image dataset. To customize pretrained model for specific dataset, minor changes to the original architecture can be done and fresh training of convolutional layers can be done. However, the model tuning process is based on trial-and-error method and the model hyperparameters are experimentally determined. A comprehensive survey on transfer learning techniques is given in [18]. A classical example of transfer learning in the context of solving classification of COVID-19 images is discussed in [19].

A typical CNN has two parts, they are convolution layers and classification layers. In transfer learning, the classifier is changed as per the new classification problem. There are different possible ways of finetuning the model, it requires training the model on a new dataset to learn problem specific features.

Hyperparameters of CNN include Learning rate, Momentum, Epochs, Batch size, Filter size, Activation layers, Number of hidden layers, Filter coefficient initialization and Dropout which are very important to achieve the performance of the model. The setting optimal values for these hyperparameters is important to achieve desired level of model accuracy and avoid model overfitting [20, 21].

## **7. Guidelines for training of pre-trained model for repurposing**

Select suitable pretrained model for new problem, the options available are VGG, InceptionV3, and ResNet5, DarkNet and YOLO etc., presented in [22], the guideline is, select pretrained model which has been trained on some medical images in case repurpose to classify another type of medical images. As it discussed in the previous sections in the technique of transfer learning, earlier layers of CNN are freezed and layer set of layers are unfreezed and retrained to learn knowledge from the new dataset. Now, decision must be made on how many layers to freeze and how many to unfreeze and retrain. This depends upon the similarity and size of the present dataset with the dataset on which the model is pre-trained. The guideline is, i) if the size of the present dataset is large in size then reuse the architecture and retrain all the layers of pretrained model, ii) if the dataset is similar to the dataset on which the model is pretrained and the present dataset is large, then just retrain the classifier layers, iii) the present dataset is small in size and different from the dataset used while pretrained model, this is a difficult situation, here the dataset needs to be enlarged through augmentation and by generating synthetic images. The pre-trained model must be trained considering the fact that freshly training deeper layers require high end computational GPPs and time.

## **8. Conclusion**

In oil and gas industry timely metal pipelines damage detection plays important role for planning on maintenance planning which is essential in avoiding hazard and reducing economic loss. The pipelines are regularly inspected for damage detection, the maintenance is carried out by conventional methods with human expert investigations. These methods are time consuming and sometimes are subjective in nature. Therefore, there is need for an automatic tool for pipelines damage detection. There exist several methods which are based on image processing, computer vision and machine learning. However, the correctness of these systems is largely dependent on the extraction of robust features techniques. The feature extraction step is the bottleneck for most of the classification algorithms. In recent years deep learning models have outperformed all other methods, these models automatically extract required features and makes classification task more accurate. The deep convolutional neural networks are very popular in solving classification problems. In this chapter convolutional neural networks are investigated for pipelines damage detection and classification. The contributions of this work include guidelines on design and implementation of convolutional neural networks, and directions are given for carrying out transfer learning to repurpose pretrained models. This chapter provides brief discussion on various methods of pipeline damage detection using convolutional neural networks by conducting survey of state-of-the-art-techniques. This chapter also provides insight into design and working of convolutional neural networks.

## **Author details**

Vidyadevi G. Biradar\*, H.C. Nagaraj, S.G. Mohan and Piyush Kumar Pareek NITTE Meenakshi Institute of Technology, Bengaluru, India

\*Address all correspondence to: vidyadevi.g.biradar@nmit.ac.in

© 2022 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## **References**

[1] Al-Janabi YT. An overview of corrosion in oil and gas industry: Upstream, midstream, and downstream sectors. Corrosion Inhibitors in the Oil and Gas Industry. 2020:1-39

[2] Mahmoodzadeh Z et al. Conditionbased maintenance with reinforcement learning for dry gas pipeline subject to internal corrosion. Sensors. 2020;**20**(19):5708

[3] Jafari R et al. Deep learning for pipeline damage detection: An overview of the concepts and a survey of the state-ofthe-art. In: 2020 IEEE 10th International Conference on Intelligent Systems (IS). Bucharest, Romania: IEEE; 2020

[4] Hashim AS, Grămescu B, Niţu C. Pipe cracks detection methods–a review. In: International Conference of Mechatronics and Cyber-Mixmechatronics. Cham: Springer; 2018

[5] Ashraful AM et al. An algorithm to detect and identify defects of industrial pipes using image processing. In: The 8th International Conference on Software, Knowledge, Information Management and Applications (SKIMA 2014). United Kingdom: IEEE; 2014

[6] Sinha SK, Karray F, Fieguth PW. Underground pipe cracks classification using image analysis and neuro-fuzzy algorithm. In: Proceedings of the 1999 IEEE International Symposium on Intelligent Control Intelligent Systems and Semiotics (Cat. No. 99CH37014). United Kingdom. 1999

[7] Huang L et al. Research on pipe crack detection based on image processing algorithm. International Journal of Wireless and Mobile Computing. 2021;**20**(4):328-335

[8] Iyer S, Sinha SK. A robust approach for automatic detection and segmentation of cracks in underground pipeline images. Image and Vision Computing. 2005;**23**(10):921-933

[9] Datta S, Sarkar S. A review on different pipeline fault detection methods. Journal of Loss Prevention in the Process Industries. 2016;**41**:97-106

[10] Khan SM, Haider SA, Unwala I. A deep learning based classifier for crack detection with robots in underground pipes. In: IEEE 17th International Conference on Smart Communities: Improving Quality of Life Using ICT. Taylor & Francis and Routledge; 2020. pp. 78-81. DOI: 10.1109/ HONET50430.2020.9322665

[11] Wu J. Introduction to convolutional neural networks. National Key Lab for Novel Software Technology. 2017;**5**(23):495

[12] Dubey SR, Singh SK, Chaudhuri BB. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing. 2022

[13] Melching D et al. Explainable machine learning for precise fatigue crack tip detection. Scientific Reports. 2022;**12**(1):1-14

[14] Yang D et al. Deep learning-based steel pipe weld defect detection. Applied Artificial Intelligence. 2021;**35**(15):1237-1249

[15] Piltan F, Kim J-M. Crack localization of pipelines using machine learning and fuzzy digital twin. In: 2021 International Conference on Information and Communication Technology Convergence (ICTC). Springeropen, IEEE; 2021

[16] Shi J et al. Automated underwater pipeline damage detection using neural nets. In: ICRA 2019 Workshop on Underwater Robotics Perception. 2019

[17] Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. Journal of Big Data. 2019;**6**(1):1-48

[18] Zhuang F et al. A comprehensive survey on transfer learning. Proceedings of the IEEE. 2020;**109**(1):43-76

[19] Sahinbas K, Catak FO. Transfer learning-based convolutional neural network for COVID-19 detection with X-ray images. In: Data Science for COVID-19. Elsevier, Academic Press; 2021. pp. 451-466

[20] Hinz T et al. Speeding up the hyperparameter optimization of deep convolutional neural networks. International Journal of Computational Intelligence and Applications. 2018;**17**(02):1850008

[21] Aszemi NM, Dominic PDD. Hyperparameter optimization in convolutional neural network using genetic algorithms. International Journal of Advanced Computer Science and Applications. 2019;**10**:6

[22] Han X et al. Pre-trained models: Past, present and future. AI Open. 2021;**2**:225-250

## **Chapter 5**
