**2. Related work**

Automated segmentation of cellular membranes has received much attention lately. Various machine learning-based approaches for segmenting the cell

#### *Deep Learning-Based Segmentation of Cellular Membranes in Colorectal Immunohistochemical… DOI: http://dx.doi.org/10.5772/intechopen.108589*

membranes in IHC images have been proposed [10–14]. Chang et al. [10] employed the colour channel to extract the morphology, texture and intensity features that then have been utilised to train the support vector machine (SVM) classifier. Tuominen et al. [12] employed conventional machine learning techniques and ImageJ in preparing their ImmunoMembrane web application. Kuo et al. [14] employed a watershed algorithm for nucleus segmentation. But all these algorithms are not up to the performance of deep learning approaches. Therefore, there is a demanding need to employ deep learning approaches, which are advanced machine learning approaches for solving the current issues related to membrane segmentation. Several researchers [15–17] investigated deep learning approaches for cell membranes segmentation. Khameneh et al. [15] employed the SVM classifier to specify ROI and then the deep U-net model for segmenting membrane regions. Saha et al. [16] proposed a long short term memory (LSTM) architecture to detect cell membrane and nucleus. Gaur et al. [17] proposed deep CNN based on the active learning technique for membrane segmentation.

An atrous convolution layer is introduced in deepLab-v1 [18] network to widen the receptive field of view over the input feature maps without a decrease in spatial dimensions and an increase in the number of network parameters. And then, multiple parallel atrous convolutional layers with different dilution rates are proposed in the DeepLab v2 [19] network to segment objects at multiple scales. These layers are known as Atrous Spatial Pyramid Pooling (ASPP) model. After that, the ASPP model is improved in DeepLab v3 [20] to concatenate the image level features, a 1x1 convolution and three 3x3 atrous convolutions with different dilution rates. Encoder-Decoder structure and ASPP model are integrated into Deeplab v3+ [21] for applying the depth-wise separable convolution in both ASPP and Decoder modules. The encoder module reduces the spatial dimensions of the feature maps through the repeated application for the convolution and pooling layers, whilst the decoder module gradually recovers the spatial dimensions by using de-convolution and upsampling layers. Then, skip connections are introduced between the encoder and decoder modules to have sharper segmentation results.

In this work, we propose a trainable CNN based-detector to incorporate encoder, ASPP and decoder. We leveraged the identity mappings proposed by He et al. (2016a, b) in their Residual architectures. So, the encoder part employs the pre-trained ResNet-50 network [22] trained on the ImageNet [23] dataset as the feature extractor. Hence, through the proposed detector, we can overcome the challenges of 1) training the entire network from scratch, 2) the data scarcity problem and its consequences, and 3) over-fitting and poor generation of features. The main contributions of our proposed framework are as follows:

