**Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes**

Luciano Cássio Lulio, Mário Luiz Tronco, Arthur José Vieira Porto, Carlos Roberto Valêncio and Rogéria Cristiane Gratão de Souza

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/51862

**1. Introduction**

[8] L.Guo, Y.M.HouandX.M.Lun, An unsupervised color image segmentation algorithm

[9] Qiyao Yu and David A. Clausi, "IRGS: Image Segmentation Using Edge Penalties and Region Merging, "*IEEE Transaction on Pattern Analysis and Machine Intelligence*

[10] Sucheta Panda and P. K. Nanda, Constrained compound Markov Random field mod‐ el with graduated penalty function for color Image Segmentation, *IEEE International conference on Control, Robotics and Cybernetics(ICCRC-2011)*, pp.VI-126-VI-132, 2011.

[11] C. Cheng, A. Koschan, C. H. Chen, D. L. Page and M. A. Abidi, Outdoor Scene Image Segmentation Based on Background Recognition and Perceptual Organization. *IEEE*

[12] A. K. Mishra, P. W. Fieguth, D. A. Clausi. Decoupled Active Contour (DAC) for Boundary Detection. *IEEE Transactions on Pattern Analysis And Machine Intelligence,* 

[13] G. Scarpa, R. Gaetano, M. Haindl and J. Zerubia, Hierarchical multiple Markov chain model for unsupervised texture segmentation *IEEE Transactions on Image Processing*

[14] N. Chow, J. Mallet-Paret and J. A. Yorke, Finding zeros of maps: homotopy methods that are constructive with probability one, *Math.computation*, vol.32, no.143, pp.

[15] V. L. Stonick and S. T. Alexander, A Relationship between recursive least square up‐ date and homotopy continuation methods, *IEEE Trans. Signal Processing*, vol.39, no.2,

[16] P. K. Nanda, K. Sunil Kumar, Sameer Gokhale and U. B. Desai, A multiresolution ap‐ proach to color image restoration and parameter estimation using homotopy contin‐ uation method, *Proc. IEEE Int. Conf. on Image Proc.*, Washington, D. C, USA, Oct.

[17] P. K. Nanda, K. Sunil Kumar, Sameer Gokhale and U. B. Desai, A multiresolution ap‐ proach to color image restoration and parameter estimation using homotopy contin‐ uation method, *Proc. IEEE Int. Conf. on Image Processing*, Washington, D.C, USA, vol

*Transactions on Image Processing,* vol.21, no.3, pp.1007-1019, March2012.

fi

*cial Intelligence* vol.21, pp.

based on context information *Pattern Recognition and Arti*

*PAMI*, vol. 30, no. 12, pp.2126-2139, 2008.

vol.33, no. 5, pp.917-930, 2011.

vol.18, pp.1830-1843, 2009.

887-899, 1978.

pp. 530-532, 1991.

2, 45-48 Oct.1995.

1995.

82-87, 2008.

102 Advances in Image Segmentation

In this approach, cognitive and statistical classifiers were implemented in order to verify the estimated and chosen regions on unstructured environments images. As inspection of crops for natural scenes demands and requires complex analysis of image processing and segmen‐ tation algorithms, since these computational methods evaluate and predict environment physical characteristics, such as color elements, complex objects composition, shadows, brightness and inhomogeneous region colors for texture, JSEG segmentation algorithm was approached to segment these ones, and ANN and Bayes recognition models to classify im‐ ages into predetermined classes (e.g. fruits, plants and general crops). The intended ap‐ proach to segment classification deploys a customized MLP topology to classify and characterize the segments, which deals with a supervised learning by error correction – propagation of pattern inputs with changes in synaptic weights in a cyclic processing, with accurate recognition as well as easy parameter adjustment, as an enhancement of iRPROP algorithm (*improved resilient back-propagation*) (Igel and Hüsken, 2003) derived from *Backpropagation* algorithm, which has a faster identification mapping process, that verifies what region maps have similar matches through the explored environment. Bayes statistical mod‐ els had the addiction of process variable as set parameters of predictive error correction.

To carry through this task, a feature vector is necessary for color channels histograms (layers of primary color in a digital image with a counting graph that measures how many pixels are at each level between black and white). After training process, the mean squared error

(MSE), denotes the best results achieved by segment classification to create the image-class map, which represents the segments into distinct feature vectors. Furthermore, a language dictionary is used for the expansion on main results, which semantic regions and negation detection are applied as data mining process with cognitive and statistical classifiers.

**2.1. Segmentation algorithm evaluation**

adopted, presenting the overall distortion *D*:

The parameters: *ci* is the centroid of cluster *Ci*

longing to class *i*, where *i=1,...,C*, and *mi*

The J-value is as follows:

unsupervised method.

And it is derived for:

ceptual weight for pixel *n*. *Di*

Natural scenes present a 24-bit chromatic resolution color image, which is coarsely quan‐ tized preserving its major quality. The main idea for a good segmentation criterion is to ex‐ tract representative colors differentiating neighboring regions in the acquired image, as an

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

Therewith, the color quantization using peer group filtering (Deng *et al.*, 2001) is applied through perceptual weighting on individual pixels, to smooth the image and remove the ex‐ isting noise. Then, new values indicating the smoothness of the local areas are obtained, and a weight is assigned to each pixel, prioritizing textured areas to smooth areas. These areas are identified with a quantization vector to the pixel colors, based on General Lloyd Algo‐ rithm (GLA) (Gersho and Gray, 1999), which the perceptually uniform L\*u\*v color space is

<sup>2</sup> () () () *i ii i in*

()() ( ) ( ) *i i vnxn <sup>c</sup> xn C v n* <sup>å</sup> = ®Î

is the total distortion for *Ci*

Hart, 1970), as the quantization parameter needed for spatial distribution.

obtained in the quantization. Then *Z* is classified into *C* classes, *Zi*

ors are replaced by their corresponding color class labels, creating a class-map.

With the centroid value, as denoted by Equation (2) - after the vector quantization and merged clusters, pixels with the same color have two or more clusters, affected by GLA global distortion. For merging close clusters with minimum distance between preset thresh‐ olds for two centroids, an agglomerative clustering algorithm is performed on *ci* (Duda and

After clustering merging for color quantization, a label is assigned for each quantized color, representing a color class for image pixels quantized to the same color. The image pixel col‐

In order to calculate the J-value, *Z* is defined as the set of all points of quantized image, then *z = (x, y)* with *z ∈ Z* and being *m* the average in all *Z* elements. *C* is the number of classes

> 1 *<sup>i</sup> z Z <sup>i</sup> m z <sup>N</sup>* <sup>Î</sup>

*D D vn xn c xn C* = = -®Î å åå (1)

.

are the element averages in *Zi*

<sup>å</sup> (2)

, *x(n)* and *v(n)* are the color vector and the per‐

.

= å (3)

are the elements of *Z* be‐

http://dx.doi.org/10.5772/51862

105

## **2. JSEG image segmentation**

Color images with homogeneous regions are segmented with an algorithm to generate clus‐ ters in the color space/class (different measures classes in spectral distribution, with distinct intensity of visible electro-magnetic radiation at many discrete wavelengths) (Deng *et* al, 1999a). One way to segment images with textures is to consider the spatial arrangement of pixels using a region-growing technique whereby a homogeneity mode is defined with pix‐ els grouped in the segmented region. Furthermore, in order to segment texture images one must consider different scales of images.

The JSEG algorithm segments images of natural scenes properly, without manual parameter adjustment for each image and simplifies texture and color. Segmentation with this algo‐ rithm passes through three stages, namely color space quantization (number reduction proc‐ ess of distinct colors in a given image), hit rate regions and similar color regions merging.

In the first stage, the color space is quantized with little perceptual degradation by using the quantization algorithm (Deng *et* al, 1999b) with minimum coloring. Each color is associated with a class. The original image pixels are replaced by classes to form the class maps in the next stage. Before performing the hit rate regions, the J-image - a class map for each win‐ dowed color region, whose positive and negative values represent the edges and textures of the processing image - must be created with pixel values used as a similarity algorithm for the hit rate region. These values are called "J-values" and are calculated from a window placed on the quantized image, where the J-value belongs.

**Figure 1.** JSEG image segmentation steps.

## **2.1. Segmentation algorithm evaluation**

(MSE), denotes the best results achieved by segment classification to create the image-class map, which represents the segments into distinct feature vectors. Furthermore, a language dictionary is used for the expansion on main results, which semantic regions and negation

Color images with homogeneous regions are segmented with an algorithm to generate clus‐ ters in the color space/class (different measures classes in spectral distribution, with distinct intensity of visible electro-magnetic radiation at many discrete wavelengths) (Deng *et* al, 1999a). One way to segment images with textures is to consider the spatial arrangement of pixels using a region-growing technique whereby a homogeneity mode is defined with pix‐ els grouped in the segmented region. Furthermore, in order to segment texture images one

The JSEG algorithm segments images of natural scenes properly, without manual parameter adjustment for each image and simplifies texture and color. Segmentation with this algo‐ rithm passes through three stages, namely color space quantization (number reduction proc‐ ess of distinct colors in a given image), hit rate regions and similar color regions merging.

In the first stage, the color space is quantized with little perceptual degradation by using the quantization algorithm (Deng *et* al, 1999b) with minimum coloring. Each color is associated with a class. The original image pixels are replaced by classes to form the class maps in the next stage. Before performing the hit rate regions, the J-image - a class map for each win‐ dowed color region, whose positive and negative values represent the edges and textures of the processing image - must be created with pixel values used as a similarity algorithm for the hit rate region. These values are called "J-values" and are calculated from a window

detection are applied as data mining process with cognitive and statistical classifiers.

**2. JSEG image segmentation**

104 Advances in Image Segmentation

must consider different scales of images.

**Figure 1.** JSEG image segmentation steps.

placed on the quantized image, where the J-value belongs.

Natural scenes present a 24-bit chromatic resolution color image, which is coarsely quan‐ tized preserving its major quality. The main idea for a good segmentation criterion is to ex‐ tract representative colors differentiating neighboring regions in the acquired image, as an unsupervised method.

Therewith, the color quantization using peer group filtering (Deng *et al.*, 2001) is applied through perceptual weighting on individual pixels, to smooth the image and remove the ex‐ isting noise. Then, new values indicating the smoothness of the local areas are obtained, and a weight is assigned to each pixel, prioritizing textured areas to smooth areas. These areas are identified with a quantization vector to the pixel colors, based on General Lloyd Algo‐ rithm (GLA) (Gersho and Gray, 1999), which the perceptually uniform L\*u\*v color space is adopted, presenting the overall distortion *D*:

$$D = \underset{i}{\text{d}}\ D\_{i} = \underset{i}{\text{d}}\ \text{d} \ w(n) \left\| x(n) \cdot c\_{i} \right\|^{2} \otimes \ x(n) \stackrel{\text{\!\! }}{\text{ }} C\_{i} \tag{1}$$

And it is derived for:

$$\mathbf{c}\_{i} = \frac{\triangleq \; \mathsf{d} \; v(n)\mathbf{x}(n)}{\triangleq \; \mathsf{d} \; v(n)} \circledast \; \mathbf{x}(n) \, \mathsf{l} \; \mathsf{C}\_{i} \tag{2}$$

The parameters: *ci* is the centroid of cluster *Ci* , *x(n)* and *v(n)* are the color vector and the per‐ ceptual weight for pixel *n*. *Di* is the total distortion for *Ci* .

With the centroid value, as denoted by Equation (2) - after the vector quantization and merged clusters, pixels with the same color have two or more clusters, affected by GLA global distortion. For merging close clusters with minimum distance between preset thresh‐ olds for two centroids, an agglomerative clustering algorithm is performed on *ci* (Duda and Hart, 1970), as the quantization parameter needed for spatial distribution.

After clustering merging for color quantization, a label is assigned for each quantized color, representing a color class for image pixels quantized to the same color. The image pixel col‐ ors are replaced by their corresponding color class labels, creating a class-map.

In order to calculate the J-value, *Z* is defined as the set of all points of quantized image, then *z = (x, y)* with *z ∈ Z* and being *m* the average in all *Z* elements. *C* is the number of classes obtained in the quantization. Then *Z* is classified into *C* classes, *Zi* are the elements of *Z* be‐ longing to class *i*, where *i=1,...,C*, and *mi* are the element averages in *Zi* .

$$m\_i = \frac{1}{N\_i} \underset{z \parallel Z}{\text{a}} \ z$$

The J-value is as follows:

$$J = \frac{S\_B}{S\_W} = \frac{(S\_T - S\_W)}{S\_W} \tag{4}$$

Below, the spatial segmentation algorithm is structured in flow steps.

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

107

**Figure 2.** Sequence for spatial segmentation algorithm.

textured side).

**3. Image processing (spatial distribution and objects quantification)**

The sequential images evince not only the color quantization (spatial distributions forming a map of classes), but also the space segmentation (J-image representing edges and regions of

Several window sizes are used by J-values: the largest detects the region boundaries by re‐ ferring to texture parameters; the lowest detects changes in color and/or intensity of light. Each window size is associated with a scale image analysis. The concept of J-image, together with different scales, allows the segmentation of regions by referring to texture parameters.

where:

$$\mathcal{S}\_{\Gamma} = \underset{z \stackrel{\text{d}}{\rightleftarrows} \mathbb{Z}}{\underset{z \stackrel{\text{d}}{\rightleftarrows} \mathbb{Z}}{\left\| z \stackrel{\text{d}}{\right\|}}} \left\| z \stackrel{\text{d}}{\right\|}\tag{5}$$

$$S\_W = \stackrel{\circ}{\mathbf{d}}\_{i=1}^{\mathbb{C}} \mathbf{A}\_i \left\| z \cdot m\_i \right\|^2 \tag{6}$$

The parameter *ST* represents the sum of quantized image points within the average in all Z elements. Thereby, the relation between *SB* and *SW*, denotes the measures of distances of this class relation, for arbitrary nonlinear class distributions. *J* for higher values indicates an in‐ creasing distance between the classes and points for each other, considering images with ho‐ mogeneous color regions. The distance and consequently, the *J* value, decrease for images with uniformly color classes.

Each segmented region could be recalculated, instead of the entire class-map, with new pa‐ rameters adjustment for *J* ¯ average. *JK* represents *J* calculated over region *k*, *Mk* is the number of points in region *k*, *N* is the total number of points in the class-map, with all regions in class-map summation.

$$\tilde{J} = \frac{1}{N} \mathbf{\hat{a}}\_k \mathbf{M}\_k J\_k \tag{7}$$

For a fixed number of regions, a criterion for *J* ¯ is intended for lower values.

#### **2.2. Spatial segmentation technique**

The global minimization of *J* ¯ is not practical, if not applied to a local area of the class-map. Therefore, the idea of *J-image* is the generation of a gray-scale image whose pixel values are the *J* values calculated over local windows centered on these pixels. With a higher value for J-image, the pixel should be near region boundaries.

Expected local windows dimensions determines the size of image regions, for intensity and color edges in smaller sizes, and the opposite occurs detecting texture boundaries.

Using a region-growing method to segment the image, this one is considered initially as one single region. The algorithm for spatial segmentation starts segment all the regions in the image at an initial large scale until the minimum specified scale is reached. This final scale is settled manually for the appropriate image size. The initial scale 1 corresponds to 64x64 im‐ age size, scale 2 to 128x128 image size, scale 3 to 256x256 image size, with due proportion for increasing scales and the double image size.

Below, the spatial segmentation algorithm is structured in flow steps.

( ) *<sup>B</sup> T W W W <sup>S</sup> S S <sup>J</sup> S S*

*<sup>T</sup> z Z S zm* Î

1 *C W i i zZ S zm* = Î

2

2

The parameter *ST* represents the sum of quantized image points within the average in all Z elements. Thereby, the relation between *SB* and *SW*, denotes the measures of distances of this class relation, for arbitrary nonlinear class distributions. *J* for higher values indicates an in‐ creasing distance between the classes and points for each other, considering images with ho‐ mogeneous color regions. The distance and consequently, the *J* value, decrease for images

Each segmented region could be recalculated, instead of the entire class-map, with new pa‐

of points in region *k*, *N* is the total number of points in the class-map, with all regions in

*k k <sup>k</sup>*

Therefore, the idea of *J-image* is the generation of a gray-scale image whose pixel values are the *J* values calculated over local windows centered on these pixels. With a higher value for

Expected local windows dimensions determines the size of image regions, for intensity and

Using a region-growing method to segment the image, this one is considered initially as one single region. The algorithm for spatial segmentation starts segment all the regions in the image at an initial large scale until the minimum specified scale is reached. This final scale is settled manually for the appropriate image size. The initial scale 1 corresponds to 64x64 im‐ age size, scale 2 to 128x128 image size, scale 3 to 256x256 image size, with due proportion for

color edges in smaller sizes, and the opposite occurs detecting texture boundaries.

1

where:

106 Advances in Image Segmentation

with uniformly color classes.

For a fixed number of regions, a criterion for *J*

J-image, the pixel should be near region boundaries.

increasing scales and the double image size.

**2.2. Spatial segmentation technique**

The global minimization of *J*

rameters adjustment for *J*

class-map summation.


= - å (5)

= - å å (6)

¯ average. *JK* represents *J* calculated over region *k*, *Mk* is the number

¯ is intended for lower values.

¯ is not practical, if not applied to a local area of the class-map.

*J MJ <sup>N</sup>* <sup>=</sup> <sup>å</sup> (7)

**Figure 2.** Sequence for spatial segmentation algorithm.

## **3. Image processing (spatial distribution and objects quantification)**

The sequential images evince not only the color quantization (spatial distributions forming a map of classes), but also the space segmentation (J-image representing edges and regions of textured side).

Several window sizes are used by J-values: the largest detects the region boundaries by re‐ ferring to texture parameters; the lowest detects changes in color and/or intensity of light. Each window size is associated with a scale image analysis. The concept of J-image, together with different scales, allows the segmentation of regions by referring to texture parameters.

Regions with the lowest values of J-image are called valleys. The lowest values are applied with a heuristic algorithm. Thus, it is possible to determine the starting point of efficient growth, which depends on the addition of similar valleys. The algorithm ends when there are spare pixels to be added to those regions.

Blue) components. Also, the network with less MSE in the neurons to color space proportion

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

109

Derived from back-propagation, the iRPROP algorithm (improved resilient back-propaga‐ tion) (Lulio, 2010) is both fast and accurate, with easy parameter adjustment. It features an Octave (Eaton, 2006) module which was adopted for the purposes of this work and it is clas‐ sified with HSV (H – hue, S – saturation, V – value) color space channels histograms of 256 categories (32, 64,128 and 256 neurons in a hidden layer training for each color space chan‐ nel: H, HS, and HSV). The output layer has three neurons, each of them having a predeter‐

is used to classify the entities.

mined class.

**Figure 5.** MSE 50% validation tests for RGB.

**Figure 4.** ANN schematic topology for fruits with three classes.

**Figure 3.** a) Original images; (b) Color quantization (map of classes); (c) J-image representing edges and regions of textured side (Spatial distributions).

It was observed that the oranges represent the largest number of image pixels, given its characteristics of high contrast with other objects on the scene.

Fig. 3, above, shows three types of scenes in orchards. The first identifies the largest part of the tree. In this category, the quantization threshold was adjusted to higher values for the fusion of regions with same color tone between branches, leaves and ground would be avoided. The second scene denotes the regions' set details in orchards, excluding darker re‐ gions. Not only irregularities of each leaf are segmented, as well as abnormalities of color tones in fruit itself, allowing later analysis of disease characteristics. The third category iden‐ tifies most of the trees, but with higher incidence of top and bottom regions.

## **4. Artificial Neural Networks (ANN) – MLP customized algorithm**

It is fundamental that an ANN-based classification method associated with a statistical pat‐ tern recognition be used. *Multi-Layer Perceptron* (MLP) (Haykin, 1999; Haykin, 2008) is suita‐ ble for default ANN topology to be implemented through a customized *back-propagation* algorithm for complex patterns (Costa and Cesar Junior, 2001).

The most appropriate segment and topology classifications are those using vectors extracted from HSV color space (Hue, Saturation, Value), matching RGB color space (Red, Green, Blue) components. Also, the network with less MSE in the neurons to color space proportion is used to classify the entities.

**Figure 4.** ANN schematic topology for fruits with three classes.

Regions with the lowest values of J-image are called valleys. The lowest values are applied with a heuristic algorithm. Thus, it is possible to determine the starting point of efficient growth, which depends on the addition of similar valleys. The algorithm ends when there

**Figure 3.** a) Original images; (b) Color quantization (map of classes); (c) J-image representing edges and regions of

It was observed that the oranges represent the largest number of image pixels, given its

Fig. 3, above, shows three types of scenes in orchards. The first identifies the largest part of the tree. In this category, the quantization threshold was adjusted to higher values for the fusion of regions with same color tone between branches, leaves and ground would be avoided. The second scene denotes the regions' set details in orchards, excluding darker re‐ gions. Not only irregularities of each leaf are segmented, as well as abnormalities of color tones in fruit itself, allowing later analysis of disease characteristics. The third category iden‐

characteristics of high contrast with other objects on the scene.

algorithm for complex patterns (Costa and Cesar Junior, 2001).

tifies most of the trees, but with higher incidence of top and bottom regions.

**4. Artificial Neural Networks (ANN) – MLP customized algorithm**

It is fundamental that an ANN-based classification method associated with a statistical pat‐ tern recognition be used. *Multi-Layer Perceptron* (MLP) (Haykin, 1999; Haykin, 2008) is suita‐ ble for default ANN topology to be implemented through a customized *back-propagation*

The most appropriate segment and topology classifications are those using vectors extracted from HSV color space (Hue, Saturation, Value), matching RGB color space (Red, Green,

are spare pixels to be added to those regions.

108 Advances in Image Segmentation

textured side (Spatial distributions).

Derived from back-propagation, the iRPROP algorithm (improved resilient back-propaga‐ tion) (Lulio, 2010) is both fast and accurate, with easy parameter adjustment. It features an Octave (Eaton, 2006) module which was adopted for the purposes of this work and it is clas‐ sified with HSV (H – hue, S – saturation, V – value) color space channels histograms of 256 categories (32, 64,128 and 256 neurons in a hidden layer training for each color space chan‐ nel: H, HS, and HSV). The output layer has three neurons, each of them having a predeter‐ mined class.

**Figure 5.** MSE 50% validation tests for RGB.

The charts below (Figures 5, 6, 7, 8) denote the ratio of mean square error (MSE) and amount of times to obtain the best performance index during the validation data towards the train‐ ing and test sets.

All ANN-based topologies are trained with a threshold lower than 0.0001 mean squared er‐ rors (MSE), the synaptic neurons weights are initiated with random values and the other al‐ gorithm parameters were set with Fast Artificial Neural Network (FANN) library (Nissen, 2006) for Matlab (Mathworks Inc.) platform, and also its Neural Network toolbox. The most appropriate segment and topology classifications are those using vectors extracted from HSV color space. Also, a network with less MSE in the H-64 was used so as to classify the planting area; for class navigable area (soil), HSV-256 was chosen; as for the class sky, the HS-32.

**Figure 7.** MSE 50% validation tests for HSV.

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

111

**Figure 8.** MSE 100% validation tests for HSV.

**Figure 6.** MSE 100% validation tests for RGB.

Figures 9 and 10 denote the regression for target-outputs of ANN classifier, for RGB and HSV classes. The higher the concentration of data at the intersection of bias and Y = T (equal to the output sampling period), the lower the linear regression of data is classified, based on confusion matrices for each set of dimensions.

The response times are given for combinations of training, testing, validation and all data sets.

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes http://dx.doi.org/10.5772/51862 111

**Figure 7.** MSE 50% validation tests for HSV.

The charts below (Figures 5, 6, 7, 8) denote the ratio of mean square error (MSE) and amount of times to obtain the best performance index during the validation data towards the train‐

All ANN-based topologies are trained with a threshold lower than 0.0001 mean squared er‐ rors (MSE), the synaptic neurons weights are initiated with random values and the other al‐ gorithm parameters were set with Fast Artificial Neural Network (FANN) library (Nissen, 2006) for Matlab (Mathworks Inc.) platform, and also its Neural Network toolbox. The most appropriate segment and topology classifications are those using vectors extracted from HSV color space. Also, a network with less MSE in the H-64 was used so as to classify the planting area; for class navigable area (soil), HSV-256 was chosen; as for the class sky, the

Figures 9 and 10 denote the regression for target-outputs of ANN classifier, for RGB and HSV classes. The higher the concentration of data at the intersection of bias and Y = T (equal to the output sampling period), the lower the linear regression of data is classified, based on

The response times are given for combinations of training, testing, validation and all data

ing and test sets.

110 Advances in Image Segmentation

HS-32.

sets.

**Figure 6.** MSE 100% validation tests for RGB.

confusion matrices for each set of dimensions.

**Figure 8.** MSE 100% validation tests for HSV.

the components (features) represent the key information contained in data, reducing the number of dimensions. Therefore, RGB space color is used to compare the total number of dimensions in feature vectors with HSV. With a smaller dimension of iterations, HSV is

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

113

chosen as the default space color in most applications (Grasso and Recce, 1996).

**Figure 10.** MSE 50% (left) and 100% (right) validation tests for RGB.

notes the solution for *p(Ci*

*Bayes Theorem* introduces a modified mathematical equation for the Probability Density Function (PDF), which estimates the training set in a conditional statistics. Equation (8) de‐

and *y* is a *n*-dimensional feature vector. *Naive Bayes* implies independence for vector fea‐

*|y)* relating the PDF to conditional class *i* (classes in natural scene),

**Figure 9.** MSE 50% (left) and 100% (right) validation tests for RGB.

## **5. Statistical pattern recognition**

Statistical methods are employed as a combination of results with ANN, showing how accu‐ racy in non-linear features vectors can be best applied in a MLP algorithm with a statistical improvement, which processing speed is essentially important, for pattern classification. *Bayes Theorem* and *Naive Bayes* (Comaniciu and Meer, 1997) both use a technique for itera‐ tions inspection, namely MCA (*Main Component Analysis*), which uses a linear transforma‐ tion that minimizes co-variance while it maximizes variance. Features found through this transformation are totally uncorrelated, so the redundancy between them is avoided. Thus, the components (features) represent the key information contained in data, reducing the number of dimensions. Therefore, RGB space color is used to compare the total number of dimensions in feature vectors with HSV. With a smaller dimension of iterations, HSV is chosen as the default space color in most applications (Grasso and Recce, 1996).

**Figure 10.** MSE 50% (left) and 100% (right) validation tests for RGB.

**Figure 9.** MSE 50% (left) and 100% (right) validation tests for RGB.

Statistical methods are employed as a combination of results with ANN, showing how accu‐ racy in non-linear features vectors can be best applied in a MLP algorithm with a statistical improvement, which processing speed is essentially important, for pattern classification. *Bayes Theorem* and *Naive Bayes* (Comaniciu and Meer, 1997) both use a technique for itera‐ tions inspection, namely MCA (*Main Component Analysis*), which uses a linear transforma‐ tion that minimizes co-variance while it maximizes variance. Features found through this transformation are totally uncorrelated, so the redundancy between them is avoided. Thus,

**5. Statistical pattern recognition**

112 Advances in Image Segmentation

*Bayes Theorem* introduces a modified mathematical equation for the Probability Density Function (PDF), which estimates the training set in a conditional statistics. Equation (8) de‐ notes the solution for *p(Ci |y)* relating the PDF to conditional class *i* (classes in natural scene), and *y* is a *n*-dimensional feature vector. *Naive Bayes* implies independence for vector fea‐ tures, what means that each class assumes the conditional parameter for the PDF, following Equation (9) (Morimoto *et* al, 2000).

$$P(\mathcal{C}\_i \mid y) = \frac{p(y \mid \mathcal{C}\_i)P(\mathcal{C}\_i)}{\mathsf{A}\_{/\pi \mid 1}^K p(y \mid \mathcal{C}\_j)P(\mathcal{C}\_j)} \tag{8}$$

$$P(y \mid \mathcal{C}\_i) = \bigotimes\_{j=1}^{\mathbb{N}} p(y\_j \mid \mathcal{C}\_i) \tag{9}$$

**Figure 12.** Mixture parameters for estimated set (oranges RGB - left, oranges HSV - right).

The classes maps are processed, as the representation by the area filling (*floodfill*) brings only solid regions which are quantified. Initially, a conversion is performed on gray level image in order to threshold regions that are outlined. Then, to determine the labels of the elements connected, it is necessary to exclude objects which are greater than 200 to 300 pixels, de‐ pending on the focal length. Thus, it is necessary to identify each element smaller than this threshold, and calculate the properties of these objects, such as area, centroid, and the boun‐ dary region. As a result, the objects that present areas near the circular geometry will be la‐

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

115

To determine the metrics and the definition of objects of orange crop, the graph-based seg‐ mentation (Gonzalez and Woods, 2007) was applied. This technique provides the adjacency relation between the binary values of the pixels, and their respective positions, highlighting

In first case, areas corresponding to small regions, as fruits partially hidden (oranges) with equivalent texture and color properties to leaves are excluded. Then, estimated elements are fully grouped, when overlap the representative segments, which denote an orange fruit. Lastly, the grouping is applied for regions which detect two or more representative seg‐

As the best classification results, related to second approach were through Bayes in HSV col‐ or space, only the maps of class from these classifiers will be presented to localization and

Then, for the RGB and HSV cases are presented, through Figures 13 to 21, the images in their respective maps of class, the pre-processing for thresholding with areas smaller than

**6. Objects quantification (post-processing)**

belled and quantified as fruits.

the local geometric properties of the image.

ments, denoting another orange fruit.

quantification of objects, compared to RGB case.

In Fig. 11, for the location of fruits in the RGB case, the discrimination of the classes fruit, sky and leaves, twigs and branches, attends constant amounts proportional to the increasing of the training sets. This amount, for HSV case, is reduced for the fruit class, as the disper‐ sion of pixels is greater in this color space. In Fig. 12, in the RGB case, the best results were obtained using Bayes classifier, having smaller ratio estimation in relation to the number of components analyzed. In this color space, the estimation in the recognition of objects related to the fruits is given by the PDF of each dimension, correcting the current values by the hope of each area not matched to the respective class.

Also in Fig. 12, the recognition of the fruit to the HSV case presents balance in the results of the two classifiers, but with a compensation of the success rate, for lower margins of the esti‐ mation ratio to the Bayes classifier. This allows the correction of the next results by priori estimation approximating, in the PDF of each dimension.

It can be seen that, the ratio of the estimation must be lesser for the increasing of the dimen‐ sions number and its subsequent classification, in all cases.

**Figure 11.** Quantity of dimensions of each set (oranges RGB - left, oranges HSV - right).

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes http://dx.doi.org/10.5772/51862 115

**Figure 12.** Mixture parameters for estimated set (oranges RGB - left, oranges HSV - right).

## **6. Objects quantification (post-processing)**

tures, what means that each class assumes the conditional parameter for the PDF, following

*i i*

<sup>=</sup> *py C PC* <sup>=</sup> <sup>å</sup> (8)

= Õ (9)

*j j j*

1 (| )( ) ( |) (| )( )

1 (| ) ( | ) *<sup>n</sup> i ji <sup>j</sup> Py C py C* =

In Fig. 11, for the location of fruits in the RGB case, the discrimination of the classes fruit, sky and leaves, twigs and branches, attends constant amounts proportional to the increasing of the training sets. This amount, for HSV case, is reduced for the fruit class, as the disper‐ sion of pixels is greater in this color space. In Fig. 12, in the RGB case, the best results were obtained using Bayes classifier, having smaller ratio estimation in relation to the number of components analyzed. In this color space, the estimation in the recognition of objects related to the fruits is given by the PDF of each dimension, correcting the current values by the

Also in Fig. 12, the recognition of the fruit to the HSV case presents balance in the results of the two classifiers, but with a compensation of the success rate, for lower margins of the esti‐ mation ratio to the Bayes classifier. This allows the correction of the next results by priori

It can be seen that, the ratio of the estimation must be lesser for the increasing of the dimen‐

*py C PC PC y*

*i K*

Equation (9) (Morimoto *et* al, 2000).

114 Advances in Image Segmentation

hope of each area not matched to the respective class.

estimation approximating, in the PDF of each dimension.

sions number and its subsequent classification, in all cases.

**Figure 11.** Quantity of dimensions of each set (oranges RGB - left, oranges HSV - right).

The classes maps are processed, as the representation by the area filling (*floodfill*) brings only solid regions which are quantified. Initially, a conversion is performed on gray level image in order to threshold regions that are outlined. Then, to determine the labels of the elements connected, it is necessary to exclude objects which are greater than 200 to 300 pixels, de‐ pending on the focal length. Thus, it is necessary to identify each element smaller than this threshold, and calculate the properties of these objects, such as area, centroid, and the boun‐ dary region. As a result, the objects that present areas near the circular geometry will be la‐ belled and quantified as fruits.

To determine the metrics and the definition of objects of orange crop, the graph-based seg‐ mentation (Gonzalez and Woods, 2007) was applied. This technique provides the adjacency relation between the binary values of the pixels, and their respective positions, highlighting the local geometric properties of the image.

In first case, areas corresponding to small regions, as fruits partially hidden (oranges) with equivalent texture and color properties to leaves are excluded. Then, estimated elements are fully grouped, when overlap the representative segments, which denote an orange fruit. Lastly, the grouping is applied for regions which detect two or more representative seg‐ ments, denoting another orange fruit.

As the best classification results, related to second approach were through Bayes in HSV col‐ or space, only the maps of class from these classifiers will be presented to localization and quantification of objects, compared to RGB case.

Then, for the RGB and HSV cases are presented, through Figures 13 to 21, the images in their respective maps of class, the pre-processing for thresholding with areas smaller than 100 and greater than 300, the geometric approximation metrics for the detection of circular objects, the boundary regions with the centroid of each object, and finally the label associat‐ ed to the fruit.

**Figure 16.** Maps of RGB (left) and HSV (right) classes - scene 2.

**Figure 17.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 2.

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

http://dx.doi.org/10.5772/51862

117

**Figure 18.** Representation of area and centroid for fruit association in two cases - scene 2.

**Figure 13.** Maps of RGB (left) and HSV (right) classes - scene 1.

**Figure 14.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 1.

**Figure 15.** Representation of area and centroid for fruit association in two cases - scene 1.

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes http://dx.doi.org/10.5772/51862 117

**Figure 16.** Maps of RGB (left) and HSV (right) classes - scene 2.

100 and greater than 300, the geometric approximation metrics for the detection of circular objects, the boundary regions with the centroid of each object, and finally the label associat‐

ed to the fruit.

116 Advances in Image Segmentation

**Figure 13.** Maps of RGB (left) and HSV (right) classes - scene 1.

**Figure 14.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 1.

**Figure 15.** Representation of area and centroid for fruit association in two cases - scene 1.

**Figure 17.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 2.

**Figure 18.** Representation of area and centroid for fruit association in two cases - scene 2.

possible. As the data provided evince, this generated algorithms fulfills the expectations as far as segmenting is concerned, so that it sorts the appropriate classes (fruits; leaves and branches; sky). As a result, a modular strategy with Bayes statistical theorem can be an op‐

Cognitive and Statistical Pattern Recognition Applied in Color and Texture Segmentation for Natural Scenes

and Rogéria Cristiane Gratão de Souza2

1 Engineering School of Sao Carlos, University of Sao Paulo (EESC/USP), Sao Carlos, São

2 Statistical and Computing Science Department, State University of Sao Paulo (DCCE/

[1] Comaniciu, D., & Meer, P. (1997). Robust analysis of feature spaces: color image seg‐ mentation. In: Conference on Computer Vision and Pattern Recognition, IEEE Com‐

[2] Costa, L. F., & Cesar, Junior. R. M. (2001). Shape analysis and classification- Theory and Practice. 1. ed. Boca Raton, Florida, EUA: CRC Press LLC. 0-84933-493-4

[3] Deng, Y., Kennedy, C., Moore, M. S., & Manjunath, B. S. (1999a). Peer group filtering and perceptual color image quantization. Proceedings of the 1999 IEEE International

[4] Deng, Y., Manjunath, B. S., & Shin, H. (1999b). Color image segmentation. Confer‐ ence on Computer Vision and Pattern Recognition, IEEE Computer Society, , 2,

[5] Deng, Y., & Manjunath, B. S. (2001). Unsupervised segmentation of color-texture re‐ gions in images and videos. IEEE Transactions on Pattern Analysis and Machine In‐

[6] Duda, R. O., & Hart, P. E. (1970). Pattern Classification and Scene Analysis, John Wi‐

[8] Gersho, A., & Gray, R. M. (1992). Vector quantization and signal compression, Kluw‐

[7] Eaton, J. W., et al. (2006). Octave. Avaliable at: http://www.octave.org

, Arthur José Vieira Porto1

,

http://dx.doi.org/10.5772/51862

119

tion for the classification of segments applied with cognitive approach.

, Mário Luiz Tronco1

Symposium on Circuits and Systems, , 4, 21-25.

telligence (PAMI'01), , 23(8), 800-810.

ley & Sons, New York

er Academic, Norwell, MA

UNESP), São Jose do Rio Preto, São Paulo, Brazil

**Author details**

Paulo, Brazil

**References**

puter Society

446-451.

Luciano Cássio Lulio1

Carlos Roberto Valêncio2

**Figure 19.** Maps of RGB (left) and HSV (right) classes - scene 3.

**Figure 20.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 3.

**Figure 21.** Representation of area and centroid for fruit association in two cases - scene 3.

## **7. Conclusions**

This chapter presented merging techniques for segmentation and statistical classification of agricultural orange crops scenes, running multiple segmentation tests with JSEG algorithm possible. As the data provided evince, this generated algorithms fulfills the expectations as far as segmenting is concerned, so that it sorts the appropriate classes (fruits; leaves and branches; sky). As a result, a modular strategy with Bayes statistical theorem can be an op‐ tion for the classification of segments applied with cognitive approach.

## **Author details**

**Figure 19.** Maps of RGB (left) and HSV (right) classes - scene 3.

118 Advances in Image Segmentation

**Figure 20.** Metric near circular geometry threshold 1.0 for RGB (left) and HSV (right) - scene 3.

**Figure 21.** Representation of area and centroid for fruit association in two cases - scene 3.

This chapter presented merging techniques for segmentation and statistical classification of agricultural orange crops scenes, running multiple segmentation tests with JSEG algorithm

**7. Conclusions**

Luciano Cássio Lulio1 , Mário Luiz Tronco1 , Arthur José Vieira Porto1 , Carlos Roberto Valêncio2 and Rogéria Cristiane Gratão de Souza2

1 Engineering School of Sao Carlos, University of Sao Paulo (EESC/USP), Sao Carlos, São Paulo, Brazil

2 Statistical and Computing Science Department, State University of Sao Paulo (DCCE/ UNESP), São Jose do Rio Preto, São Paulo, Brazil

## **References**


[9] Gonzalez, R. C., & Woods, R. E. (2007). Digital Image Processing. 3 ed. New Jersey,

[10] Grasso, G. M., & Recce, M. (1996). Scene Analysis for an Orange Picking Robot,"In:

[11] Haykin, S. (1999). Neural networks: a comprehensive foundation. 2. ed. New Jersey,

[12] Haykin, S. (2008). Neural Networks and Learning Machines. 3.ed. McMaster Univer‐

[13] Igel, C., & Hüsken, M. (2003). Empirical evaluation of the improved Rprop learning

[14] Lulio, L. C. (2011). Computer vision techniques applied to natural scenes recognition and autonomous locomotion of agricultural mobile robots. São Carlos, 353 p. Disser‐ tation (Master of Science)- School of Engineering of São Carlos, University of São

[15] Morimoto, T., Takeuchi, T., Miyata, H., & Hashimoto, Y. (2000). Pattern recognition of fruit shapes based on the concept of chaos and neural networks,"Computers and

[16] Nissen, S., et al. (2006). Fann: fast artificial neural network library. Avaliable at:

International Congress for Computer Technology in Agri-culture

EUA: Prentice-Hall Inc

120 Advances in Image Segmentation

Paulo, São Carlos

http://leenissen.dk/fann/

EUA: Prentice-Hall. 0-13273-350-1

sity, Canada: Prentice-Hall. 0-13147-139-2

algorithm. Neurocomputing, , 50, 105-123.

Electronics in Agriculture, , 26, 171-186.

## *Edited by Pei-Gee Peter Ho*

The field of digital image segmentation is continually evolving. Most recently, the advanced segmentation methods such as Template Matching, Spatial and Temporal ARMA Processes, Mean Shift Iterative Algorithm, Constrained Compound Markov Random Field (CCMRF) model and Statistical Pattern Recognition (SPR) methods form the core of a modernization effort that resulted in the current text. This new edition of "Advanced Image Segmentation" is but a reflection of the significant progress that has been made in the field of image segmentation in just the past few years. The book presented chapters that highlight frontier works in image information processing.

Photo by zhazhin\_sergey / iStock

Advances in Image Segmentation

Advances in

Image Segmentation