2. Clustering criteria

Clustering aims to seek a partition of the data in the same homogenous clusters [7]. This homogeneity and finding the exact clustering are evaluated only by using criterion functions. One of the most popular techniques of the criterion function is the summing of squared-error (SSE) criterion and similar methods are used such as mean-square-error (MSE), normalized mean-square-error (NMSE), and so on. Sum of squared error criterion can be defined as:

#### Partitional Clustering http://dx.doi.org/10.5772/intechopen.75836 21

$$J(\Gamma, \mathbf{M}) = \sum\_{i=1}^{K} \sum\_{j=1}^{N} \gamma\_{ij} \left\| \mathbf{x}\_{j} - m\_{i} \right\|^{2} = \sum\_{i=1}^{K} \sum\_{j=1}^{N} \gamma\_{ij} \left( \mathbf{x}\_{j} - m\_{i} \right)^{T} \left( \mathbf{x}\_{j} - m\_{i} \right) \tag{1}$$

where Γ is a partition matrix of γij and defined as;

$$\gamma\_{ij} = \begin{cases} 1 & \text{if } \ x\_j \in cluster \text{ i} \\ 0 & \text{otherwise} \end{cases} \tag{2}$$

M is the cluster prototype or centroid matrix and mi defined as;

$$m\_i = \frac{1}{N\_i} \sum\_{i=1}^{N} \gamma\_{\vec{\eta}} \mathbf{x}\_{\vec{\eta}} \text{ is the sample mean for the } i^{\text{th}} \text{ number cluster corresponding to } N\_i \text{ objects.} $$

The partition minimizes the sum-square-error (SSE). When the SSE regarded is minimum, minimum variance partition will be achieved. As a result of this calculation, optimum cluster is determined.

#### 3. K-means algorithm

In contrast to hierarchical clustering methods, partitional clustering aims successive clusters using some iterative processes. Partitional clustering assigns a set of data points into k-clusters by using iterative processes. In these processes, n data are classified into k-clusters. The predef-

Figure 1. Clustering techniques: (a) data set; (b) partitional clustering; and (c) hierarchical clustering.

Figure 1 represents the hierarchical clustering and partitional clustering. In addition, hierarchical clustering, all sub-clusters defined in another sub-cluster shown in Figure 1. Figure 1a represents the raw data, Figure 1b shows the partitional clustering and Figure 1c represents the hierarchical clustering. In hierarchical clustering, raw data are firstly clustered in some subgroups (three-clustered shape). After that procedure, subgroups hierarchically defined in two green clusters. Last procedure includes all these clusters which are defined in the

This chapter starts with an introduction to clustering criteria and continues with K-Means algorithm, different fuzzy clustering techniques and genetic algorithm-based clustering.

Clustering aims to seek a partition of the data in the same homogenous clusters [7]. This homogeneity and finding the exact clustering are evaluated only by using criterion functions. One of the most popular techniques of the criterion function is the summing of squared-error (SSE) criterion and similar methods are used such as mean-square-error (MSE), normalized mean-square-error (NMSE), and so on. Sum of squared error criterion can be defined as:

th number set according to the maximization

ined criterion function J assigns the datum into k

and minimization calculation in k sets.

20 Recent Applications in Data Clustering

union set.

2. Clustering criteria

The K-Means clusters were first developed by Mac Queen [8]. In the K-Means clusters, clusters are formed using Euclidean distance. In the K-Means algorithm, unsupervised learning is used and k classes are created which minimize the error function [9].

In K-Means clustering, k cluster centers are created from the selected data set. It is then placed at the nearest cluster using Euclidean distance. New cluster centers are formed according to the results of the clustering. From the calculations of the clustering, the cluster center is recalculated. The arithmetic average is used as the calculation method, and the new cluster center is determined. All samples are reclassified according to the new center. This process is repeated until it is determined that the samples in the set have not passed to another set.

The partitioning of the k pieces of data x is represented by the minimization of the J parameter as in (3).

$$J = \min\left(\sum\_{k} \sum\_{x \in c\_k} w\_x \text{dist}(\mathbf{x}, o\_k)\right) \tag{3}$$

If the data are classified in a cluster near the center of the nearest cluster, the J value will be the minimum. If the data x are classified in the kth number cluster, the value can be optimized by changing the weighting value of wx to obtain the minimum J value. dist xð Þ ; ok is the notation which represents the distance function. In this formula, x represents the pixel data, ok is the center of the cluster. k sets are shown as in (4).


Figure 2. Flow chart of the K-Means algorithm.

$$\mathbf{c}\_{k} = (\mathbf{c}\_{k,1}, \mathbf{c}\_{k,2}, \dots, \mathbf{c}\_{k,n}) \tag{4}$$

sea for the given landscape, cluster-2 shows the the green areas in the landscape and cluster-3

Fuzzy theory is firstly developed by Zadeh [10] for defining adjustable degrees of memberships. Fuzzy theory creates intermediate sets rather than classical sets. In classical sets, each data item is assigned into only one cluster. In contrast, data in fuzzy clusters can be represented in multiple clusters. This multiset assignment can belong to all the clusters with a certain degree of membership defined by Bezdek [11]. This one item in multiset representation

The fuzzy C-Mean algorithm (FCM) is frequently used because of its ease of operation and reliability in many applications [12–15]. Conventional Fuzzy C-Mean (FCM) works with the

i¼1

numbered cluster. Total cluster size (c) given in the range of 1½ � ; <sup>c</sup> . In the formula (7), <sup>d</sup><sup>2</sup>

9 = ;

�1

xk

FCM algorithm flow chart is shown in Figure 4. This algorithm will be done in the given iteration step specified in this traditional clustering method. The cluster centers are updated in

When the FCM algorithm is applied for the given original image shown in Figure 5a for three clusters, this image can be represented in three clusters as in Figure 5b–d, respectively. This algorithm can only be applied for gray-scaled images. There is no crisp clustering for the given

Xn k¼1 um ikd<sup>2</sup>

ik is the membership function in the range [0,1]. This membership represents the

th-cluster center (vi).

ik (6)

<sup>i</sup>¼<sup>1</sup> uik <sup>¼</sup> 1 for each pixel. Using

th

Partitional Clustering

23

http://dx.doi.org/10.5772/intechopen.75836

ik

th pixel. <sup>k</sup> is defined in the range of <sup>k</sup> <sup>∈</sup>½ � <sup>1</sup>; <sup>n</sup> for the <sup>i</sup>

ik and cluster center vi, FCM targets to reach local minimums

ik ¼ k k xk � vi (7)

∀i ∈½ � 1; c and k ∈½ � 1; n (8)

∀i∈ ½ � 1; c (9)

principle of minimizing the objective function [16] shown in the following formula (6):

JFCM <sup>¼</sup> <sup>X</sup><sup>c</sup>

d2

The fuzzy set theory aims that the membership function is P<sup>c</sup>

shows the roads for the given landscape.

can be useful for sharply separated cluster boundaries.

4. Fuzzy clustering

where um

membership degree of xk for the k

the FCM membership function um

represents the distance between x<sup>k</sup> and i

by using the equivalence (8) and (9), respectively.

ik <sup>¼</sup> <sup>X</sup><sup>c</sup>

each iteration step to calculate the membership function.

8 < :

j¼1

dik djk � �<sup>2</sup>=ð Þ <sup>m</sup>�<sup>1</sup>

v∗ <sup>i</sup> ¼ P<sup>n</sup> <sup>k</sup>¼<sup>1</sup> um ik

P

n <sup>k</sup>¼<sup>1</sup> um ik

u∗

The distance function can be calculated as in Euclidean distance for k clusters in the next formula (5).

$$\text{dist}\left(\mathbf{x}\_{\text{j}}^{i},o\_{\text{k}}\right) = \sqrt{\left(\mathbf{x}\_{\text{j},1}^{i} - o\_{\text{k},1}\right)^{2} + \left(\mathbf{x}\_{\text{j},1}^{i} - o\_{\text{k},2}\right)^{2} + \dots + \left(\mathbf{x}\_{\text{j},n}^{i} - o\_{\text{k},n}\right)^{2}}\tag{5}$$

The iterative operations are repeated as in the flow diagram in Figure 2. This flowchart represents the iterative distance controlled operations to minimize the J parameter.

When this algorithm is applied for three-cluster for the original image shown in Figure 3a, this image can be represented in three clusters as in Figure 3b–d, respectively. Cluster-1 shows the

Figure 3. K-means algorithm results. (a) Original image; (b) K-Means Cluster No:1; (c) K-Means Cluster No:2; and (d) K-Means Cluster No:3.

sea for the given landscape, cluster-2 shows the the green areas in the landscape and cluster-3 shows the roads for the given landscape.
