**6.4 Experiment 3: Quantitative evaluation of anomaly detection algorithms**

Here we provide a quantitative evaluation and comparison of *m*-Medoids based anomaly detection algorithms, as proposed in MMC-GFS and LMC-ES frameworks, with competitors. We implemented three different anomaly detection techniques based on statistical test as proposed in (Khalid & Naftel, 2006), Grown When Required (GWR) novelty filter as proposed in (Marsland et al., 2002) and one-class classifier based anomaly detection as proposed in (Tax, 2001). (Khalid & Naftel, 2006) performs anomaly detection by using Mahalanobis classifier and conducting Hotelling's *T*<sup>2</sup> test. (Tax, 2001) perform anomaly detection by generating model of one class (referred to as target class) and distinguishing it from samples belonging to all other classes. There generation of model of the target class is done using SVM and GMM. For SVM-based one class classifier (OCC-SVM), we have used RBF kernel for the modeing of target class. For GMM-based one class classifier (OCC-GMM), we have used the approach as specified in Experiment 2 to generate the GMM-based model.

The experiment has been conducted using different number of word classes from ASL dataset. We have extracted half of the samples belonging to each word class for training purposes leaving the other half of the samples to be used as test data. DFT-MOD based coefficient feature vector representation of sign trajectories from training data is generated and used to generate models as required by the different classification approaches. MMC-GFS and LMC-ES framework based model of each class is generated using the algorithm as presented in section 3 and (Khalid, 2010a) respectively. Patterns are modeled using 20 medoids per pattern.

Once the model learning phase is over, anomaly detection using different techniques is carried out using test dataset. We would expect that few instances drawn from class *X* would be recorded as anomalous when tested against the same class, whereas nearly all instances would be detected as anomalous when tested against a different class *Y*. The experiment is repeated with different numbers and combinations of word classes. Each anomaly detection experiment is averaged over 50 runs to reduce any bias resulting from favorable word selection.

Fig. 11 reports the result in terms of percentage of correct anomaly detection using various number of word classes from ASL dataset. The results demonstrate the superiority of anomaly detection using *m*-Medoids based MMC-GFS and LMC-ES frameworks. The anomaly detection accuracies obtained using MMC-GFS algorith are higher than unimodal LMC-ES based anomaly detection algorithm. MMC-GFS and LMC-ES performs better than OCC-SVM, OCC-GMM, GWR and Mahalanobis framework-based Naftel's method. The superior performance of proposed approach as compared to state-of-the-art techniques is due to the fact that our approach gives importance to correct classification of normal sample and to the filtration of abnormal samples during the model generation phase. On the other hand, OCC-SVM generates good model of normal classes but classifies many of the abnormal samples as member of normal classes whereas GWR gives extra importance to filtering abnormal samples and in the process, identifies many normal samples as abnormal.

Fig. 11. Percentage anomaly detection accuracies for different number of classes from ASL dataset
