**7. Discussion and conclusions**

18 Will-be-set-by-IN-TECH

Similar experiment with ASL dataset (using similar experimental settings) has been conducted by (Bashir et al., 2007) using their proposed GMM and HMM-based classification system. They reported classification accuracies of 0.96, 0.92, 0.86 and 0.78 for 2, 4, 8 and 16 word classes respectively. A comparison of these classification accuracies with the results obtained using our approach reveals that classifiers from *m*-Medoids classifier family performs better than GMM and HMM-based recognition system (Bashir et al., 2007) despite the fact that our proposed classification approach is conceptually simpler and computationally less expensive.

Here we provide a quantitative evaluation and comparison of *m*-Medoids based anomaly detection algorithms, as proposed in MMC-GFS and LMC-ES frameworks, with competitors. We implemented three different anomaly detection techniques based on statistical test as proposed in (Khalid & Naftel, 2006), Grown When Required (GWR) novelty filter as proposed in (Marsland et al., 2002) and one-class classifier based anomaly detection as proposed in (Tax, 2001). (Khalid & Naftel, 2006) performs anomaly detection by using Mahalanobis classifier and conducting Hotelling's *T*<sup>2</sup> test. (Tax, 2001) perform anomaly detection by generating model of one class (referred to as target class) and distinguishing it from samples belonging to all other classes. There generation of model of the target class is done using SVM and GMM. For SVM-based one class classifier (OCC-SVM), we have used RBF kernel for the modeing of target class. For GMM-based one class classifier (OCC-GMM), we have used the approach as

The experiment has been conducted using different number of word classes from ASL dataset. We have extracted half of the samples belonging to each word class for training purposes leaving the other half of the samples to be used as test data. DFT-MOD based coefficient feature vector representation of sign trajectories from training data is generated and used to generate models as required by the different classification approaches. MMC-GFS and LMC-ES framework based model of each class is generated using the algorithm as presented in section 3 and (Khalid, 2010a) respectively. Patterns are modeled using 20 medoids per

Once the model learning phase is over, anomaly detection using different techniques is carried out using test dataset. We would expect that few instances drawn from class *X* would be recorded as anomalous when tested against the same class, whereas nearly all instances would be detected as anomalous when tested against a different class *Y*. The experiment is repeated with different numbers and combinations of word classes. Each anomaly detection experiment is averaged over 50 runs to reduce any bias resulting from favorable word

Fig. 11 reports the result in terms of percentage of correct anomaly detection using various number of word classes from ASL dataset. The results demonstrate the superiority of anomaly detection using *m*-Medoids based MMC-GFS and LMC-ES frameworks. The anomaly detection accuracies obtained using MMC-GFS algorith are higher than unimodal LMC-ES based anomaly detection algorithm. MMC-GFS and LMC-ES performs better than OCC-SVM, OCC-GMM, GWR and Mahalanobis framework-based Naftel's method. The superior performance of proposed approach as compared to state-of-the-art techniques is due to the fact that our approach gives importance to correct classification of normal sample

**6.4 Experiment 3: Quantitative evaluation of anomaly detection algorithms**

specified in Experiment 2 to generate the GMM-based model.

pattern.

selection.

In this chapter, we have presented an extended *m*-Medoids based framework, referred to as MMC-GFS, for modeling of trajectory-based motion patterns. The strength of the proposed approach is its ability to model complex patterns without imposing any restriction on the distribution of samples within a given pattern. Once the multimodal *m*-Medoids model for all the classes have been learnt, the classification of new trajectories and anomaly detection is then performed using a proposed soft classification and anomaly detection algorithm which is adaptive to multimodal distributions of samples within a pattern. The strength of this technique is its ability to model complex patterns without imposing any restriction on the shape of patterns. MMC-GFS can be used for modeling, classification and anomaly detection in any feature space with a computable similarity function.

Experimental results are presented to show the effectiveness of proposed MMC-GFS classifier. Modeling of pattern and classification using proposed frameworks is unaffected by variation of sample distribution within a pattern as demonstrated in Fig. 4. Quantitative comparison of MMC-GFS based classifiers with competitive techniques demonstrates the superiority of our multimodal approach as it performs consistently better than commonly used Mahalanobis, GMM and HMM-based classifiers.

Chang, S. F., Chen, W., Horace, J. M., Sundaram, H. & Zhong, D. (1998). A fully

Behavior Recognition Using any Feature Space Representation of Motion Trajectories 121

Dagtas, S., Ali-Khatib, W., Ghafor, A. & Kashyap, R. (2000). Models for motion-based video

Faloutsos, C., Ranganathan, M. & Manolopoulos, Y. (1994). Fast sub-sequence matching in

Hamid, R., Johnson, A., Batta, S., Bobick, A., Isbell, C. & Coleman, G. (2005). Detection

Hsu, C.-T. & Teng, S.-J. (2002). Motion trajectory based video indexing and retrieval, *IEEE*

Hu, W., Tan, T., Wang, L. & Maybank, S. (2004). A survey on visual surveillance of object

Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T. & Maybank, S. (2006). A system for learning statistical

Hu, W., Xiao, X., Xie, D., Tan, T. & Maybank, S. (2004). Traffic accident prediction using

Hu, W., Xie, D., Fu, Z., Zeng, W. & Maybank, S. (2007). Semantic based surveillance video

Hu, W., Xie, D., Tan, T. & Maybank, S. (2004). Learning activity patterns using fuzzy

Jeanin, S. & Divakaran, A. (2001). Mpeg-7 visual motion descriptors, *IEEE Trans. Circuits Syst.*

Jin, Y. & Mokhtarian, F. (2004). Efficient video retrieval by motion trajectory, *Proceedings of*

Johnson, N. & Hogg, D. (1995). Learning the distribution of object trajectories for event recognition, *Proceedings of British Conference on Machine Vision*, pp. 582–592. Keogh, E., Chakrabarti, K., Pazzani, M. & Mehrota, S. (2001). Locally adaptive dimensionality

Khalid, S. (2010a). Activity classification and anomaly detection using m-mediods based modeling of motion patterns, *Pattern Recognition*, Vol. 43, pp. 3636–3647. Khalid, S. (2010b). Motion based behaviour learning, profiling and classification in the

Khalid, S. & Naftel, A. (2005). Evaluation of matching metrics for trajectory based indexing

Khalid, S. & Naftel, A. (2006). Classifying spatiotemporal object trajectories using

presence on anomalies, *Pattern Recognition*, Vol. 43, pp. 173–186.

*International Conference on Image Processing*, Vol. 1, pp. 605–608.

retrieval, *IEEE Transactions on ImageProcessing*, pp. 1168–1181.

*British Machine Vision Conference*, Kingston, pp. 667–676.

*on Management of Data*, pp. 419–429. Frey, B. & Dueck, D. (2007). Science, Vol. 315, pp. 972–976.

n-grams., *CVPR*, pp. 1031–1038.

pp. 334–352.

pp. 677–694.

pp. 151–162.

pp. 227–238.

*9*, pp. 1450–1464.

Vol. 34 of *3*, pp. 1618–1626.

*Video Technol.*, Vol. 11 of *6*, pp. 720–724.

automated content based video search engine supporting spatiotemporal queries, *IEEE Transactions on Circuits and System for Video Technology*, Vol. 8 of *5*, pp. 602–615.

indexing and retrieval, *IEEE Transactions on Image Processing*, Vol. 9 of *1*, pp. 88–101.

time-series databases, *Proceedings of the 1994 ACM SIGMOD International Conference*

and explanation of anomalous activities: Representing activities as bags of events

motion and behaviors, *IEEE Transactions on Systems, Man & Cybernetic*, Vol. 34 of *3*,

motion patterns, *IEEE Transactions on Pattern Analysis and Machine Learning*, Vol. 28 of

3-d model based vehicle tracking, *IEEE Transactions on Vehicular Tech*, Vol. 53 of *3*,

self-organizing neural networks, *IEEE Transactions on Systems, Man & Cybernetic*,

reduction for indexing large time series databases, *Proc. ACM SIGMOD Conference*,

and retrieval of video clips, *Proceedings of IEEE WACV*, Colorado, USA, pp. 242–249.

unsupervised learning in the coefficient feature space, *Multimedia Systems*, Vol. 12,

Experiments are also conducted to show the effectiveness of anomaly detection capabilities of proposed frameworks. Anomaly detection results for different classes of ASL datasets, using different variants of proposed anomaly detection algorithm, are presented. It has been shown that anomaly detection using multimodal MMC-GFS frameworks gives better anomaly detection accuracies as compared to the unimodal LMC-ES approach. Although LMC-ES enables the anomaly detection system to adapt to the normality distribution of individual classes, it is insensitive to the variation of distributions within a pattern which results in degradation of its performance as compared to MMC-GFS frameworks. Comparison of proposed anomaly detection algorithms with an existing approach demonstrates the superiority of our approach approach as they consistently perform better for different number of classes.

The application of proposed MMC-GFS based modeling and recognition system is not only limited to trajectory-based behavior recognition, but can also be applied to other recognition tasks that are critical in video surveillance application. Some of the applications where MMC-GFS based modeling and classification system can be applied include but is not limited to object recognition in surveillance videos, gait recognition, scene recognition etc.

#### **8. References**


Experiments are also conducted to show the effectiveness of anomaly detection capabilities of proposed frameworks. Anomaly detection results for different classes of ASL datasets, using different variants of proposed anomaly detection algorithm, are presented. It has been shown that anomaly detection using multimodal MMC-GFS frameworks gives better anomaly detection accuracies as compared to the unimodal LMC-ES approach. Although LMC-ES enables the anomaly detection system to adapt to the normality distribution of individual classes, it is insensitive to the variation of distributions within a pattern which results in degradation of its performance as compared to MMC-GFS frameworks. Comparison of proposed anomaly detection algorithms with an existing approach demonstrates the superiority of our approach approach as they consistently perform better for different number

The application of proposed MMC-GFS based modeling and recognition system is not only limited to trajectory-based behavior recognition, but can also be applied to other recognition tasks that are critical in video surveillance application. Some of the applications where MMC-GFS based modeling and classification system can be applied include but is not limited

Aghbari, Z., Kaneko, K. & Makinouchi, A. (2003). Content-trajectory approach for searching video databases, *IEEE Transanction on Multimedia*, Vol. 5 of *4*, pp. 516–531. Bashir, F. I., Khokhar, A. A. & Schonfeld, D. (2006). View-invariant motion trajectory

*on Machine Learning Approaches to Multimedia Information retrieval*, pp. 45–54. Bashir, F. I., Khokhar, A. A. & Schonfield, D. (2003). Segmented trajectory based indexing

Bashir, F. I., Khokhar, A. A. & Schonfield, D. (2004). A hybrid system for affine-invariant

Bashir, F. I., Khokhar, A. A. & Schonfield, D. (2005a). Automatic object trajectory based

Bashir, F. I., Khokhar, A. A. & Schonfield, D. (2005b). Hmm based motion recognition system

Bashir, F. I., Khokhar, A. A. & Schonfield, D. (2007). Object trajectory-based activity

Brotherton, T., Johnson, T. & Chadderdon, G. (1998). Classification and novelty detection

Cai, Y. & Ng, R. (2004). Indexing spatio-temporal trajectories with chebyshev polynomials,

Chan, K. & Fu, A. (1999). Efficient time series matching by wavelets, *Proc. of International*

based activity classification and recognition, *ACM Multimedia Systems, special issue*

and retrieval of video data, *IEEE International Conference on Image Processing*, Vol. 3,

motion recognition using gaussian mixture models, *IEEE International Conference on*

using segmented pca, *IEEE International Conference on Image Processing*, Genova, Italy,

classification and recognition using hidden markov models, *IEEE Transactions on*

using linear models and a class dependent elliptical basis function neural network,

to object recognition in surveillance videos, gait recognition, scene recognition etc.

of classes.

**8. References**

Barcelona, Spain, pp. II– 623–6.

pp. 1288–1291.

trajectory retrieval, *Proc. MIR'04*, pp. 235–242.

*Multimedia and Expo*, Netherland, pp. 1532– 1535.

*Image Processing*, Vol. 16 of *7*, pp. 1912–1919.

*Proc. IJCNN Conference*, Vol. 2, Anchorage, pp. 876–879.

*ACM SIGMOD/PODS Conference*, France, pp. 599–610.

*Conference on Data Engineering*, Sydney, pp. 126–133.


122 Recent Developments in Video Surveillance

Marsland, S., Shapiro, J. & Nehmzow, U. (2002). A self-organising network that grows when

Melo, J., Naftel, A., Bernardino, A. & Santos-Victor, J. (2004). Viewpoint independent detection

Owens, J. & Hunter, A. (2000). Application of the self-organising map for trajectory

Rea, N., Dahyot, R. & Kokaram, A. (2004). Semantic event detection in sports through motion

Roberts, S. & Tarassenko, L. (1994). A probabilistic resource allocating network for novelty

Shim, C. & Chang, J. (2001). Content based retrieval using trajectories of moving objects in

Shim, C. & Chang, J. (2004). Trajectory based video retrieval for multimedia information

Vlachos, M., Kollios, G. & Gunopulos, D. (2002). Discovering similar multidimensional

Xiang, T. & Gong, S. (2005). Video behaviour profiling and abnormality detection without

Xiang, T. & Gong, S. (2006). Incremental visual behaviour modelling, *The 6th IEEE Workshop*

Yacoob, Y. & Black, M. (1999). Parameterized modeling and recognition of activities, *Computer*

Yeung, D. & Chow, C. (2002). Parzen window network intrusion detectors, *Proc. International*

Zhang, D., Gatica-Perez, Bengio, S. & McCowan, I. (2005). Semi-supervised adapted hmms

Zhong, H., Shi, J. & Visontai, M. (2004). Detecting unusual activity in videos, *Proc. Of CVPR*,

for unusual event detection, *Proceedings of IEEE International Conference on Computer*

Tax, D. (2001). One-class classification, *PhD thesis, Delft University of Technology*.

of vehicle trajectories and lane geometry from uncalibrated traffic surveilllance cameras, *Proc. of International Confernce of Image Analysis and Recognition*, Porto,

classification, *Proceedings of Third IEEE International Workshop on Visual Surveillance*,

understanding, *Proceedings of Conference on Image and Video Retrieval*, Dublin, Ireland,

video databases, *Proceedings of IEEE 7th International Conference on Database Systems*

trajectories, *Proceedings of the International Conference on Data Engineering*, San Jose,

manual labelling, *Proc. IEEE International Conference on Computer Vision*, Vol. 2,

required, *Neural Networks*, Vol. 15, pp. 1041–1058.

detection, *Neural Computation*, Vol. 6, pp. 270–284.

*for Advanced Applications*, pp. 169–170.

systems, *Proceedings of ADVIS*, pp. 372–382.

Portugal, pp. 454–462.

Dublin, Ireland, p. 77.

pp. 88–97.

CA, pp. 673–684.

London, UK, pp. 1238– 1245.

*on Visual Surveillance*, Graz, pp. 65–72.

*Vision and Pattern Recognition*, pp. 611–618.

Washington D.C., pp. 819–826.

*Vision and Image Understanding*, Vol. 73, pp. 232–247.

*Conference on Pattern Recognition, IEEE*, Canada, pp. 385–388.
