**1. Introduction**

38 Recent Developments in Video Surveillance

Lange, D. B. & Oshima, M. (1999). Seven good reasons for mobile agents, *Communications of* 

Gray, R. S.; Cybenko, G.; Kotz, D.; Peterson, R. A. & Rus, D. D. (2002). D 'agents:

Motomura, S.; Kawamura, T. & Sugahara, K. (2005). Maglog: A mobile agent framework for

Kawamura, T.; Motomura, S. & Sugahara, K. (2005). Implementation of a logic-based multi

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints, *International* 

Ke, Y. & Sukthankar, R. (2004). Pca-sift: A more distinctive representation for local image descriptors, *Computer Vision and Pattern Recognition (CVPR)*, vol. 2, pp. 506–513 Bay, H.; Ess, A.; Tuytelaars, T. & Gool, L. V. (2008). Speeded-up robust features (surf), *Computer Vision and Image Understanding (CVIU)*, vol. 110, no. 3, pp. 346–359 Open Service Gateway Initiative Alliance, *OSGi Alliance Specifications OSGi Service Platform* 

*on Integration of Knowledge Intensive Multi-Agent Systems*, pp. 486–491 Cabri, G.; Leonardi, L. & Zambonelli, F. (2000). Mobile-agent coordination models for internet applications," *Computer*, vol. 33, no. 2, pp. 82–89, February 2000 Valetto, G.; Kaiser, G. & Kc, G. S. (2001). A mobile agent approach to process-based dynamic

Applications and performance of a mobile-agent system, *Software: Practice and* 

distributed models, *Proceedings of the IASTED International Conference on Parallel and* 

agent framework on java environment, *Proceedings of IEEE International Conference* 

adaptation of complex software systems, *Lecture Notes in Computer Science*, vol.

*the ACM*, vol. 42, no. 3, pp. 88–89

2077, pp. 102–116

*Experience*, vol. 32, no. 6, pp. 543–573

*Distributed Computing and Systems*, pp. 414–420

*Journal of Computer Vision*, vol. 60, no. 2, pp. 91–110

*Release 1*, last access May 2011. [Online]. Available:

http://www.osgi.org/Specifications/HomePage

Video surveillance is a rapidly growing industry. Driven by low-hardware costs, heightened security fears and increased capabilities, video surveillance equipment is being deployed more widely and with greater storage than ever. This provides a huge amount of video data. Associating to these video data, retrieval facilities become very useful for many purposes and many kinds of staff. Recently, several approaches have been dedicated to retrieval facilities for surveillance data (Le, Thonnat et al. 2009) (Zhang, Chen et al. 2009). Figure 1 shows how indexing and retrieval facility can be integrated in a surveillance system. Videos coming from cameras will be interpreted by the video analysis module. There are two modes for using the analysed results: (1) the corresponding alarms are sent to members of the security staff to inform them about the situation; (2) the analysed results are stored in order to be used in the future. In this chapter, we focus on analysing current achievements in surveillance video indexing and retrieval. Video analysis (Senior 2009) is beyond the scope of this chapter.

Video analysis module provides two main result types of result: objects and events. Thus, surveillance video indexing and retrieval approaches can divided into two categories: surveillance video indexing and retrieval at the object level (Calderara, Cucchiara et al. 2006; Ma and Cohen 2007; Le, Thonnat et al. 2009) and at the event level (Zhang, Chen et al. 2009; Velipasalar, Brown et al. 2010). As events of interest may vary significantly among different applications and users, this chapter focuses on presenting the work done for surveillance video indexing and retrieval at the object level.

The remaining of the chapter is organized as follows: In Section 2, we give a brief overview of surveillance object retrieval. Section 3 aims at analysing in detail appearance-based surveillance object retrieval. We first give some definitions and point out the existing challenges. Then, we describe the solutions proposed for two important tasks: object signature building and object matching in order to overcome these challenges. Section 4 presents current achievements and discusses about open problems in this domain.

Appearance-Based Retrieval for Tracked Objects in Surveillance Videos 41

Fig. 2. Late fusion object retrieval approach: the object detection and tracking is performed on video stream of each camera. Then, the object matching compares the query and the detected objects of each camera. The matching result is then fused to form the retrieval

Since objects in video surveillance are physical objects (e.g. people, vehicles) that are present in the scene at a certain time, in general, they are detected and tracked in a large number of frames. Objects in videos possess two main characteristics named spatial and temporal characteristics. Spatial characteristics of an object may be its positions in frames (in 2D coordinates) and positions in scene (in 3D coordinates), its spatial relationships with other objects and its appearance. Temporal characteristics of an object contain its movement and its temporal relationships with other objects. Therefore, an object may be represented by one sole or several characteristics. However, among these characteristics, object movement and object appearance are the two most important characteristics and are widely used in the

Concerning the object representation based on object movement, in the literature, a number of different approaches have been proposed for object movement representation and matching (Broilo, Piotto et al. 2010). Certain approaches directly use detected object positions across frames that are represented in trajectory form (Zheng, Feng et al. 2005). As object trajectory may be very complex, other authors try to segment an object trajectory into several sub-trajectories (Buchin, Driemel et al. 2010) with the purpose that each sub-

results.

literature.

Fig. 3. Early fusion object retrieval approach.

**2.2 Object feature extraction and representation** 

Fig. 1. Indexing and retrieval facility in a surveillance system. Videos coming from cameras will be interpreted by the video analysis module. There are two modes for using the analysed results: (1) the corresponding alarms are sent to security staffs to inform them about the situation; (2) the analysed results are stored in order to be used in the future.
