**3.1 Problem of influence by change of view distance of video camera**

If a mobile agent tracks a target entity, the mobile agent has to know the deployed location of the video cameras in the system. However the abilities of the neighbor cameras are also determined by their view distances. A problem caused by a difference in the view distances can occur. This problem occurs when there is a difference in expected overlap of a view or an interrupt of view.

A scenario in which a neighbor video camera's location is influenced by view distance is shown in Fig.3. The upper side figures of Fig.3 show four diagrams portraying a floor plan with four video cameras each, considering the view distances of each video camera are different and assuming that the target entity to be tracked moves from the location of video camera A to video camera D. The underside figures of Fig.3 show neighbors of each video camera with arrows. The neighbor of video camera A in object (a-1) of Fig.3 is video camera B but not C and not D as the arrows in object (a-2) show. In object (a-1) of Fig.3, video camera C and D are also not considered neighbors of video camera A, because video camera B blocks the view of video camera C and D. And the target entity can be captured at an earlier time on video camera B. But in the case of object (b-1) of Fig.3, the neighbors of video camera A are video camera B and C but not camera D as the arrows in object (b-2) of Fig. 3 show. In the case of object (c-1) of Fig.3, the neighbors of video camera A are all video cameras as the arrows in object (c-2) of Fig.3 show. Thus neighbor video camera's location indicates the difference in view distances of video cameras. The case of object (d-1) in Fig.3 is more complicated. The neighbors of video camera A in object (d-1) of Fig.3 are video camera B, C, and D as the arrows in object (d-2) of Fig.3 show. And video camera B is not considered the neighbor of video camera C. It is because video camera A exists as a neighbor between video camera B and C. When it is assumed that a target entity moves from A to D, the target entity is sure to be captured by video camera A, B, A, and C in that order.

Fig. 3. Example of influence by change of view distance.

24 Recent Developments in Video Surveillance

The system architecture is shown in Fig. 2. The GUI is operated only on the agent monitoring terminal. The GUI is able to register images of the entities and monitor the status of all the mobile agents. The mobile agent server is executed on the feature extraction server and allows the mobile agents to execute. The Feature extraction function is able to extract features of the captured entities, which is then utilized in the tracking of those entities as mobile agents. OSGi (Open Service Gateway Initiative Alliance) S/W acts as a mediator for the different software, allowing the components to utilize each other. The Agent information manager manages all mobile agent information and provides the information to the agent monitoring terminal. The Video recording S/W records all video, and provides the video movie to agent monitoring terminal. Each PC is equipped with an Intel Pentium IV 2.0 GHz processor and 1 GB memory. The system has an imposed condition requirement that maximum execution time of feature judgment is 1 second and maximum execution time of

Here is indicated a problem that a change of view distance of video camera makes change

If a mobile agent tracks a target entity, the mobile agent has to know the deployed location of the video cameras in the system. However the abilities of the neighbor cameras are also determined by their view distances. A problem caused by a difference in the view distances can occur. This problem occurs when there is a difference in expected overlap of a view or

A scenario in which a neighbor video camera's location is influenced by view distance is shown in Fig.3. The upper side figures of Fig.3 show four diagrams portraying a floor plan with four video cameras each, considering the view distances of each video camera are different and assuming that the target entity to be tracked moves from the location of video camera A to video camera D. The underside figures of Fig.3 show neighbors of each video camera with arrows. The neighbor of video camera A in object (a-1) of Fig.3 is video camera B but not C and not D as the arrows in object (a-2) show. In object (a-1) of Fig.3, video camera C and D are also not considered neighbors of video camera A, because video camera B blocks the view of video camera C and D. And the target entity can be captured at an earlier time on video camera B. But in the case of object (b-1) of Fig.3, the neighbors of video camera A are video camera B and C but not camera D as the arrows in object (b-2) of Fig. 3 show. In the case of object (c-1) of Fig.3, the neighbors of video camera A are all video cameras as the arrows in object (c-2) of Fig.3 show. Thus neighbor video camera's location indicates the difference in view distances of video cameras. The case of object (d-1) in Fig.3 is more complicated. The neighbors of video camera A in object (d-1) of Fig.3 are video camera B, C, and D as the arrows in object (d-2) of Fig.3 show. And video camera B is not considered the neighbor of video camera C. It is because video camera A exists as a neighbor between video camera B and C. When it is assumed that a target entity moves from A to D, the target entity is sure to be captured by video camera A, B, A, and C in that

mobile agent transfer is 200 milliseconds.

an interrupt of view.

order.

**3. Influence by change of view distance of video camera** 

for neighbor cameras. And a solution for the problem is also indicated.

**3.1 Problem of influence by change of view distance of video camera** 

This scenario indicates that the definition of "neighbor" cannot be determined clearly because the determination of the neighbor definition is influenced by the change of view distance and it becomes more complicated as the number of video cameras increases.

## **3.2 Neighbor node determination algorithm to resolve the problem**

Neighbor node determination algorithm can easily determine the neighbor video camera's location without regard to the influence of view distances and any modification of the information of the currently installed cameras. The modification information is set in the system to compute neighbor video cameras on the diagram, which is expressed as a graph. Nodes are used to compute neighbor video camera's information in this algorithm. The nodes are defined as camera node and non-camera node. Camera node is the location of video camera that is labeled as camera node. The nodes are defined as *A* = *{a1, a2, ..., ap}*. This node is also a server with video camera. Non-camera node is defined as *V* = *{v1, v2, ..., vq}*.

The conditions of a non-camera node are stated below; i) either of crossover, corner, terminal of passage, ii) the position where a video camera is installed, or iii) the end point of the view distance of a video camera. In addition, the point where the above conditions are overlapped is treated as one node. When the view distance of the video camera reaches a non-camera node, the non-camera node is defined as the neighbor of the camera node. When two non-camera nodes are next to each other on a course, those nodes are specified as neighbors. Fig.4 shows an example of these definitions applied and shows the view distances of the video cameras.

The algorithm accomplishes neighbor node determination using an adjacency matrix. Two kinds of adjacency matrix are used by the algorithm. One is an adjacency matrix *X* made from camera nodes' locations as rows and non-camera nodes' locations as columns. Element

A Construction Method for Automatic Human Tracking System with Mobile Agent Technology 27

by excluding unnecessary non-camera nodes from adjacency matrix *X* and *Y*. v) Calculate neighbor's location matrix by multiplying adjacency matrix and transposed matrix *X'T*. This neighbor's location matrix is the neighbor's node information. An unnecessary non-camera node is a non-camera node which has no camera node as a neighbor. Adjacency matrix *X'* and *Y'* are computed without unnecessary nodes, and using the procedure shown later. There are reasons why it might be better to include the unnecessary nodes in the diagram from the beginning as we have done. Since the risk of committing an error will be higher as the diagram becomes larger, we include the unnecessary nodes from the beginning and remove them at the end. Finally, matrix *E* which indicates the neighbor nodes is derived as

1 is neighbour node to ' ' '

Human tracking method consists of Follower method and Detection method. Follower method is used for tracking a moving target. Detection method is used for detecting a target when an agent has lost the target. In the tracking method, an agent has three statuses as "Catching", "Not catching" and "Lost". At first, an agent is assumed that it stays on a certain camera node. If the feature parameter the agent keeps is similar to the feature parameter extracted on the node, agent's status is indicated as "Catching". If the parameter the agent keeps is not similar to the feature parameter extracted on the node, agent's status is indicated as "Not catching". If the agent keeps "Not catching" status on a certain time, the

In Follower method, an agent deploys its copies to neighbor nodes when agent's status becomes "Catching". When one of the copies has "Catching" status, all agents except that copy are removed from the system. And that copy becomes original agent. After that, the agent deploys its copies to neighbor nodes again. The follower method realizes tracking by

The detection method in this chapter is used to re-detect a target when the automatic tracking system loses the target. This method improves the tracking function, because an individual can not be accurately identified in the current image processing. As such the reliability of the system is further improved, because it enhances the continuous tracking function and re-detection of the target even if a target is lost for a long period of time. In this chapter, if a target is not captured within a certain period of time, the mobile agent then concludes that the target is lost. On such case the system can also conclude that the target is

We are proposing two types of detection method: (a) "Ripple detection method" and (b)

"Stationary net detection method". These methods are shown in Fig. 5.

*E XYX*

**4. Human tracking method** 

**4.1 Follower method** 

repeating those routine.

**4.2 Detection method** 

lost.

agent decides that it lost a target, and agent's status is indicated as "Lost".

0 is not neighbour node to *<sup>T</sup> i j*

*i j a a*

*a a*

. .

(4)

(4).

Fig. 4. Figure that sets non-camera nodes.

*xij* of matrix *X* is defined as (1). Another one is as adjacency matrix *Y* made from non-camera nodes' location as rows and columns. Element *yij* of matrix *Y* is defined as (2). The neighbor information for video cameras is calculated from the connection information of non-camera nodes by using adjacency matrix *X* and *Y*.

$$\mathbf{x}\psi = \begin{cases} 1 & \text{There is the line which links camera node } a \text{ and non-camera node } v \text{.} \\ 0 & \text{There is no link.} \end{cases} \tag{1}$$

$$y\_{\langle\rangle} = \begin{cases} 1 & \text{There is the line which links two non-canner nodes, } vi \text{ and } vj. \\ 0 & \text{There is no link or (3) is satisfied.} \end{cases} \tag{2}$$

$$\mathbf{y}\_{\mathbf{j}\mathbf{j}} = \mathbf{y}\_{\mathbf{j}\mathbf{i}} = \mathbf{1}, \ \sum\_{n=1}^{m} \mathbf{x}\_{n\mathbf{i}} > \mathbf{1}, \ \sum\_{n=1}^{m} \mathbf{x}\_{n\mathbf{j}} > \mathbf{1} \tag{3}$$

Below is the algorithm to determine neighbor nodes: i) Set camera nodes and non-camera nodes on the diagram as shown in object (b) of Fig.4. ii) Transform the diagram to a graph as shown in object (c) of Fig.4. iii) Generate an adjacency matrix *X* from camera node locations and non-camera node locations on the graph, and generate an adjacency matrix *Y* from noncamera node locations on the graph. Adjacency matrix *X* indicates that rows are camera nodes and columns are non-camera nodes. Adjacency matrix *Y* indicates that rows and columns are non-camera nodes, which results in adjacency matrix *Y* resolving an overlap problem of view distances between video cameras. iv) Calculate adjacency matrix *X'* and *Y'*  by excluding unnecessary non-camera nodes from adjacency matrix *X* and *Y*. v) Calculate neighbor's location matrix by multiplying adjacency matrix and transposed matrix *X'T*. This neighbor's location matrix is the neighbor's node information. An unnecessary non-camera node is a non-camera node which has no camera node as a neighbor. Adjacency matrix *X'* and *Y'* are computed without unnecessary nodes, and using the procedure shown later. There are reasons why it might be better to include the unnecessary nodes in the diagram from the beginning as we have done. Since the risk of committing an error will be higher as the diagram becomes larger, we include the unnecessary nodes from the beginning and remove them at the end. Finally, matrix *E* which indicates the neighbor nodes is derived as (4).

$$E = X^\prime Y^\prime X^\prime \begin{cases} \ge 1 & a\text{ is neighbor node to }a\text{\textquotedblleft} \\ = 0 & a\text{\textquotedblright} \text{ is not neighbor node to }a\text{\textquotedblleft} \end{cases} \tag{4}$$
