**4. Human tracking method**

26 Recent Developments in Video Surveillance

*xij* of matrix *X* is defined as (1). Another one is as adjacency matrix *Y* made from non-camera nodes' location as rows and columns. Element *yij* of matrix *Y* is defined as (2). The neighbor information for video cameras is calculated from the connection information of non-camera

1 There is the line which links camera node and non-camera node .

1 There is the line which links two non-camera nodes, and .

Below is the algorithm to determine neighbor nodes: i) Set camera nodes and non-camera nodes on the diagram as shown in object (b) of Fig.4. ii) Transform the diagram to a graph as shown in object (c) of Fig.4. iii) Generate an adjacency matrix *X* from camera node locations and non-camera node locations on the graph, and generate an adjacency matrix *Y* from noncamera node locations on the graph. Adjacency matrix *X* indicates that rows are camera nodes and columns are non-camera nodes. Adjacency matrix *Y* indicates that rows and columns are non-camera nodes, which results in adjacency matrix *Y* resolving an overlap problem of view distances between video cameras. iv) Calculate adjacency matrix *X'* and *Y'* 

1 1 1, 1, 1 *m m ij ji ni nj n n yy x x* (3)

0 There is no link or (3) is satisfied.

*i j*

*i j*

*v v*

(1)

(2)

*a v*

0 There is no link.

Fig. 4. Figure that sets non-camera nodes.

nodes by using adjacency matrix *X* and *Y*.

*ij*

*ij*

*y* 

*x*  Human tracking method consists of Follower method and Detection method. Follower method is used for tracking a moving target. Detection method is used for detecting a target when an agent has lost the target. In the tracking method, an agent has three statuses as "Catching", "Not catching" and "Lost". At first, an agent is assumed that it stays on a certain camera node. If the feature parameter the agent keeps is similar to the feature parameter extracted on the node, agent's status is indicated as "Catching". If the parameter the agent keeps is not similar to the feature parameter extracted on the node, agent's status is indicated as "Not catching". If the agent keeps "Not catching" status on a certain time, the agent decides that it lost a target, and agent's status is indicated as "Lost".

#### **4.1 Follower method**

In Follower method, an agent deploys its copies to neighbor nodes when agent's status becomes "Catching". When one of the copies has "Catching" status, all agents except that copy are removed from the system. And that copy becomes original agent. After that, the agent deploys its copies to neighbor nodes again. The follower method realizes tracking by repeating those routine.

#### **4.2 Detection method**

The detection method in this chapter is used to re-detect a target when the automatic tracking system loses the target. This method improves the tracking function, because an individual can not be accurately identified in the current image processing. As such the reliability of the system is further improved, because it enhances the continuous tracking function and re-detection of the target even if a target is lost for a long period of time. In this chapter, if a target is not captured within a certain period of time, the mobile agent then concludes that the target is lost. On such case the system can also conclude that the target is lost.

We are proposing two types of detection method: (a) "Ripple detection method" and (b) "Stationary net detection method". These methods are shown in Fig. 5.

A Construction Method for Automatic Human Tracking System with Mobile Agent Technology 29

Stationary net detection method widens a search like setting a stationary net with the Neighbor node determination algorithm from where an agent lost a target to give top

<sup>1</sup> 1 is neighbour node to '( ') ' 0 is not neighbour node to

In this equation, adjacency matrix *E* indicates the node that can reach via *n* non-camera nodes and *n* is always set to *n ≥* 2. In this method, the coefficient *n* is set to *n* = 4 because camera nodes are set with a certain interval. The interval between cameras in the real system may be close, but in that case, number of non-camera nodes between the cameras decreases. Therefore it is enough interval to re-detect a target if *n* consists of *n ≥* 4. This method has a feature that agents are deployed to neighbor camera nodes via *n* next non-camera nodes and catch a target like a stationary net. In addition, this method also deletes other agents immediately after discovering the target, and suppresses the waste of the resource. The Stationary net detection method is developed and is experimented in search property. In the

When a mobile agent lost a target, copy agents are deployed to the next nodes of (11) expressed by (12), and search is started. *X'Y'*2*X'T* shows neighbor camera nodes via two noncamera nodes, because the elements of *X'Y'*2*X'T* larger than 1 can be reached if the elements are larger than 1. If copy agents are deployed at each camera nodes via non-camera nodes more than two, detection range of target widens. And, excepting neighbor node information *E* of camera nodes, automatic human tracking system uses a minimum resource by

Similarly, it becomes like (13) and (14) to calculate the next camera node of more wide

As mentioned above, the equation (15) is derived when deploying agents efficiently to the next camera nodes via *n* non-camera nodes. *n* is larger than 2 and is incremented one by one

*En X Y X Em X Y X X Y X*

Here are two types of experiment. One is an experiment by simulator, and the other one is an experiment by real environment. In the experiment by simulator, follower method and

'' ' '' ' '' ' *<sup>n</sup> n T nT n T*

1

1

*m*

*<sup>n</sup> a a*

*<sup>T</sup> i j*

*i j*

*a a*

. .

*E E XYX* 1 ''' (11)

<sup>2</sup> 2 '' ' . *<sup>T</sup> E XY X E* (12)

<sup>3</sup> 3 ' ' ' ( 2 1) *<sup>T</sup> E XY X E E* (13)

<sup>4</sup> 4 ' ' ' ( 3 2 1) *<sup>T</sup> E XY X E E E* (14)

1

(15)

(10)

priority to re-detect. This method uses equation (10) in the algorithm.

Stationary net detection method, the neighbor camera nodes are shown as (11).

*E XY X*

deploying copy agents.

**5. Experimentation** 

when this equation is used for detection.

range.

Fig. 5. Figure that sets non-camera nodes.

Ripple detection method widens a search like a ripple from where an agent lost a target to give top priority to re-detect. This method has a feature that the discovery time becomes shorter and usual tracking can resume more quickly, if the target exists near where the agent lost. In addition, this method deletes other agents immediately after discovering the target, and suppresses the waste of the resource. The Ripple detection method is developed and is experimented in search propriety. In the Ripple detection method, the neighbor camera nodes are shown as (5).

$$E1 = E = X'Y'X^{i^T} \tag{5}$$

When a mobile agent lost a target, copy agents are deployed to the next nodes of (5) expressed by (6), and search is started. *E*2 shows next neighbor camera nodes, because the elements of *E*2 larger than 1 can be reached if the elements are larger than 1. Therefore, except neighbor node information E of camera nodes, automatic human tracking system uses a minimum resource by deploying copy agents.

$$E2 = E^2 - E1 = E^2 - E \tag{6}$$

Similarly, it becomes like (7) and (8) to calculate the next camera node further.

$$E\mathcal{B} = E^3 - (E\mathcal{Q} + E\mathcal{I}) = E^3 - E^2\tag{7}$$

$$E4 = E^4 - (E3 + E2 + E1) = E^4 - E^3 \tag{8}$$

As mentioned above, the equation (9) is derived when deploying agents efficiently to the *n*  next camera nodes. *n* is larger than 2 and is incremented one by one when this equation is used for detection.

$$En = E^n - \sum\_{m=1}^{n-1} Em = E^n - E^{n-1} \tag{9}$$

Stationary net detection method widens a search like setting a stationary net with the Neighbor node determination algorithm from where an agent lost a target to give top priority to re-detect. This method uses equation (10) in the algorithm.

$$E = X'(Y')^{n-1} \cdot X^T \begin{cases} \ge 1 & a \text{ is neighbor node to } a \text{.} \\ = 0 & a \text{ is not neighbor node to } a \text{.} \end{cases} \tag{10}$$

In this equation, adjacency matrix *E* indicates the node that can reach via *n* non-camera nodes and *n* is always set to *n ≥* 2. In this method, the coefficient *n* is set to *n* = 4 because camera nodes are set with a certain interval. The interval between cameras in the real system may be close, but in that case, number of non-camera nodes between the cameras decreases. Therefore it is enough interval to re-detect a target if *n* consists of *n ≥* 4. This method has a feature that agents are deployed to neighbor camera nodes via *n* next non-camera nodes and catch a target like a stationary net. In addition, this method also deletes other agents immediately after discovering the target, and suppresses the waste of the resource. The Stationary net detection method is developed and is experimented in search property. In the Stationary net detection method, the neighbor camera nodes are shown as (11).

$$E1 = E = X'Y'X'\tag{11}$$

When a mobile agent lost a target, copy agents are deployed to the next nodes of (11) expressed by (12), and search is started. *X'Y'*2*X'T* shows neighbor camera nodes via two noncamera nodes, because the elements of *X'Y'*2*X'T* larger than 1 can be reached if the elements are larger than 1. If copy agents are deployed at each camera nodes via non-camera nodes more than two, detection range of target widens. And, excepting neighbor node information *E* of camera nodes, automatic human tracking system uses a minimum resource by deploying copy agents.

$$E\mathfrak{D} = X'Y'^2X'^T - E.\tag{12}$$

Similarly, it becomes like (13) and (14) to calculate the next camera node of more wide range.

$$E\mathfrak{B} = X'Y'^3X'^T - \left(E\mathfrak{B} + E1\right) \tag{13}$$

$$E4 = X'Y'^4X^{i^T} - \left(E3 + E2 + E1\right) \tag{14}$$

As mentioned above, the equation (15) is derived when deploying agents efficiently to the next camera nodes via *n* non-camera nodes. *n* is larger than 2 and is incremented one by one when this equation is used for detection.

$$En = X'Y''^{\prime\prime}X^{\prime T} - \sum\_{m=1}^{n-1} Em = X'Y''^{\prime\prime}X^{\prime T} - X'Y''^{\prime\prime -1}X^{\prime T} \tag{15}$$

#### **5. Experimentation**

28 Recent Developments in Video Surveillance

Ripple detection method widens a search like a ripple from where an agent lost a target to give top priority to re-detect. This method has a feature that the discovery time becomes shorter and usual tracking can resume more quickly, if the target exists near where the agent lost. In addition, this method deletes other agents immediately after discovering the target, and suppresses the waste of the resource. The Ripple detection method is developed and is experimented in search propriety. In the Ripple detection method, the neighbor camera

When a mobile agent lost a target, copy agents are deployed to the next nodes of (5) expressed by (6), and search is started. *E*2 shows next neighbor camera nodes, because the elements of *E*2 larger than 1 can be reached if the elements are larger than 1. Therefore, except neighbor node information E of camera nodes, automatic human tracking system

As mentioned above, the equation (9) is derived when deploying agents efficiently to the *n*  next camera nodes. *n* is larger than 2 and is incremented one by one when this equation is

1

*<sup>n</sup> <sup>n</sup> n n*

1

*m En E Em E E*

Similarly, it becomes like (7) and (8) to calculate the next camera node further.

1 '''*<sup>T</sup> E E XYX* (5)

2 2 *E EEEE* 2 1 (6)

3 3 <sup>2</sup> *E E EE EE* 3 ( 2 1) (7)

4 4 <sup>3</sup> *E E EEE EE* 4 ( 3 2 1) (8)

1

(9)

Fig. 5. Figure that sets non-camera nodes.

uses a minimum resource by deploying copy agents.

nodes are shown as (5).

used for detection.

Here are two types of experiment. One is an experiment by simulator, and the other one is an experiment by real environment. In the experiment by simulator, follower method and

A Construction Method for Automatic Human Tracking System with Mobile Agent Technology 31

on the floor map 2, if a floor map is complicated, the discovery time of the Stationary net detection method becomes shorter than the discovery time of the Ripple detection method and the number of agents of the Stationary net detection method becomes less than the

On the whole, the result of both methods shows that a number of agents decreases by searching a target near search cycle but the agents can not search the target if the search cycle time is longer than the waking speed. In addition based on the results, when the walking speed is faster, the discovery time is shortened or equal and the number of agents

Search Cycle(9*s*) Number of Agents 6 12 6 6

Search Cycle(6*s*) Number of Agents 7 6 6 7

Net (n=2)

Number of Agents 6 12 12 6 Discovery Time (*s*) 20.6 - - 20.5

Discovery Time (*s*) 20.5 - 20.5 20.5

Discovery Time (*s*) 20.5 20.5 20.5 20.5

Stationary Net (n=3) Stationary Net (n=4)

Walking Speed (1.5*m/s*) Ripple Stationary

Table 1. Detection time on floor map 1 by walking speed 1.5*m/s.* 

Fig. 6. Floor map 1 for experiment of detection methods.

number of agents of the Ripple detection method.

decreases or is equal.

Search Cycle(12*s*)

detection methods are experimented, and the effectiveness is verified. In the experiment by real environment, the tracking method is verified for whether the plural targets can be tracked continuously.
