3. Co-tracking

A single detector may have difficulties in distinguishing the target from the background in certain scenarios. In those cases, it is beneficial to consult another detector with higher robustness. These second detector may have complimentary characteristics to the first one or simply may be a more sophisticated detector that trades computational complexity with speed.

step by an integration of all the information. Finally, the classifiers are updated using only the samples that they successfully labeled in the previous frame to reflect the latest target changes.

Figure 3. Collaborative tracking. A detector and an auxiliary classifier trust each other to handle the sample difficult for

Active Collaboration of Classifiers for Visual Tracking http://dx.doi.org/10.5772/intechopen.74199 109

For this experiment, we selected a naive classifier with complementary properties to the main classifier in the previous section. This classifier is a KNN classifier using HOC and HOG features, trained on the samples trained from the first frame and updated with all the labeled samples by the collaboration of the classifiers. Not being pre-trained, the performance of this auxiliary classifier is poor in the beginning but gradually gets better. The quick classification of the KNN (owning to its kd-tree implementations and lightweight features) and lack of pretraining grant it high speed and generalization which is in contrast to the main detector. However, it should be noted that without being supervised by the main SVM-based detector, this classifier cannot perform well in isolation for tracking task. Figure 5 presents the performance of this auxiliary tracker as T2. As observed in the figure, the performance of the obtained co-tracker (T3) is better than the main detector (T1) and the auxiliary classifier (T2)

The co-tracking framework provides a means for classifiers to exchange information. This framework utilizes a utility measure (e.g., the classification confidence in [34]) to select the data for which one of the collaborators fails to classify with high confidence and then trains the other classifier on those data. This approach has two main shortcomings: (1) the redundant labeling of all samples for both classifiers and (2) training the collaborator with "all" of the uncertain samples. While the former increases the complexity of the system, the latter is not the optimal solution for tracking a target with non-stationary appearance distributions [35].

In this view, a principled ordering of samples for training [70] and selecting a subset of them based on criteria [37] can reduce the cost of labeling leading to faster performance increase as a

as a result of co-labeling, data exchange, and co-learning.

3.2. Evaluation

them to classify.

4. Active co-tracking

Collaborative discriminative trackers utilize classifiers that exchange their information, to achieve more robust tracking. These information exchanges are in the form of queries that one classifier sends to another. The purpose of this information exchange is to bridge across long-term and short-term memories [62]; accommodate multi-memory dictionaries [67], mixture of deep and shallow models [68]; facilitate multi-view on the data [34]; and enable learning from mistakes [58].

#### 3.1. Formalization

Built on co-training principle [69], collaborative tracking (co-tracking) provides a framework in which two classifiers exchange their information to promote tracking results and break selflearning loop (Figure 3). In this two-classifier framework [34], the challenging samples for one classifier are labeled by the other one, i.e., if a classifier finds a sample difficult to label, it relies on the other classifier to label it for this frame and similar samples in the future. In this case, we calculate the discrimination score s j <sup>t</sup> as a weighted sum of the two discriminant functions, s j <sup>t</sup> <sup>¼</sup> <sup>P</sup><sup>2</sup> <sup>c</sup>¼<sup>1</sup> <sup>α</sup>ð Þ<sup>c</sup> <sup>t</sup> h x j tjθð Þ<sup>c</sup> t � � where <sup>α</sup>ð Þ<sup>c</sup> <sup>t</sup> denotes the weight of each discriminator <sup>θ</sup>ð Þ<sup>c</sup> <sup>t</sup> , c ¼ 1, 2. At the CLASSIFYING step, the corresponding sample x j <sup>t</sup> is considered as a challenging sample for the cth discriminator when τ<sup>l</sup> < h x j tjθð Þ<sup>c</sup> t � � <sup>&</sup>lt; <sup>τ</sup><sup>u</sup> holds because it locates close to the corresponding discrimination boundary. When one of the two discriminators answered it challenging, the score of the sample is calculated with using the other score:

$$\boldsymbol{s}\_{t}^{j} = \begin{cases} \boldsymbol{a}\_{t}^{(2)} h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(2)}\right) & \text{, } h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(1)}\right) \in (\boldsymbol{\tau}\_{l}, \boldsymbol{\tau}\_{u}) \text{ and } h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(2)}\right) \notin (\boldsymbol{\tau}\_{l}, \boldsymbol{\tau}\_{u}) \\\ \boldsymbol{a}\_{t}^{(1)} h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(1)}\right) & \text{, } h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(2)}\right) \in (\boldsymbol{\tau}\_{l}, \boldsymbol{\tau}\_{u}) \text{ and } h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(1)}\right) \notin (\boldsymbol{\tau}\_{l}, \boldsymbol{\tau}\_{u}) \\\ \sum\_{c=1}^{2} \boldsymbol{a}\_{t}^{(c)} h\left(\mathbf{x}\_{t}^{j} | \boldsymbol{\theta}\_{t}^{(c)}\right) & \text{, otherwise} \end{cases} \tag{5}$$

At the UPDATING step, the weight αð Þ<sup>c</sup> <sup>t</sup> of the discriminator c is adjusted according to the degree of contradiction to the provisional answers that are determined at the ESTIMATION

Figure 3. Collaborative tracking. A detector and an auxiliary classifier trust each other to handle the sample difficult for them to classify.

step by an integration of all the information. Finally, the classifiers are updated using only the samples that they successfully labeled in the previous frame to reflect the latest target changes.

#### 3.2. Evaluation

update) as T0 to emphasize the role of updating. The figure demonstrates that without the model update, the detector cannot reflect the changes in target appearance and lose the target rapidly in most of the scenarios (comparing T0 and T1). However, it is also evident that having a single tracker is not robust against all of the target's variations (in line with [60]) and the

A single detector may have difficulties in distinguishing the target from the background in certain scenarios. In those cases, it is beneficial to consult another detector with higher robustness. These second detector may have complimentary characteristics to the first one or simply may be a more sophisticated detector that trades computational complexity with speed.

Collaborative discriminative trackers utilize classifiers that exchange their information, to achieve more robust tracking. These information exchanges are in the form of queries that one classifier sends to another. The purpose of this information exchange is to bridge across long-term and short-term memories [62]; accommodate multi-memory dictionaries [67], mixture of deep and shallow models [68]; facilitate multi-view on the data [34]; and enable

Built on co-training principle [69], collaborative tracking (co-tracking) provides a framework in which two classifiers exchange their information to promote tracking results and break selflearning loop (Figure 3). In this two-classifier framework [34], the challenging samples for one classifier are labeled by the other one, i.e., if a classifier finds a sample difficult to label, it relies on the other classifier to label it for this frame and similar samples in the future. In this case, we

<sup>t</sup> as a weighted sum of the two discriminant functions,

<sup>t</sup> is considered as a challenging sample

=∈ τ<sup>l</sup> ð Þ ; τ<sup>u</sup>

=∈ τ<sup>l</sup> ð Þ ; τ<sup>u</sup>

< τ<sup>u</sup> holds because it locates close to the

j tjθð Þ<sup>2</sup> t � �

j tjθð Þ<sup>1</sup> t � �

<sup>t</sup> of the discriminator c is adjusted according to the

<sup>t</sup> , c ¼ 1, 2. At

(5)

<sup>t</sup> denotes the weight of each discriminator <sup>θ</sup>ð Þ<sup>c</sup>

∈ τ<sup>l</sup> ð Þ ; τ<sup>u</sup> and h x

∈ τ<sup>l</sup> ð Þ ; τ<sup>u</sup> and h x

j

j

challenging, the score of the sample is calculated with using the other score:

, h x j tjθð Þ<sup>1</sup> t � �

, h x j tjθð Þ<sup>2</sup> t � �

, otherwise

degree of contradiction to the provisional answers that are determined at the ESTIMATION

j tjθð Þ<sup>c</sup> t � �

corresponding discrimination boundary. When one of the two discriminators answered it

where αð Þ<sup>c</sup>

the CLASSIFYING step, the corresponding sample x

performance of T1 is still low.

108 Human-Robot Interaction - Theory and Application

learning from mistakes [58].

calculate the discrimination score s

for the cth discriminator when τ<sup>l</sup> < h x

αð Þ<sup>2</sup> <sup>t</sup> h x j tjθð Þ<sup>2</sup> t � �

0

BBBB@

αð Þ<sup>1</sup> <sup>t</sup> h x j tjθð Þ<sup>1</sup> t � �

P<sup>2</sup> <sup>c</sup>¼<sup>1</sup> <sup>α</sup>ð Þ<sup>c</sup> <sup>t</sup> h x j tjθð Þ<sup>c</sup> t � �

At the UPDATING step, the weight αð Þ<sup>c</sup>

3.1. Formalization

<sup>c</sup>¼<sup>1</sup> <sup>α</sup>ð Þ<sup>c</sup> <sup>t</sup> h x j tjθð Þ<sup>c</sup> t � �

> s j <sup>t</sup> ¼

s j <sup>t</sup> <sup>¼</sup> <sup>P</sup><sup>2</sup>

3. Co-tracking

For this experiment, we selected a naive classifier with complementary properties to the main classifier in the previous section. This classifier is a KNN classifier using HOC and HOG features, trained on the samples trained from the first frame and updated with all the labeled samples by the collaboration of the classifiers. Not being pre-trained, the performance of this auxiliary classifier is poor in the beginning but gradually gets better. The quick classification of the KNN (owning to its kd-tree implementations and lightweight features) and lack of pretraining grant it high speed and generalization which is in contrast to the main detector. However, it should be noted that without being supervised by the main SVM-based detector, this classifier cannot perform well in isolation for tracking task. Figure 5 presents the performance of this auxiliary tracker as T2. As observed in the figure, the performance of the obtained co-tracker (T3) is better than the main detector (T1) and the auxiliary classifier (T2) as a result of co-labeling, data exchange, and co-learning.
