**5.4 GM/LM based text occluded region recovery**

The corresponding block diagram of the proposed GM/LM based text occluded region recovery (TORR) approach is shown in Fig.6. It consists of the following steps.

Fig. 6. Block diagram of the GM/LM based text occluded region recovery (TORR). The input video is with text occluded regions and the output video is with text occluded region recovery.

Fig. 7. Diagram of recovering a pixel in text occluded region of current frame *j* from its previous frame *i* and next frame *k*. The dash lines means the pixel cannot be recovered from its reference frames. The solid lines means the pixels can be recovered from its reference frames.

Global Motion Estimation and Its Applications 97

(a) video frames with detected text lines

(b) video frames after carrying out TORR

sequence *Foxes of the Kalahari*.

Fig. 9. GM/LM based text occluded region recovery for the three frames for the test video

(a) video frames with detected text lines

(b) video frames after carrying out TORR

Fig. 8. GM/LM based text occluded region recovery for the three frames for the test video sequence *News1.* 

(a) video frames with detected text lines

(b) video frames after carrying out TORR

sequence *News1.* 

Fig. 8. GM/LM based text occluded region recovery for the three frames for the test video

(a) video frames with detected text lines

(b) video frames after carrying out TORR

Fig. 9. GM/LM based text occluded region recovery for the three frames for the test video sequence *Foxes of the Kalahari*.

Global Motion Estimation and Its Applications 99

This work is supported in part by National Natural Science Foundations of China (NSFC)

[1] F. Dufaux and J. Konrad, "Efficient, robust, and fast global motion estimation for video coding", IEEE Trans. Image Processing, Vol. 9, No.3, Mar. 2000, pp.497-501. [2] "MPEG-4 Video Verification Model version 18.0", ISO/IEC JTC1/SC29/WG11, 2001. [3] Y. P. Su, M. T. Sun, and V. Hsu, "Global Motion Estimation From Coarsely Sampled

[4] Y. P. Su, M. T. Sun, "A Non-iterative motion vector based global motion estimation

[5] X. Qian, G. Liu, "Global motion estimation from randomly selected motion vector groups

[6] Y. Keller, A. Averbuch, "Fast Gradient Methods Based on Global Motion Estimation for

[7]H. Wang, J. Wang, Q. Liu, H. Lu, Fast progressive model refinement global motion

[8] H. Alzoubi, W. Pan, "very fast global motion estimation using partial data", in Proc.

[9] H. Alzoubi, W. Pan, "efficient global motion estimation using fixed and random

[10] Y. Chen, and I. Bajic´, "Motion vector outlier rejection cascade for global motion estimation,"IEEE Signal Processing Letters, vol. 17, no. 2, 2010, pp. 197-200. [11] M. Tok, A. Glantz, M. Arvanitidou, A. Krutz, and T. Sikora, "Compressed domain

[12] Y. Chen, I. Bajic´, and P. Saeedi, "Motion region segmentation from compressed video

[13] P. Xu, L. Xie, and S. Chang, "Algorithms and systems for segmentation and structure

[14] L. Duan, M. Xu, Q. Tian, C. Xu, and J. S. Jin, "A unified framework for semantic shot

[15] B. Li, J. Errico, H. Pan, and M. Sezan, "Bridging the semantic gap in sports video retrieval and summarization," J. Vis. Commun. Image R. vol.17, pp.393-424, 2004. [16] A. Ekin, and A. Tekalp, "Generic play-break event detection for summarization and

global motion estimation using the Helmholtz tradeoff estimator," in Proc. ICIP

using global motion estimation and Markov random fields," IEEE Transactions on

analysis in soccer video," in Proc. Int. Conf. Multimedia & Expo, 2001, pp. 184-187.

classification in sports video," IEEE Trans. Multimedia, vol.7, No.6, 2005, pp.1066-

hierarchical sports video analysis," in Proc. Int. Conf. Mulmedia and Expo, vol.1,

estimation algorithm with prediction, in proc. ICME 2006.

Multimedia, vol. 13, no. 3, 2011, pp. 421-431.

Motion Vector Field and the Applications", IEEE Trans. Circuits Syst. Video

algorithm", Proc. Int. Conf. Multimedia and Expo, Taipei, Taiwan, June 27-30, 2004,

and GM/LM based applications", signal image and video processing, 2007, pp.179-

Video Compression", IEEE Trans. Circuits Syst. Video Technol., Vol. 13, No. 4, Apr.

No.60903121, No.61173109, and Foundation of Microsoft Research Asia.

Technol., Vol. 15, No. 2, Feb. 2005, pp. 232-242.

**7. Acknowledgement** 

pp.703-706.

2003, pp. 300-309.

ICASSP 2006.

2010,pp.777-780.

2003, pp. 169-172.

1083.

subsampling patterns"

189.

**8. Reference** 


Text occluded region recovery using GM/LM information. From the estimated global motion parameters **m** and the text occluded regions 1 (,) *N i i i TR x y* , where *N* is the total

pixels in the text occluded region of current frame. The corresponding diagram of a pixel in text occluded region is shown in Fig.7. TORR is carried out bi-directional and iteratively. The bi-directional approach means that a pixel in text occluded region of current frame *j* can be recovered by forward previous frame *i* and backward replacement from its next frame *k* (with *i*<*j*<*k*). From Fig.7 we find that the first pixel can be recovered (denoted by the solid lines) from its previous frame *i* and cannot recovered (denoted by the dash lines) from its next frame *k*. However, for the second pixel, its replacement in frame *i* is also in text occluded region. Moreover, its replacement in frame *k* is in local motion region (LMR). So the above two directional replacement are both invalid. Thus iteratively carrying out TORR is needed for the video frame. The iteration stops when all pixels in TORR are recovered. Alternatively, the replacement can be carried out by using more than one frame. It is likely that the second pixel in frame *j* can find correct replacement in its previous frames *i*-*n* or *k*+*n* (with *n*>0).

Fig.8 and Fig.9 show the subjective text occluded region recovery results. The text occluded frames in Fig.8(a) and Fig.9 (a) are from MPEG-7 test video sequences *News*1 and a documentary film of National Geography *Foxes of the Kalahari*. Fig.8 (a) and Fig.9 (a) are the video frames with detected text lines. Fig.8 (b) and Fig.9 (b) show video frames after carrying out TORR using the GM/LM based method. From the recovery results we find that the detail information of the anchorperson is kept well. This further shows the effectiveness of our GM/LM based text occluded recovery method.

#### **6. Conclusion**

In this chapter, a systematic review of the pixel domain based global motion estimation approaches is presented. With respect to its shortcomings in noise filtering and computational cost, the improvement approaches including hierarchical global motion estimation, partial pixel set based global motion estimation and compressed domain based global motion estimation are provided. Four global motion based applications including GMC/LMC in MPEG-4 video coding standard, global motion based sport video shot classification, GM/LM based error concealment and text occluded region recovery are described. The applications show the effectiveness of global motion based approaches.
