**4.3 Partial pixel points based GME**

Just as its name implies, partial pixel points based GME approaches only use sub-set of the whole pixels for estimating global motion parameters. In [6], the subset utilized for GME is selected based on gradient magnitudes information. The top 10% pixels with the largest gradient magnitudes are selected and severed as reliable points for GME. This method divides the whole image into 100 sub-regions and selects the top 10% pixels as feature points which can avoid numerical instability. This subset selection approach reduce the computational cost by reduce the number of pixels at the cost of calculating the gradient image and ranking the gradient of the whole pixels. To further reduce the computational

Global Motion Estimation and Its Applications 89

number of MVs and (1 ) *w i MVNum <sup>i</sup>* is the weighting factor for the *i*-th MB, with {0,1} *wi* . How to reject the outlier motion vectors is also very important to improve global estimation performances [10]. Intuitively, *wi* can be set to be "0" if one of the following three conditions is satisfied: 1) this MB is located in a smooth region (which can be indicated by the standard deviation of the luminance component), 2) the matching error of this MB is large enough (which can be measured by the DC coefficient of the residual component), 3) this MB is intra-coded. Global motion estimation is carried out using the

In this Section, four global motion based applications are illustrated. They are 1) the GMC (global motion compensation) and LMC (local motion compensation) based video coding in MPEG-4 advanced simple profile (ASP), 2) GM and LM based mid-level semantic classification for sport video, 3) GM/LM based video error concealment, and 4) GM/LM

The aim of this part is to illustrate how video compression performances can be improved by utilizing adaptive GMC/LMC mode determination. GMC/LMC based motion compensation mode selection approach in MPEG-4 is given [1], [2]. Global motion estimation and compensation is used in MPEG-4 advanced simple profiles (ASP) to remove the residual information of global motion. Global motion compensation (GMC) is a new coding technology for video compression in MPEG-4 standard. By extracting camera motion, MPEG-4 coder can remove the global motion redundancy from the video. In MPEG-4 ASP, each macro block (MB) can be selected to be coded use GMC or local motion compensation (LMC) adaptively during mode determination. Intuitively, some types of motion, e.g., panning, zooming or rotation; could be described using one set of motion parameters for the entire VOP (video object plane). For example, each MB could potentially have the exact same MV for the panning. GMC allows the encoder to pass one set of motion parameters in the VOP header to describe the motion of all MBs. Additionally MPEG-4

In MPEG-4 Advanced simple profile, the main target of Global Motion Compensation (GMC) is to encode the global motion in a VOP (video object plane) using a small number of parameters. Each MB can be predicted either from the previous VOP by global motion compensation (GMC) using warping parameters or from the previous VOP by local motion compensation (LMC) using local motion vectors as in the classical scheme. The selection is made based on which predictor leads to the lower prediction error. In this Section we only expressed the GMC/LMC mode selection approach. More detail expression for the INTER4V/INTER/field prediction, GMC/LMC, and INTRA/INTER can be found in the Section 18.8.2 GMC prediction and MB type selection [2]. The pseudo-code of GMC/LMC

allows each MB to specify its own MV to be used in place of the global MV.

MBs with their weights set to be "1".

based text occluded region recovery.

**5.1 GMC and LMC based video coding** 

mode decision in MPEG-4 AS is as follows:

if (*SAD* GMC - *P* < *SAD* LMC) then GMC

else LMC

**5. Applications of global motion estimation** 

cost, a random subset selection method was proposed in [4] for GME in fast image-based tracking. Pixel selection can also follow certain fixed subsampling pattern. Alzoubi and Pan apply the subsampling method that combines random and fixed subsampling patterns to global motion estimation [9]. The corresponding combined subsampling patterns can provide significantly improved tradeoffs between motion estimation accuracy and complexity than those achievable by using either fixed or random patterns alone. Wang et al., [7] proposed a fast progressive model refinement algorithm to select the appropriate motion model to describe different camera motions. Based on the correlation of motion model and model parameters between neighbor frames, an intermediate-level model prediction method is utilized.

#### **4.4 Compressed domain based GME**

In video coding standards, the motion estimation algorithms calculate the motions between successive video frames and predict the current frame from previously transmitted frames using the motion information. Hence, the motion vectors have some relationship with the global motion [10]-[12]. A global motion estimation method is proposed based on randomly selected MV groups from motion vector field with adaptive parametric model determination [5]. A non-iterative GME approach is proposed by Su *et al.* by solving a set of exactly-determined matrix equations corresponding to a set of motion vector groups [4]. Each MV group consists of four MVs selected from the MV field by a fixed spatial pattern. The global motion parameters for each of the MV group are obtained by solving the exactlydetermined matrix equation using singular value decomposition (SVD) based pseudoinverse technique. The final global motion parameters are obtained by a weighted histogram-based method. Moreover, a least-square based GME method by coarsely sampled MVs from the input motion vector field is proposed for compressed video sequences [5]. The global motion parameters are optimized by minimizing the fitting error between the input motion vectors and the wrapped ones from estimated global motion parameters. In order to estimate global motions robustly, motion vectors in local motion region, homogeneous region with zero or near-zero amplitude and regions with larger matching errors are rejected.

The objective function of compressed domain based GME approaches is to minimize the weighted mean matching error (MME) of the input motion vectors and the generated ones by virtue of the estimated global motion parameters, which is expressed as follows

$$\text{MME} = \sum\_{i=1}^{\text{MVNum}} w\_i (\text{ex}\_i^2 + \text{ey}\_i^2) \Big/ \sum\_{i=1}^{\text{MVNum}} w\_i \tag{10}$$

$$\begin{cases} e\mathbf{x}\_i = MV\mathbf{x}\_i - \mathbf{x}\_i' + \mathbf{x}\_i \\ e y\_i = MVy\_i - y\_i' + y\_i \end{cases} \tag{11}$$

where (,) *MVx MVy i i* denotes the input MV of the *i*-th macro-block (MB) at the spatial coordinates (,) *i i x y* ,(,) *i i ex ey* denote the errors vector between the decoded MV and the MV generated by the estimated global motion parameters. And let(,) *i i x y* denote the warped coordinates for (,) *i i x y* with respect to the global motion parameters **m**. *MVNum* denotes the

cost, a random subset selection method was proposed in [4] for GME in fast image-based tracking. Pixel selection can also follow certain fixed subsampling pattern. Alzoubi and Pan apply the subsampling method that combines random and fixed subsampling patterns to global motion estimation [9]. The corresponding combined subsampling patterns can provide significantly improved tradeoffs between motion estimation accuracy and complexity than those achievable by using either fixed or random patterns alone. Wang et al., [7] proposed a fast progressive model refinement algorithm to select the appropriate motion model to describe different camera motions. Based on the correlation of motion model and model parameters between neighbor frames, an intermediate-level model

In video coding standards, the motion estimation algorithms calculate the motions between successive video frames and predict the current frame from previously transmitted frames using the motion information. Hence, the motion vectors have some relationship with the global motion [10]-[12]. A global motion estimation method is proposed based on randomly selected MV groups from motion vector field with adaptive parametric model determination [5]. A non-iterative GME approach is proposed by Su *et al.* by solving a set of exactly-determined matrix equations corresponding to a set of motion vector groups [4]. Each MV group consists of four MVs selected from the MV field by a fixed spatial pattern. The global motion parameters for each of the MV group are obtained by solving the exactlydetermined matrix equation using singular value decomposition (SVD) based pseudoinverse technique. The final global motion parameters are obtained by a weighted histogram-based method. Moreover, a least-square based GME method by coarsely sampled MVs from the input motion vector field is proposed for compressed video sequences [5]. The global motion parameters are optimized by minimizing the fitting error between the input motion vectors and the wrapped ones from estimated global motion parameters. In order to estimate global motions robustly, motion vectors in local motion region, homogeneous region with zero or near-zero amplitude and regions with larger matching

The objective function of compressed domain based GME approaches is to minimize the weighted mean matching error (MME) of the input motion vectors and the generated ones

> 2 2 1 1 ( ) *MVNum MVNum*

*i i MME w ex ey w* 

> *i iii i iii ex MVx x x ey MVy y y*

where (,) *MVx MVy i i* denotes the input MV of the *i*-th macro-block (MB) at the spatial coordinates (,) *i i x y* ,(,) *i i ex ey* denote the errors vector between the decoded MV and the MV generated by the estimated global motion parameters. And let(,) *i i x y* denote the warped coordinates for (,) *i i x y* with respect to the global motion parameters **m**. *MVNum* denotes the

*ii i i*

(10)

(11)

by virtue of the estimated global motion parameters, which is expressed as follows

prediction method is utilized.

errors are rejected.

**4.4 Compressed domain based GME** 

number of MVs and (1 ) *w i MVNum <sup>i</sup>* is the weighting factor for the *i*-th MB, with {0,1} *wi* . How to reject the outlier motion vectors is also very important to improve global estimation performances [10]. Intuitively, *wi* can be set to be "0" if one of the following three conditions is satisfied: 1) this MB is located in a smooth region (which can be indicated by the standard deviation of the luminance component), 2) the matching error of this MB is large enough (which can be measured by the DC coefficient of the residual component), 3) this MB is intra-coded. Global motion estimation is carried out using the MBs with their weights set to be "1".
