**5.2 Full-reference metrics**

In this section, the most representative FR metrics are defined, classified in four different groups of metrics: based on statistics and pixel difference, based on structural similarity, based on artifact detection, and based on vision models.

Pixel-based metrics

Most of the quality metrics are full-reference, especially the pixel-based (or statistics-based) metrics, focused on comparing pixel-by-pixel the difference between the original image used as a reference and the impaired image.

These metrics offer a good estimation of the global quality measured objectively, but are widely criticized for not correlating well with the perceived quality measurement obtained by subjective methods, that incorporates human vision models.

The most important are among others: MSE, SNR and PSNR.

#### **MSE (Mean Square Error)**

The mean squared error (MSE) is one the most popular difference metrics in image and video processing. The MSE is the mean of the squared differences between the luminance level values of pixels between two images (X and Y), normally the original frame used as a reference and an impaired image, obtained by processing of the first image. M and N are the horizontal and vertical dimensions of each frame of the sequence.

$$MSE = \frac{1}{M \cdot N} \sum\_{i=0, j=0}^{M-1} \left( X\_i - Y\_i \right)^2$$

#### **SNR (Single-To-Noise Ratio)**

Also based on pixel by pixel comparison, this metric measures the relation between the original image and the degraded image, in order to evaluate the degradation on image.

SNR was widely substituted by its evolution PSNR, because it offers a higher efficiency than SNR, and a global extension more easy to compare in studies with different signals.

$$PSNR = 10\log\_{10}\left(\frac{\sum\_{i=0, j=0}^{M-1, N-1} \left|X\_i\right|^2}{\sum\_{\substack{M-1, N-1\\i=0, j=0}}^{} \left(X\_i - Y\_i\right)^2}\right) \text{ [dB]}$$

In this section, the most representative FR metrics are defined, classified in four different groups of metrics: based on statistics and pixel difference, based on structural similarity,

Most of the quality metrics are full-reference, especially the pixel-based (or statistics-based) metrics, focused on comparing pixel-by-pixel the difference between the original image used

These metrics offer a good estimation of the global quality measured objectively, but are widely criticized for not correlating well with the perceived quality measurement obtained

The mean squared error (MSE) is one the most popular difference metrics in image and video processing. The MSE is the mean of the squared differences between the luminance level values of pixels between two images (X and Y), normally the original frame used as a reference and an impaired image, obtained by processing of the first image. M and N are the

0, 0

*i j MSE X Y M N*

Also based on pixel by pixel comparison, this metric measures the relation between the original image and the degraded image, in order to evaluate the degradation on image.

SNR was widely substituted by its evolution PSNR, because it offers a higher efficiency than

SNR, and a global extension more easy to compare in studies with different signals.

10log

*PSNR*

 

1 *M N*

 1, 1 <sup>2</sup>

 

*X Y*

*i i*

*i*

[dB]

*X*

1, 1 <sup>2</sup>

 

0, 0 <sup>10</sup> 1, 1 <sup>2</sup>

*M N*

*i j M N*

 

0, 0

*i j*

*i i*

such as mobile or internet multimedia services.

based on artifact detection, and based on vision models.

by subjective methods, that incorporates human vision models. The most important are among others: MSE, SNR and PSNR.

horizontal and vertical dimensions of each frame of the sequence.

Fig. 13. No Reference (NR) metric diagram

as a reference and the impaired image.

**5.2 Full-reference metrics** 

Pixel-based metrics

**MSE (Mean Square Error)** 

**SNR (Single-To-Noise Ratio)** 

are of vital importance in environments which are difficult to provide any reference,

#### **PSNR (Peak Single-To-Noise Ratio)**

As SNR used the same signal to compare with, it is more difficult to export the conclusion from one study to another, that is why the original signal was changed by the peak value (255 in a RGB channel, or 240 in luminance, for example) to obtain more general results, with a more efficient method.

$$PSNR = 10\log\_{10}\left(\frac{\sum\_{i=0, j=0}^{M-1, N-1} Max^2}{\sum\_{i=0, j=0}^{M-1, N-1} \left(X\_i - Y\_i\right)^2}\right) \text{ [dB]}$$

Based on artifacts

A collection of metrics attempt to assess the effects of artifacts described on section 0, instead of offering a global idea of quality, as with pixel-based metrics. Some examples of most representative metrics are introduced next.

#### **Blockiness or tiling**

The metric defined by MSU Graphics & Media Lab measures subjective blocking effect in video sequence, based on energy calculation and gradients. In contrast areas of the frame blocking is not appreciable, but in smooth areas these edges are conspicuous.

Other metrics are based on the structure of the pixelized image. The model included in Lee et al. research, first extracts edge pixels and computes horizontal ( H(t,i, j) ) and vertical (V (t,i, j) ) gradient component of the edge pixels. The gradient is calculated using the Sobel operators. From the horizontal and vertical gradient images, the magnitude (R) and angle () are extracted:

$$R(t\_{\prime}i\_{\prime}j) = \sqrt{H(t\_{\prime}i\_{\prime}j)^2 + V(t\_{\prime}i\_{\prime}j)^2}$$

$$\theta(t\_{\prime}i\_{\prime}j) = \tan^{-1}\left[\frac{V(t\_{\prime}i\_{\prime}j)}{H(t\_{\prime}i\_{\prime}j)}\right]$$

Analyzing the angles of the gradient, the pixels with gradient parallel to the picture frame are considered as belonging to blocking region, if they have a determined magnitude. Comparing to the original image, errors are avoided due to real edge pixels.

Other interesting metric on this field are the ones by Winkler et al., 2001 and Wang et al., 2002.

#### **Blurring**

Blurring metrics are based on the analysis of energy in high frequencies and analysis of edges and their spread. As it proposes Marziliano et al. in 2004. The reduction of edge energy between the original and the impaired image shows the loss of quality due to blurring artifact.

Other metrics, such as proposed in Lee et al. Research, utilizes the gradient calculated in every pixel of the image to detect the blur artifact by analyzing the diminution of this

Video Quality Assessment 145

of disparity, to detect the image variation.

and masking.

**5.3 No-reference metrics** 

metrics to evaluate the degradation.

**Blocking Effect or Blockiness** 

by evaluating the blocky signal.

compressed in MPEG-2.

**Blur** 

quantization.

groups, depending on the artifact characterized.

use of deblocking filters in H.246 and other encoders.

Daly. The model is based on the comparison of two images after creating a diagram

 Moving Picture Quality Metric (MPQM). As PSNR does not take the visual masking phenomenon into consideration, every single pixel error contributes to the decrease of the PSNR, even if this error is not perceived. This method includes characteristics of Human Visual System intensively studied: contrast sensitivity

 Perceptual Distortion Metric (PDM). This model of vision was developed by Winkler, S. Based on the HVS, allowing the system to find similarities with the human eye. The structure of the model is based on the fact of finding the optimus components of the model, modifying both the reference and impaired image.

When the reference is not available to design the objective quality method, for example in environments such as internet or video mobile, then it is necessary to utilize no-reference

Most of the times, these kind of studies are focus on analyzing impairments due to artifacts, that degrade the perception of the user. So, the no-reference metrics are distributed in

Most existing no-reference metrics focus on estimating blockiness, which is still relatively easy to detect due to its regular structure, although in practice that is not so easy due to the

Different techniques are used, such as Wu and Yuen whose research in developing a NR metric based on measuring the horizontal and vertical differences between rows and columns at block boundaries, offers interesting results. Means and standard deviations of

On the other hand, Wang et al. model the blocky image as a non-blocky image an then appear the interference with a pure blocky signal. The level of blockiness artifact is detected

Other alternatives are the approach proposed by Baroncini and Pierotti, with the use of multiple filters which extract significant vertical and horizontal edge segments due to blockiness, and also Vlachos proposed an algorithm based on the cross-correlation of subsampled images, and Tan and Ghanbari a metric for blocking detection based in videos

Another typical artifact defined for no-reference metrics is blurring. It appears in almost all processing phases in communications chain of production. Blurring manifests as a loss of spatial detail in moderate to high spatial activity regions of images. Blurring is directly related to the suppression of the higher order AC DCT coefficients through coarse

the blocks adjacent to each boundary determined masking effect pondering.

magnitude between the original and the impaired image. SI is the root mean square of the spatial gradient (SG), so blurring is computed as follows:

$$BL(\mathbf{x}) = \frac{1}{\sqrt{F\mathbf{x}\mathbf{C}}} \left( \sqrt{\sum\_{i=0}^{F-1} \sum\_{j=0}^{C-1} SG\_{ref}^2(i, j)} - \sqrt{\sum\_{i=0}^{F-1} \sum\_{j=0}^{C-1} SG\_{impurity}^2} \right)$$
