**4.2 Description of requirements**

Once the decision of developing subjective quality assessment made, requirements to complete a suitable subjective study must be consider. Before developing the study the conditions must be established to facilitate the repetition of it or the comparison with similar studies. Requirements will be listed in this section.

Viewing conditions.

There are different viewing conditions in case of referring to a laboratory environment than the ones used in home environment. Assuring the best conditions is a basic aim in order to allow a meaningful evaluation to be made derived from the assessment. The distance to the screen of the observers depends on the size of the screen. This is a very important aim to obtain suitable results.

The user's opinion in video technologies is important. For that reason, the evaluation of a video compression codec must be developed with its help, in order to know the level of

Several studies have been developed to measure the quality of new technology systems every period. In first instance, it was used to analyze the impact of digital television in the transition from analogue to digital. And in the transition from standard definition to highdefinition TV, including the studies of main encoders (MPEG-2, H.264/AVC) in the adaptation to different formats and storage, last time assessment was developed for blu-ray codecs. Also for the Definition of settings or quality parameters: bitrate, resolution (e.g. in

Next generation of encoding codecs HVC (High-efficiency Video Codec) which represents the evolution of H.264 is being analyzed by subjective assessment, to measure the impact on

So, it is clear that subjective quality evaluation is still important and it will always be together with the objective one. Methods and technologies change, but the purpose of video

In this section, the most frequent techniques to develop a subjective quality study, in order to evaluate a determined system. There is a great variety of subjective testing methods,

Then, there are single or double stimulus experiments, depending on if the original image is

The problem with this kind of studies is the requirement for a large number of observers viewing a limited amount of video contents, because if the time of contents viewed is extended for two long, the tiredness and fatigue of the observers will affect to the wished measurement. But it is still very used nowadays and will be necessary to introduce the

Once the decision of developing subjective quality assessment made, requirements to complete a suitable subjective study must be consider. Before developing the study the conditions must be established to facilitate the repetition of it or the comparison with similar

There are different viewing conditions in case of referring to a laboratory environment than the ones used in home environment. Assuring the best conditions is a basic aim in order to allow a meaningful evaluation to be made derived from the assessment. The distance to the screen of the observers depends on the size of the screen. This is a very

depending on the way of presenting the video sequences to the observers.

**4. Subjective quality assessment** 

HD: 720p or 1080i) or features for a TV channel, for example.

users and the necessity of its establishment.

presented or only the degraded one.

**4.2 Description of requirements** 

Viewing conditions.

objective assessment to explain in next section.

studies. Requirements will be listed in this section.

important aim to obtain suitable results.

quality is still the same.

**4.1 Introduction** 

acceptance of it.

The screen must be arranged in contrast and brightness and the angle of observation must not exceed 30º.

Conditioning of the room.

The room must be conditioned in lighting, with seat comfort and no reflections on screen. Because the viewers will spend a considerable time and this must not affect to their scores.

Observers

The minimum number of participants in the study, for the sample to be enough is 15, but it is recommended to find as many observers as possible, 60 or more. The viewers are preferred to be non-professional, possessing no trained-eyes. Professional viewers tend to search for the impairment and their opinion is not always so impartial.

There must be a representative variety of age and sex in viewers. The observers must make a previous training to understand the objective of the test.

Materials / Test sequences.

There must be a selection of sufficient test material, including different type of contents emitted by a conventional TV channel (sports, movies, news, documentary, etc.), and with different settings of spatial (more or less level of detail and high frequencies) and temporal (faster or slower contents) complexity.

Each sequence must have a duration of between 10 and 20 seconds due to human memory and perception, to assure a correct viewing, neither too short than the viewers do not have time to observe the image in detail, nor too long to avoid causing fatigue in the observer.

A number of organizations have developed test still pictures and sequences, whose use is recommended for the assessment.

Presentation of results

The results must be presented in detail. All information is necessary to validate the study and verify its good performance. Data given must include: details of the test configuration and materials, type of source and displays, number of subjects or observers that participated, reference system used and its specific variations, scores and mean scores adjusted to 95% confidence interval.

Logistic curve-fitting and logarithmic axis will allow a straight line representation, which is the preferred form of presentation, as legible as possible.

Sound.

Audio quality assessment is preferably developed independently of the video assessment. In fact, it is recommended not to use sound or audio in video studies, in order to avoid distractions in the observer, modifying their opinion, off target.

Furthermore, in case of using it, selection of the accompanying audio material should be considered at the same level of importance as the selection of video material.

Video Quality Assessment 137

methods are more related to double stimulus methods, although it is possible to obtain

First type of methods utilizes indistinctly the quality or the impairment scale, while the second type utilizes a scale called "comparison scale", which assigns the relation between

There are different options of combining video in a continuous war: one program or a series of sequences of different or the same type of content. These programs may include one or various quality parameters under evaluation (e.g. bitrate). Each program should have

The time of response of the viewer must be fast to identify the impairment observed. Nevertheless, the varying delay may influence the assessment results if only the average over a program segment is calculated. Studies are being carried out to evaluate the impact of

There are different types of scale, depending on desirable results that the researcher

This scale is used in different methods to evaluate in absolute the perceived quality

expects to obtain from the study. The four most representatives appear below.

of a video sequence. There are variations with different number of grades.

5 Excellent 4 Good 3 Fair 2 Poor 1 Bad

absolute measurements with full reference.

the members of each pair of sequences.

duration of at least 5 minutes

Type of scale

1. Quality Scale (QS)

Continuous or discrete (non-continuous) evaluation methods

the response time of different viewers on the resulting quality grade.

Fig. 5. Example of data from a continuous assessment

Additionally, it is important that the sound be synchronized with the video. This is most noticeable for speech and lip synchronization, for which time lags of more than approximately 100 ms are considered very annoying.
