**4. What kinds of objective metrics are needed?**

In the traditional assessment tests video quality is defined as a satisfaction level of end users, thus QoE. This definition can be generalized over different applications in the area of entertainment. The sources of potential quality degradation are located in different parts of the end-to-end video delivery chain. The first group of distortions can be introduced at the time of image acquisition. The most common problems are noise, lack of focus or improper exposure. Other distortions appear as a result of video compression and processing. Problems can also arise when scaling video sequences in the quality, temporal and spatial domains, as well as, for example, when introducing digital watermarks. Then, for transmission over the network, there may be some artifacts caused by packet loss. At the end of the transmission chain, problems may relate to the equipment used to present video sequences.

In the task-based scenario QoE can be substituted with QoR. The definition of QoR will change along with the specific task requirements and cannot be universally defined. Additionally, subjective answers can be strictly classified as correct or incorrect (i.e. a ground truth is available, e.g. a license plate to be recognized). This is in contradiction to the traditional quality assessment case where there is no ground truth regarding quality.

The task-based QoE is substantially different from the traditional QoE by different manners. Firstly, as long as a user is able to perform the task we do not have to care if he/she is happy with the overall quality or not up to the level when watching such quality result in fast fatigue of the viewer. Therefore, question about the overall quality does not make much sense. It

The second difference between QoE and QoR tests are the source sequences. Let us assume that the task is to recognize if a person on the screen is carrying a gun. In this case more than one source sequence is needed since some alternatives have to be provided. Such alternative sequences have to be very carefully prepared since they should differ from the "gun" sequence by only this one detail. It means that lighting, clouds or any objects at the camera view have

The third difference is subjective experiment preparation. In the most traditional QoE experiment a set of parameters of HRC (Hypothetical Reference Circuit) is chosen to produce so called PVS (Processed Video Stream) i.e. a sequence presented to the subjects. A single SRC (Source Reference Circuit) distorted by *n* different HRCs result in generating *n* PVSes. In the QoE all those PVSes are shown to a subject so the impact of HRCs can be analyzed. In case of the QoR such methodology is difficult to use. For example in case of plate recognition if a subject recognizes the plates once, he/she can remember them making the next recognition

Issues of quality measurements for task-based video are partially addressed in the ITU-T P.912 Recommendation "Subjective video quality assessment methods for recognition tasks" (ITU-T, 2008). This Recommendation introduces basic definitions, methods of testing and ways of conducting psycho-physical experiments (e.g. Multiple Choice Method, Single Answer Method, and Timed Task Method), as well as the distinction between Real-Time- and Viewer-Controlled Viewing scenarios. While these concepts have been introduced specifically for task-based video applications in ITU-T P.912, more research is necessary to validate the methods and refine the data analysis methods. In this chapter we present detailed description

In the traditional assessment tests video quality is defined as a satisfaction level of end users, thus QoE. This definition can be generalized over different applications in the area of entertainment. The sources of potential quality degradation are located in different parts of the end-to-end video delivery chain. The first group of distortions can be introduced at the time of image acquisition. The most common problems are noise, lack of focus or improper exposure. Other distortions appear as a result of video compression and processing. Problems can also arise when scaling video sequences in the quality, temporal and spatial domains, as well as, for example, when introducing digital watermarks. Then, for transmission over the network, there may be some artifacts caused by packet loss. At the end of the transmission

In the task-based scenario QoE can be substituted with QoR. The definition of QoR will change along with the specific task requirements and cannot be universally defined. Additionally, subjective answers can be strictly classified as correct or incorrect (i.e. a ground truth is available, e.g. a license plate to be recognized). This is in contradiction to the traditional

for which task-based experiments which methodology can be used.

chain, problems may relate to the equipment used to present video sequences.

quality assessment case where there is no ground truth regarding quality.

**4. What kinds of objective metrics are needed?**

**3. How it influences the subjective experiment?**

obviously changes the subjective quality tests significantly.

to be exactly the same.

questionable.

Because of the above reasons there are additional requirements related to quality metrics. These requirements reflect a specific recognition task but also the viewing scenario. The Real-Time viewing scenario is more similar to the traditional quality assessment tests, although even here additional parameter such as relative target size has to be taken into account. In the case of the Viewer-Controlled viewing scenario additional quality parameters are related to a single shot quality. This is especially important for monitoring objects with a significant velocity. Sharpness of a single video frame (referred to as motion blur) may be a crucial parameter determining the ability to perform a recognition task.

There is one another quality parameter inherent for both viewing scenarios, i.e. source quality of a target. It reflects the ability to perform a given recognition task under the perfect conditions (when additional quality degradation factors do not exist). An example of two similar targets having completely different source quality is two pictures containing car license plate, one taken in a car dealer showroom and one during an off-road race. The second plate may be not only soiled but also blurred due to high velocity of the car. In such a case the license plate source quality is much lower for the second picture what affects significantly recognition ability.

All the additional factors have to be taken into account while assessing QoR for the task-based scenario. The definition of QoR changes between different recognition tasks and requires implementation of dedicated quality metrics.

In the rest of this chapter, as we have already mentioned at the end of Section 1, we would like to review the development of techniques for assessing video surveillance quality. In particular, we introduce a typical usage of task-based video: surveillance video for accurate license plate recognition. Furthermore, we also present the field of task-based video quality assessment from subjective psycho-physical experiments to objective quality models. Example test results and models are provided alongside the descriptions.
