**3. Tasks to be solved**

The functional and requirement analysis of the methods will be worked out as well as the standardization task. This work should ensure that the developed tools meet the requirements of the end-users and the police forces, and it is therefore considered as a key work package.

For each of the considered multimedia resources, one considers the special needs related to the automated processing of the specific data. Therefore, five tasks are related to the treatment of images and videos, text, handwriting and audio and speech signals. Feature extraction methods are considered for all multimedia sources in work package. This strategy guarantees the synergy that can be obtained in quality improvement for processing when using information from different media types. The reasoning unit is considered in a single work package. All developed method will be linked to the CBR system. They provide a case description for each new case to the CBR system.

Novelty detection has aspects that go beyond CBR. Therefore, the development of the novelty detection unit is left to a single task.

The legal aspects are worked out in a special task. It should run over the whole project with the aim to ensure that each RTD task meets the legal aspects as well as to identify new aspects that arise when developing novel techniques for the analysis of forensic data.

All the proposed methods should be integrated into a single reasoning system. After this has been done, the final evaluation of the system will be done. The final system will be demonstrated to police forces and end users.

The proposed methodology of the work is shown in **Figure 1**.

### **3.1 Video and image enhancement, filtering and assessment**

The goal of this package is to improve the quality of an image or video sequence in order to support easy classification or recognition tasks and to detect in images traces of tampering without using protecting pre-extracted or pre-embedded information. Fragile watermarking schemes to protect legal evidence audio and speech data will also be developed.

In many image modalities, synthetic aperture radar (SAR) images, passive millimeter wave (PMMW) images, commercial photography and images and videos acquired for security purposes, the captured images and video represent a degraded version of the original scene. The observed degraded image is usually the result of convolving the original image (to be estimated) with a known or unknown blurring function and the addition of noise. The removal of noise and blur from the observation in order to obtain a good estimate of the original image is the goal of deconvolution and blind deconvolution techniques. In this task, we will develop and utilize Bayesian deconvolution and blind deconvolution techniques to improve the quality of the observed images. The improved images will either be used by humans for identification or be the input to classification and recognition methods.

are acquired either by multiple sensors imaging a single scene or by a single sensor imaging a scene over a period of time. In motion free SR images are upsampled by learning the relationship between corresponding low resolution and high-resolution image patches in a database and combining this learning process with the observed LR image. The improved images will either be used by humans for identification or

In order to trust the information extracted from images and videos, it is necessary to make sure that the image and video have been recorded by a camera and that no artifact has been added or object removed. The trustworthiness of images and videos has clearly an essential role in many security areas, including forensic investigation, criminal investigation, surveillance systems, and intelligence services.

be the input to classification and recognition methods.

*Novel Methods for Forensic Multimedia Data Analysis: Part II*

*DOI: http://dx.doi.org/10.5772/intechopen.92548*

*3.1.2 Blind methods for detecting image forgery*

**Figure 1.** *Methodology.*

**139**

### *3.1.1 Image and video super resolution*

We use the term super resolution (SR) to describe the process of obtaining a high-resolution image or a sequence of high-resolution images from a set of lowresolution (LR) observations. In this task we will utilize and develop motion-based spatial super resolution techniques as well as motion free super resolution techniques. In motion-based SR, the LR observed images are under-sampled, and they

*Novel Methods for Forensic Multimedia Data Analysis: Part II DOI: http://dx.doi.org/10.5772/intechopen.92548*

**3. Tasks to be solved**

*Digital Forensic Science*

considered as a key work package.

each new case to the CBR system.

of forensic data.

data will also be developed.

*3.1.1 Image and video super resolution*

**138**

of the novelty detection unit is left to a single task.

system will be demonstrated to police forces and end users.

**3.1 Video and image enhancement, filtering and assessment**

The proposed methodology of the work is shown in **Figure 1**.

The functional and requirement analysis of the methods will be worked out as well as the standardization task. This work should ensure that the developed tools meet the requirements of the end-users and the police forces, and it is therefore

For each of the considered multimedia resources, one considers the special needs related to the automated processing of the specific data. Therefore, five tasks are related to the treatment of images and videos, text, handwriting and audio and speech signals. Feature extraction methods are considered for all multimedia sources in work package. This strategy guarantees the synergy that can be obtained in quality improvement for processing when using information from different media types. The reasoning unit is considered in a single work package. All developed method will be linked to the CBR system. They provide a case description for

Novelty detection has aspects that go beyond CBR. Therefore, the development

The legal aspects are worked out in a special task. It should run over the whole project with the aim to ensure that each RTD task meets the legal aspects as well as to identify new aspects that arise when developing novel techniques for the analysis

All the proposed methods should be integrated into a single reasoning system. After this has been done, the final evaluation of the system will be done. The final

The goal of this package is to improve the quality of an image or video sequence in order to support easy classification or recognition tasks and to detect in images traces of tampering without using protecting pre-extracted or pre-embedded information. Fragile watermarking schemes to protect legal evidence audio and speech

In many image modalities, synthetic aperture radar (SAR) images, passive millimeter wave (PMMW) images, commercial photography and images and videos acquired for security purposes, the captured images and video represent a degraded version of the original scene. The observed degraded image is usually the result of convolving the original image (to be estimated) with a known or unknown blurring function and the addition of noise. The removal of noise and blur from the observation in order to obtain a good estimate of the original image is the goal of

deconvolution and blind deconvolution techniques. In this task, we will develop and utilize Bayesian deconvolution and blind deconvolution techniques to improve the quality of the observed images. The improved images will either be used by humans

We use the term super resolution (SR) to describe the process of obtaining a high-resolution image or a sequence of high-resolution images from a set of lowresolution (LR) observations. In this task we will utilize and develop motion-based spatial super resolution techniques as well as motion free super resolution techniques. In motion-based SR, the LR observed images are under-sampled, and they

for identification or be the input to classification and recognition methods.

are acquired either by multiple sensors imaging a single scene or by a single sensor imaging a scene over a period of time. In motion free SR images are upsampled by learning the relationship between corresponding low resolution and high-resolution image patches in a database and combining this learning process with the observed LR image. The improved images will either be used by humans for identification or be the input to classification and recognition methods.

### *3.1.2 Blind methods for detecting image forgery*

In order to trust the information extracted from images and videos, it is necessary to make sure that the image and video have been recorded by a camera and that no artifact has been added or object removed. The trustworthiness of images and videos has clearly an essential role in many security areas, including forensic investigation, criminal investigation, surveillance systems, and intelligence services.

In this task we will utilize and develop blind methods for detecting image forgery, that is, methods that use the image function to perform the forgery detection task. The methods will try to identify various traces of tampering and detect them separately. The final decision about the forgery will be carried out by fusion of results of separate detectors.

• Development and implementation of a taxonomy over the similarity measures As an outcome of these tasks, a taxonomy over similarities will be developed. This taxonomy will be represented by a hierarchical concept, and the user will be guided through this taxonomy for his special needs. A conversational strategy will

The indexing structure over the different data types will be developed and

• Development and implementation of meta-learning methods over image and

The preprocessing and feature extraction methods developed will be evaluated, and it will be decided where meta-learning has to be applied. The architecture and CBR methods for meta-learning will be defined. The meta-learning architecture that fits the CBR system will be developed. The meta-learning algorithm will be

• Development and implementation of a generalization mechanism over cases,

The methods for meta-learning over case, case-classes and higher-order classes will be defined for the different multimedia data sources. The interaction between these different multimedia-representations will be studied, and methods that can

A recent learning mechanism will be studied, and based on that a new learning mechanism will be developed for feature weighting of the different data types and deciding about the correct similarity measure. These strategies will be implemented

• Development and implementation of a life cycle and maintenance function for

The life cycle of a multimedia CBR system will be studied based on the experience the partners have with different data sources. A life cycle and maintenance function that can take this into account will be developed and

This work package will investigate, define and evaluate feature extraction methods to detect, describe and relate the multimedia data content relevant to forensic activities. The activities will concentrate on different biometric parameters that characterize individuals in terms of appearance, behavior, voice and handwrit-

Features pertaining to face characteristics, distinctive individual marks, morphometric measurements of the body and gait analysis will be the subject of investigation for videos and images. Similarly, the exploration will regard distinctive features from audio signals for speaker's identification and recognition as well as text analysis to extract information pertinent to the forensic investigations.

ings, so as to enable the process of detection and recognition.

be developed that helps the user to figure out what his needs are.

*Novel Methods for Forensic Multimedia Data Analysis: Part II*

*DOI: http://dx.doi.org/10.5772/intechopen.92548*

• Development and implementation of an indexing structure

implemented.

video processing chain

implemented and evaluated.

case-classes and higher-order classes

improve the performance will be developed.

into the CBR system and evaluated.

**3.3 Multimedia feature extraction**

implemented.

**141**

the case-based reasoning system

• Development of a learning mechanism over similarities
