**4. Image retargeting based on saliency maps**

In previous sections, the compression algorithms do not modify the original spatial (frame resolution) and temporal (video length) size of the signal: an obvious idea which drastically compresses an image is of course to decrease its size. This size decrease can be brutal (zoom on a region and the rest of the image is discarded) or softer (the resolution of the context of the region of interest is decreased but not fully discarded). The first approach will of course be more efficient from a compression point of view, but it will fully discard the context of the regions of interest which can be disturbing.

The direct image cropping will be called here "perceptual zoom" while the second approach which will keep some context around the region of interest will be called "anisotropic resolution". Both approaches provide image retargeting. Retargeting is the process of resizing images while minimizing visual distortion and keeping at best the salient content.

Images can be resized (zoomed) according to two families of methods, either taking into account the relevance of the content or not. In the first case, it not only requires to preserve the content but also the structure of the original image Shamir & Sorkine (2009) so the cropping should avoid to be too restrictive.

The second case comprises methods such as letter and pillar boxing, fixed windowing cropping and scaling. These methods are fast but often give poor results. Indeed, fixed windowing cropping can leave relevant contents outside the window, while scaling can engender losses of important information which could eventually lead to an unrecognizable image Liu & Gleicher (2005). Therefore, these classical methods are not mentioned further in this section.

### **4.1 Spatio-temporal visual data repurposing: perceptual zoom**

Human beings are naturally able to perceive interesting areas of an image, as illustrated in the previous sections of this chapter. Zooming in images should therefore focus on such regions of interest.

Images manipulation programs provide tools to manually draw these rectangles of interest, but the process can be automated with the help of attention algorithms presented above in this chapter. Interestingly, such techniques can also be used for real time spatio-temporal images broadcast Chamaret et al. (2010).

Figure 14 shows several perceptual zooms depending on a parameter which will threshold the smoothed Mancas (2009) saliency map.

In the next section, the main general retargeting methods based on content-aware cropping are presented. Then, attention-based retargeting methods are more specifically described.

#### **4.1.1 General retargeting methods**

An interactive cropping method is proposed by Santella et al. (2006), whose purpose is to create aesthetically pleasing crops without explicit interaction. The location of important content of the image is realized by segmentation and by eye-tracking. With a collection of gaze-based crops and an optimization function, the method identifies the best crops.

Another approach is to calculate automatically an "active window" with a predefined size, as presented by Tao et al. (2007). In this case the retargeted image has to satisfy two 16 will be set by intech

In previous sections, the compression algorithms do not modify the original spatial (frame resolution) and temporal (video length) size of the signal: an obvious idea which drastically compresses an image is of course to decrease its size. This size decrease can be brutal (zoom on a region and the rest of the image is discarded) or softer (the resolution of the context of the region of interest is decreased but not fully discarded). The first approach will of course be more efficient from a compression point of view, but it will fully discard the context of the

The direct image cropping will be called here "perceptual zoom" while the second approach which will keep some context around the region of interest will be called "anisotropic resolution". Both approaches provide image retargeting. Retargeting is the process of resizing

Images can be resized (zoomed) according to two families of methods, either taking into account the relevance of the content or not. In the first case, it not only requires to preserve the content but also the structure of the original image Shamir & Sorkine (2009) so the cropping

The second case comprises methods such as letter and pillar boxing, fixed windowing cropping and scaling. These methods are fast but often give poor results. Indeed, fixed windowing cropping can leave relevant contents outside the window, while scaling can engender losses of important information which could eventually lead to an unrecognizable image Liu & Gleicher (2005). Therefore, these classical methods are not mentioned further in

Human beings are naturally able to perceive interesting areas of an image, as illustrated in the previous sections of this chapter. Zooming in images should therefore focus on such regions

Images manipulation programs provide tools to manually draw these rectangles of interest, but the process can be automated with the help of attention algorithms presented above in this chapter. Interestingly, such techniques can also be used for real time spatio-temporal images

Figure 14 shows several perceptual zooms depending on a parameter which will threshold

In the next section, the main general retargeting methods based on content-aware cropping are presented. Then, attention-based retargeting methods are more specifically described.

An interactive cropping method is proposed by Santella et al. (2006), whose purpose is to create aesthetically pleasing crops without explicit interaction. The location of important content of the image is realized by segmentation and by eye-tracking. With a collection of

Another approach is to calculate automatically an "active window" with a predefined size, as presented by Tao et al. (2007). In this case the retargeted image has to satisfy two

gaze-based crops and an optimization function, the method identifies the best crops.

images while minimizing visual distortion and keeping at best the salient content.

**4.1 Spatio-temporal visual data repurposing: perceptual zoom**

**4. Image retargeting based on saliency maps**

regions of interest which can be disturbing.

should avoid to be too restrictive.

broadcast Chamaret et al. (2010).

**4.1.1 General retargeting methods**

the smoothed Mancas (2009) saliency map.

this section.

of interest.

Fig. 14. Example of images along with rectangles providing different attention-based automatic zooms. After a saliency map (Mancas (2009)) is computed and low-pass filtered, several threshold values are used to extract the bounding boxes of the more interesting areas. Depending on this threshold, the zoom is more or less precise/important.

requirements: to preserve temporal and spatial distance, and to contain useful information such as objects shapes and motions. These methods are composed of the following three steps. First, extraction of the image foreground, for example by minimizing an energy function to automatically separate foreground from background pixels. Second, optimization of the active window to fit in the target size. Third, clustering to reduce the number of parameters to be estimated in the optimization process.
