3. Low-level feature extraction

Choosing appropriate aesthetic features in photos is essential for distinguishing professional and nonprofessional photos because it helps to predict whether a photo is favorable. Next, we introduce several types of aesthetic features as well as illustrate them. In this section, we focus on low-level feature extraction to measure some elements for the perception of beauty. Such low-level features include color component, sharpness, brightness, contrast, saturation, color balance, colorfulness, and simplicity.

#### 3.1. Color component

scores as Figure 2(a) shows. In the example-based option, the user selects a number of interested photos, and the system extracts the features of these selected photos to produce weights for features automatically as shown in Figure 2(b). The authors also proposed some new

In 2012, an intelligent photographing interface with on-device aesthetic quality assessment system is proposed [3], which makes use of five aesthetics perspectives of photography, such as saturation, color, composition, contrast, and richness. The aesthetic quality assessment system works on a tablet with a camera and runs in real time. Figure 3 graphically shows the system, where Figure 3(a) is the overall rating of features in a photo, while Figure 3(b) is a

In consideration of the subjectivity, a digital photo challenge (DPC) platform is established. The platform allows experts to rate a photo at one of 10 aesthetic quality levels, from good to bad. Figure 4 illustrates some photos in the database for example. In Figure 4(a), the left photo is focused on the flowers successfully, and its theme is harmonic, which makes a comfortable feeling. The color of the right photo is also harmonic and looks comfortable. However, in Figure 4(b), two photos are out of focus, with messy colors and motion blur. Most people

Figure 3. An instant aesthetics quality assessment system: (a) five aesthetics perspectives of photography; (b) an assess-

Figure 4. Photos ranked by professionals in a DPC platform: (a) high-score photos; (b) low-score photos.

would agree that photos in Figure 4(a) are better than those of Figure 4(b).

features for photo ranking.

100 Perception of Beauty

working screenshot of the system.

ment example of the system.

The color component feature is acquired from extracting the levels of specific colors in a photo [1]. Figure 6 shows photos with different major color components.

Figure 6. Three photos with different major color components: (a) blue; (b) green; (c) red.

For accuracy, we choose the color component in the CIELab color space to measure. To achieve this, a photo in the RGB color space is necessary to be converted into the CIEXYZ color space first. The conversion matrix is expressed as follows:

$$
\begin{bmatrix} X \\ Y \\ Z \end{bmatrix} = \begin{bmatrix} 0.412453 & 0.357580 & 0.180423 \\ 0.212671 & 0.715160 & 0.072169 \\ 0.019334 & 0.119193 & 0.950227 \end{bmatrix} \begin{bmatrix} R \\ G \\ B \end{bmatrix} \tag{1}
$$

After the photo is transformed into the CIEXYZ color space, it can be then transformed into the CIELab color space by the relation between these two color spaces depicted below

$$L^\* = \begin{cases} 116 \times \left(\frac{Y}{Y\_n}\right)^{\frac{1}{5}} - 16, & \frac{Y}{Y\_n} > 0.008856\\ & \text{ } 903.3 \times \frac{Y}{Y\_n}, \text{ otherwise} \end{cases} \tag{2}$$

$$a^\* = 500 \left[ f\left(\frac{X}{X\_n}\right) - f\left(\frac{Y}{Y\_n}\right) \right]$$

$$b^\* = 200 \left[ f\left(\frac{Y}{Y\_n}\right) - f\left(\frac{Z}{Z\_n}\right) \right]$$

The color component is extracted by the following equation:

$$f\_{\text{colorор{ратор ион}}} = \sum\_{x=1}^{\text{width }h} \sum\_{y=1}^{h \text{ig} \, ht} \frac{D(\mathbf{c}\_{\text{l}} \cdot \mathbf{c}(x, y))}{\text{width} \times h \, \text{ig} \, ht} \tag{3}$$

where width and height are the dimensions of the photo, and D(cl, c(x, y)) is the Euclidean distance between two colors in the CIELab color space, cl is the color component we want to extract in the CIELab color space, and c(x, y) is the color value of coordinate (x, y) in the same color space.

#### 3.2. Sharpness

A blurry photo is almost worse than a sharp photo of the same scene. However, a partially blurred photo is not necessarily unfavorable because the blur may be produced from background defocus using high-end cameras. Figure 7 shows two photos with different degrees of On the Design of a Photo Beauty Measurement Mechanism Based on Image Composition and Machine Learning http://dx.doi.org/10.5772/intechopen.69502 103

Figure 7. Two photos with different degrees of sharpness and blur: (a) a sharp photo; (b) a photo with background defocus.

sharpness and blur in which Figure 7(a) has high sharpness whereas Figure 7(b) has more blurred regions.

A quality measurement for the sharpness of a photo is stated as follows:

$$f\_{\text{sharpness}} = \frac{|\{(u, v) || F(u, v) | > \xi\}|}{\text{width} \times height} \propto \frac{1}{\sigma} \tag{4}$$

$$f\_{\text{blur}} \propto \frac{1}{f\_{\text{sharpness}}} \tag{5}$$

where Iblur = G<sup>σ</sup> ∗ I is the blurred photo derived through convolving the original photo I with a Gaussian filter Gσ, σ is its standard deviation, and F(u, v) = FFT (Iblur(x, y)) is the blurred photo transformed into the frequency domain via the fast Fourier transform. Here, ξ is set to 5.

#### 3.3. Brightness

For accuracy, we choose the color component in the CIELab color space to measure. To achieve this, a photo in the RGB color space is necessary to be converted into the CIEXYZ color space

> 0:412453 0:357580 0:180423 0:212671 0:715160 0:072169 0:019334 0:119193 0:950227

After the photo is transformed into the CIEXYZ color space, it can be then transformed into the

� <sup>16</sup>, <sup>Y</sup> Yn

> Y Yn

Xn � �

Yn � �

width X x¼1

where width and height are the dimensions of the photo, and D(cl, c(x, y)) is the Euclidean distance between two colors in the CIELab color space, cl is the color component we want to extract in the CIELab color space, and c(x, y) is the color value of coordinate (x, y) in the same

A blurry photo is almost worse than a sharp photo of the same scene. However, a partially blurred photo is not necessarily unfavorable because the blur may be produced from background defocus using high-end cameras. Figure 7 shows two photos with different degrees of

height X y¼1

CIELab color space by the relation between these two color spaces depicted below

903:3 �

<sup>116</sup> � <sup>Y</sup> Yn � �<sup>1</sup> 3

<sup>a</sup>� <sup>¼</sup> <sup>500</sup> <sup>f</sup> <sup>X</sup>

<sup>b</sup>� <sup>¼</sup> <sup>200</sup> <sup>f</sup> <sup>Y</sup>

3 5

> 0:008856

, otherwise

� <sup>f</sup> <sup>Y</sup> Yn

� <sup>f</sup> <sup>Z</sup> Zn

Dðcl, cðx, yÞÞ

width � height <sup>ð</sup>3<sup>Þ</sup>

� � � �

� � � �

2 4

R G B 3

5 ð1Þ

ð2Þ

first. The conversion matrix is expressed as follows:

X Y Z 3 5 ¼

L� ¼

The color component is extracted by the following equation:

color space.

102 Perception of Beauty

3.2. Sharpness

f colorcomponent ¼

2 4

Figure 6. Three photos with different major color components: (a) blue; (b) green; (c) red.

8 >>><

>>>:

2 4

> For an input photo, the global brightness can be obtained from various kinds of methods, including software and hardware measurements. In a software method, the global brightness can be calculated by the use of the mean or median of all pixel values. As shown in Figure 8, applying two brightness settings to the same scene yields quite different effects for viewers. Figure 8(a) is a brighter version, while Figure 8(b) is a darker version of the same scene.

Figure 8. Two photos with different degrees of brightness: (a) brighter; (b) darker.

For an input photo, the global brightness can be derived from the following equation:

$$f\_{\text{brightness}} = \frac{\sum\_{\mathbf{x}=1}^{width} \sum\_{\mathbf{y}=1}^{height} I(\mathbf{x}, \mathbf{y})}{\text{width} \times \text{height}} \tag{6}$$

where I(x, y) is the intensity of a pixel at (x, y).

#### 3.4. Contrast

Color contrast is essential for photo quality measurement because better cameras produce better color contrast. The comparison of different degrees of color contrast is shown in Figure 9, where Figure 9(a) has both bright areas and dark areas with various colors, while Figure 9(b) has only dim white colors.

The color contrast feature is defined as

$$f\_{\text{contrast}} = \sum\_{i=1}^{n-1} \sum\_{j=i+1}^{n} (1 - d(i,j)) \frac{D(i,j)}{A\_i A\_j} \tag{7}$$

where d(i, j) is the spatial distance between the centroids of two segmentations Ai and Aj; D(i, j) is the color distance between the two segmentations in the CIELab color space.

#### 3.5. Saturation

Appealing photos usually have a higher saturation degree. Figure 10 shows an example for comparison in which Figure 10(a) has more vivid colors whereas most pixels in Figure 10(b) appear pale and white.

The color saturation feature is defined as

$$f\_{\text{saturation}} = \sum\_{\mathbf{x}=1}^{width} \sum\_{y=1}^{height} \frac{s(\mathbf{x}, y)}{\mathbf{width} \times height} \tag{8}$$

where s(x, y) is the saturation of a pixel in the "hue", "saturation", and "value" (HSV) color space.

Figure 9. Two photos with different degrees of contrast: (a) more contrast; (b) less contrast.

On the Design of a Photo Beauty Measurement Mechanism Based on Image Composition and Machine Learning http://dx.doi.org/10.5772/intechopen.69502 105

Figure 10. Two photos with different degrees of saturation: (a) more saturated; (b) less saturated.

#### 3.6. Color balance

For an input photo, the global brightness can be derived from the following equation:

Xwidth x¼1

Color contrast is essential for photo quality measurement because better cameras produce better color contrast. The comparison of different degrees of color contrast is shown in Figure 9, where Figure 9(a) has both bright areas and dark areas with various colors, while Figure 9(b)

Xheight

<sup>y</sup>¼<sup>1</sup> <sup>I</sup>ðx, y<sup>Þ</sup>

<sup>ð</sup><sup>1</sup> � <sup>d</sup>ði, jÞÞ <sup>D</sup>ði, j<sup>Þ</sup>

sðx, yÞ

width � height <sup>ð</sup>8<sup>Þ</sup>

AiAj

width � height <sup>ð</sup>6<sup>Þ</sup>

ð7Þ

f brightness ¼

<sup>f</sup> contrast <sup>¼</sup> <sup>X</sup><sup>n</sup>�<sup>1</sup>

f saturation ¼

Figure 9. Two photos with different degrees of contrast: (a) more contrast; (b) less contrast.

i¼1

is the color distance between the two segmentations in the CIELab color space.

Xn j¼iþ1

where d(i, j) is the spatial distance between the centroids of two segmentations Ai and Aj; D(i, j)

Appealing photos usually have a higher saturation degree. Figure 10 shows an example for comparison in which Figure 10(a) has more vivid colors whereas most pixels in Figure 10(b)

> height X y¼1

where s(x, y) is the saturation of a pixel in the "hue", "saturation", and "value" (HSV) color

width X x¼1

where I(x, y) is the intensity of a pixel at (x, y).

3.4. Contrast

104 Perception of Beauty

3.5. Saturation

space.

appear pale and white.

has only dim white colors.

The color contrast feature is defined as

The color saturation feature is defined as

In the photo aesthetics field, the balance degree of a photo is a good criterion for distinguishing whether a picture is taken by a professional photographer. Professional photographers tend to distribute the color intensity of a photo in a more balanced fashion. A comparison of balanced and unbalanced photos is illustrated in Figure 11. In Figure 11(a), the left and right parts of the photo are more balanced, while in Figure 11(b), the photo is less balanced. Usually, a balanced photo has better composition but it is not necessary that unbalanced photos are unfavorable, which is based on the content and the emotion that the photographer wants to express via the combination of varied photo features.

The difference of brightness of the two separated areas can be adopted to obtain this feature. The balance degree of a photo is calculated by

$$f\_{\text{balance\\_horizon}} = e^{-\left(l\_{\text{left}} - l\_{\text{right}}\right)^2} \tag{9}$$

where Ileft and Iupper are the average intensities of the left and right parts of the photo, respectively. For the vertical balance feature, the similar equation can be acquired from simply replacing Ileft with Iupper as well as replacing Iright with Ilower .
