2. Local pattern descriptor

A local pattern considers the variations of subregion in an image, which is also called a micropattern. In this section, we introduce the basic and several popular techniques of local pattern descriptor for facial recognition.

## 2.1. Local binary pattern

Local binary pattern (LBP) [5] is designed to describe the texture in a local neighborhood is an invariant texture measure and has been various comparative studies, such as fingerprint recognition [6], face recognition [7], and license plate recognition [8]. The main characteristics of LBP are: (1) highly discriminative capability (2) and computational efficiency.

The basic LBP encodes the pixels of an image by thresholding 3 � 3 neighborhood of each pixel with the given referenced pixel Xc and concatenates the results to form a binary expression, as shown in Figure 1. The equation of basic LBP operator is formulated as follows:

$$LBP\_P(X\_c) = \left\{ f\_1\left(I(X\_p), I(X\_c)\right) \right\} \big|\_{p=1,2,\ldots,P} \tag{1}$$

f <sup>1</sup> I Xp

Figure 2. Example of generating LBP for local region.

Figure 1. Example of 8-neighborhood surrounding a referenced pixel Xc.

; I Xð Þ<sup>c</sup>

<sup>¼</sup> <sup>1</sup>, if I Xp

and its neighborhoods. In practical, the threshold T<sup>1</sup> can be set to 0, if f <sup>1</sup> I Xp

<sup>0</sup>, else

where p is the index of the neighborhoods which is surrounding the referenced pixel Xc. and T<sup>1</sup> is the threshold. f <sup>1</sup>ð Þ �; � represents the gradient variation between a given referenced pixel Xc

means the neighborhoods Xp have higher gradient information compared with referenced pixel Xc. Figure 2 is an example of generating an LBP micropattern. Figure 2 demonstrates that LBP is generated by using Eqs. (1) and (2) from X<sup>1</sup> to X<sup>8</sup> and encodes the binary pattern of a give reference pixel Xc as 10011111. Figure 3 demonstrates the spatial distribution of the example of LBP as shown in Figure 2 in one-dimensional. In Figure 3, the neighborhoods, which are encoded as 1, are arranged on the right of reference pixel Xc, and the others are

 � I Xð Þ<sup>c</sup> ≥ T<sup>1</sup>

(2)

 ; I Xð Þ<sup>c</sup> <sup>¼</sup> 0, it

Local Patterns for Face Recognition

93

http://dx.doi.org/10.5772/intechopen.76571

where Xp, p ¼ 1, …, 8 is a neighborhood of the referenced pixel Xc in the local subregion of an image I. f <sup>1</sup>ð Þ �; � is the coding scheme which decides the binary number of each neighborhood, called the threshold function and this can be expressed as

Figure 1. Example of 8-neighborhood surrounding a referenced pixel Xc.

The facial descriptor refers to the process of extracting the discriminative features to represent a given face image. Numerous methodologies are proposed to recognize face and those can be classified as global and local facial descriptors. The global facial descriptor describes the facial characteristics with the whole face image, such as principal component analysis (PCA) [1, 2] and linear discriminant analysis (LDA) [3, 4]. PCA converts the global facial descriptor from high dimension to low dimension by using the linear transform methodology to reduce the computational cost. Linear discriminant analysis (LDA) also called the Fishers Linear Discriminant is similar to PCA, while it is a supervised methodology. Although the global facial descriptor can extract the principal component from the facial images, reduces the computational cost, and maintains the variance of the facial image, the performance is sensitive to the

The flexibilities of the local facial descriptors are better than the global facial descriptors because they successfully and effectively represent the spatial structure information of an input image. A well local facial descriptor generates discriminative and robust features to achieve good recognition results with computational simplicity. In this chapter, we represent a number of approaches in the local facial descriptor including the local binary pattern (LBP), local derivation pattern (LDP), local tetra pattern (LTrP), local vector pattern (LVP) and local

A local pattern considers the variations of subregion in an image, which is also called a micropattern. In this section, we introduce the basic and several popular techniques of local

Local binary pattern (LBP) [5] is designed to describe the texture in a local neighborhood is an invariant texture measure and has been various comparative studies, such as fingerprint recognition [6], face recognition [7], and license plate recognition [8]. The main characteristics

The basic LBP encodes the pixels of an image by thresholding 3 � 3 neighborhood of each pixel with the given referenced pixel Xc and concatenates the results to form a binary expression, as

where Xp, p ¼ 1, …, 8 is a neighborhood of the referenced pixel Xc in the local subregion of an image I. f <sup>1</sup>ð Þ �; � is the coding scheme which decides the binary number of each neighborhood,

 ; I Xð Þ<sup>c</sup> <sup>p</sup>¼1,2,…,P

(1)

of LBP are: (1) highly discriminative capability (2) and computational efficiency.

shown in Figure 1. The equation of basic LBP operator is formulated as follows:

LBPPð Þ¼ Xc f <sup>1</sup> I Xp

called the threshold function and this can be expressed as

change of the environment, such as the change of light.

92 From Natural to Artificial Intelligence - Algorithms and Applications

clustering pattern (LCP).

2.1. Local binary pattern

2. Local pattern descriptor

pattern descriptor for facial recognition.

Figure 2. Example of generating LBP for local region.

$$f\_1\left(I\left(X\_p\right), I(X\_c)\right) = \begin{cases} 1, & \text{if}\left(I\left(X\_p\right) - I(X\_c)\right) \ge T\_1\\ 0, & \text{else} \end{cases} \tag{2}$$

where p is the index of the neighborhoods which is surrounding the referenced pixel Xc. and T<sup>1</sup> is the threshold. f <sup>1</sup>ð Þ �; � represents the gradient variation between a given referenced pixel Xc and its neighborhoods. In practical, the threshold T<sup>1</sup> can be set to 0, if f <sup>1</sup> I Xp ; I Xð Þ<sup>c</sup> <sup>¼</sup> 0, it means the neighborhoods Xp have higher gradient information compared with referenced pixel Xc. Figure 2 is an example of generating an LBP micropattern. Figure 2 demonstrates that LBP is generated by using Eqs. (1) and (2) from X<sup>1</sup> to X<sup>8</sup> and encodes the binary pattern of a give reference pixel Xc as 10011111. Figure 3 demonstrates the spatial distribution of the example of LBP as shown in Figure 2 in one-dimensional. In Figure 3, the neighborhoods, which are encoded as 1, are arranged on the right of reference pixel Xc, and the others are

Figure 3. Spatial distribution of example of LBP.

arranged on the left of reference pixel Xc. The distance is the gradient variant between reference pixel and its neighborhoods as shown in Figure 3.

Furthermore, to address the problem of the textures at different scales, there are some followers which extend to use neighborhoods with various scales [9, 10]. To compare with basic LBP, the local neighborhoods are evenly spaced on a circle centered at the reference pixel Xc, and the formulation of Eqs. (1) and (2) is re-formulated as follows:

$$LBP\_{P,R}(X\_c) = \left\{ f\_1\left(I\{X\_{p,r}\}, I(X\_c)\right) \right\} \big|\_{p=1,2,\cdots,P; r=1,\ldots R} \tag{3}$$

extension of LBP with directional high-order local derivative pattern [11]. To encode the nthorder LDP, the ð Þ <sup>n</sup> � <sup>1</sup> th-order local derivative variations with various distinctive spatial rela-

where I is a given image, Xc is the referenced pixel and Xp, p ¼ 1, 2, 3, 4 are the neighborhoods

and f <sup>2</sup>ð Þ �; � is the binary coding function which describes the spatial relationship between referenced pixel Xc and its neighborhoods Xp in various derivative directions, and that can be

> <sup>α</sup>ð Þ� Xc I 0 <sup>α</sup> Xp � � ≤ 0

<sup>α</sup>ð Þ� Xc I 0 <sup>α</sup> Xp � � <sup>&</sup>gt; <sup>0</sup> <sup>p</sup> <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>⋯</sup>, P (

The spatial relationship between two pixels includes the conditions of turning and monotoni-

<sup>α</sup>ð Þ Xc <sup>α</sup> <sup>¼</sup> <sup>0</sup><sup>∘</sup>

; 45<sup>∘</sup> ; 90<sup>∘</sup> ; <sup>135</sup><sup>∘</sup> j g: � (11)

n�2

n�2

n�2

n�2

Finally, the second-order LDP is defined as the concatenation of the four directional LDPs

, 45<sup>∘</sup> , 90<sup>∘</sup>

> n�2 <sup>0</sup><sup>∘</sup> ð Þ� Xc I

> n�2 <sup>45</sup><sup>∘</sup> ð Þ� Xc I

> n�2 <sup>90</sup><sup>∘</sup> ð Þ� Xc I

n�2 <sup>135</sup><sup>∘</sup> ð Þ� Xc I

0 <sup>α</sup>ð Þ Xc ; I 0

0, if I<sup>0</sup>

cally increasing/decreasing and be coded as 1 and 0 in LDP, respectively.

ð Þ¼ Xc LDP<sup>2</sup>

I n�1 <sup>0</sup><sup>∘</sup> ¼ I

I n�1 <sup>45</sup><sup>∘</sup> ¼ I

I n�1 <sup>90</sup><sup>∘</sup> ¼ I

I n�1 <sup>135</sup> <sup>∘</sup> ¼ I

, and 135<sup>∘</sup> directions are used. The first-order derivatives of the

<sup>0</sup><sup>∘</sup> ¼ I Xð Þ� <sup>c</sup> I Xð Þ<sup>4</sup> (5)

<sup>45</sup><sup>∘</sup> ¼ I Xð Þ� <sup>c</sup> I Xð Þ<sup>3</sup> (6)

<sup>90</sup><sup>∘</sup> ¼ I Xð Þ� <sup>c</sup> I Xð Þ<sup>2</sup> (7)

<sup>135</sup><sup>∘</sup> ¼ I Xð Þ� <sup>c</sup> I Xð Þ<sup>1</sup> (8)

0 <sup>α</sup>ð Þ Xc ; I 0 <sup>α</sup> Xp

, 45<sup>∘</sup> , 90<sup>∘</sup>

, and 135<sup>∘</sup> directions are calculated by modi-

<sup>0</sup><sup>∘</sup> ð Þ X4,R (12)

<sup>45</sup><sup>∘</sup> ð Þ X3,R (13)

<sup>90</sup><sup>∘</sup> ð Þ X2,R (14)

<sup>135</sup><sup>∘</sup> ð Þ X1,R (15)

<sup>α</sup>ð Þ Xc , in α derivative direction at referenced pixel Xc is

�

Local Patterns for Face Recognition

95

http://dx.doi.org/10.5772/intechopen.76571

� (9)

(10)

, and 135<sup>∘</sup> directions

, and 135<sup>∘</sup> directions can be written as

<sup>α</sup>ð Þ X<sup>2</sup> ; ⋯; f <sup>2</sup>I

n o � � <sup>p</sup>¼1,2,⋯<sup>P</sup>

tionships along 0<sup>∘</sup>

LDP<sup>2</sup>

expressed as

<sup>α</sup>ð Þ¼ Xc f <sup>2</sup> I

f <sup>2</sup> I 0 <sup>α</sup>ð Þ Xc ; I 0 <sup>α</sup> Xp � � � � <sup>¼</sup> <sup>1</sup>, if I<sup>0</sup>

0 <sup>α</sup>ð Þ Xc ; I 0 <sup>α</sup>ð Þ X<sup>1</sup> � �; <sup>f</sup> <sup>2</sup><sup>I</sup>

LDP<sup>2</sup>

The ð Þ <sup>n</sup> � <sup>1</sup> th-order derivatives along 0<sup>∘</sup>

Then, the nth-order LDP, LDP<sup>n</sup>

expressed as

fying Eqs. (5)–(8) and be re-formulated as

where α is derivative direction at referenced pixel Xc along 0<sup>∘</sup>

referenced pixel Xc along 0<sup>∘</sup>

, 45<sup>∘</sup>

, 90<sup>∘</sup>

, 45<sup>∘</sup> , 90<sup>∘</sup>

> I 0

I 0

I 0

I 0

of Xc as shown in Figure 1. Then, the second-order LDP can be encoded as,

$$\begin{aligned} \, \_1f\_1\left(I\{X\_{\mathbb{p},R}\}, I(X\_c)\right) = \begin{cases} 1, & \text{if}\left(I\{X\_{\mathbb{p},R}\} - I(X\_c)\right) \ge T\_1\\ 0, & \text{else} \end{cases} \end{aligned} \tag{4}$$

where r is the radius between the referenced pixel Xc and its neighborhood pixels Xp. Figure 4 illustrates examples of circular neighborhoods with any radius and number of sampling points. The neighbor that does not fall in the center of a pixel is estimated by using bilinear interpolation.

#### 2.2. Local derivative pattern

LBP is a nondirectional first-order local derivative pattern of images and fails to extract more detailed information, such as the directions between neighborhoods and referenced pixel, and the high-order gradient information. Local derivative pattern (LDP) can be considered as an

Figure 4. Circularly symmetric neighborhoods sets for different ð Þ P; R . (a) (P=4,R=1), (b) (P=8,R=1), (c) (P=8,R=2).

extension of LBP with directional high-order local derivative pattern [11]. To encode the nthorder LDP, the ð Þ <sup>n</sup> � <sup>1</sup> th-order local derivative variations with various distinctive spatial relationships along 0<sup>∘</sup> , 45<sup>∘</sup> , 90<sup>∘</sup> , and 135<sup>∘</sup> directions are used. The first-order derivatives of the referenced pixel Xc along 0<sup>∘</sup> , 45<sup>∘</sup> , 90<sup>∘</sup> , and 135<sup>∘</sup> directions can be written as

$$\boldsymbol{I}\_{0^\*} = \boldsymbol{I}(\boldsymbol{X\_c}) - \boldsymbol{I}(\boldsymbol{X\_4}) \tag{5}$$

$$\boldsymbol{I}\_{\mathsf{45}^\*}^\prime = \boldsymbol{I}(\mathbf{X}\_c) - \boldsymbol{I}(\mathbf{X}\_3) \tag{6}$$

$$\boldsymbol{I}\_{90^\circ}^{'} = \boldsymbol{I}(\mathbf{X}\_c) - \boldsymbol{I}(\mathbf{X}\_2) \tag{7}$$

$$\boldsymbol{I}\_{135^{\*}}^{'} = \boldsymbol{I}(\boldsymbol{X}\_{c}) - \boldsymbol{I}(\boldsymbol{X}\_{1}) \tag{8}$$

where I is a given image, Xc is the referenced pixel and Xp, p ¼ 1, 2, 3, 4 are the neighborhoods of Xc as shown in Figure 1. Then, the second-order LDP can be encoded as,

arranged on the left of reference pixel Xc. The distance is the gradient variant between refer-

Furthermore, to address the problem of the textures at different scales, there are some followers which extend to use neighborhoods with various scales [9, 10]. To compare with basic LBP, the local neighborhoods are evenly spaced on a circle centered at the reference pixel Xc,

; I Xð Þ<sup>c</sup>

0, else

where r is the radius between the referenced pixel Xc and its neighborhood pixels Xp. Figure 4 illustrates examples of circular neighborhoods with any radius and number of sampling points. The neighbor that does not fall in the center of a pixel is estimated by using bilinear

LBP is a nondirectional first-order local derivative pattern of images and fails to extract more detailed information, such as the directions between neighborhoods and referenced pixel, and the high-order gradient information. Local derivative pattern (LDP) can be considered as an

Figure 4. Circularly symmetric neighborhoods sets for different ð Þ P; R . (a) (P=4,R=1), (b) (P=8,R=1), (c) (P=8,R=2).

 <sup>p</sup>¼1, <sup>2</sup>,⋯,P;r¼1,::<sup>R</sup> 

> � I Xð Þ<sup>c</sup> ≥ T<sup>1</sup>

(3)

(4)

ence pixel and its neighborhoods as shown in Figure 3.

94 From Natural to Artificial Intelligence - Algorithms and Applications

Figure 3. Spatial distribution of example of LBP.

f <sup>1</sup> I Xp,R

interpolation.

2.2. Local derivative pattern

and the formulation of Eqs. (1) and (2) is re-formulated as follows:

; I Xð Þ<sup>c</sup>

LBPP,Rð Þ¼ Xc f <sup>1</sup> I Xp,r

<sup>¼</sup> <sup>1</sup>, if I Xp,R

$$LDP\_a^2(\mathbf{X}\_\varepsilon) = \left\{ f\_2\left(\dot{I}\_a(\mathbf{X}\_\varepsilon), \dot{I}\_a(\mathbf{X}\_1)\right), f\_2\dot{I}\_a(\mathbf{X}\_\varepsilon), \dot{I}\_a(\mathbf{X}\_2), \dots, f\_2\dot{I}\_a(\mathbf{X}\_\varepsilon), \dot{I}\_a(\mathbf{X}\_\varepsilon)\right\}|\_{\mathcal{V} = 1, 2, \dots, p} \tag{9}$$

where α is derivative direction at referenced pixel Xc along 0<sup>∘</sup> , 45<sup>∘</sup> , 90<sup>∘</sup> , and 135<sup>∘</sup> directions and f <sup>2</sup>ð Þ �; � is the binary coding function which describes the spatial relationship between referenced pixel Xc and its neighborhoods Xp in various derivative directions, and that can be expressed as

$$f\_2\left(\mathbf{i}'\_a(\mathbf{X}\_c), \mathbf{i}'\_a(\mathbf{X}\_p)\right) = \begin{cases} 1, & \text{if } \mathbf{i}'\_a(\mathbf{X}\_c) \cdot \mathbf{i}'\_a(\mathbf{X}\_p) \le 0 \\ 0, & \text{if } \mathbf{i}'\_a(\mathbf{X}\_c) \cdot \mathbf{i}'\_a(\mathbf{X}\_p) > 0 \end{cases} \quad p = 1, 2, \dots, P \tag{10}$$

The spatial relationship between two pixels includes the conditions of turning and monotonically increasing/decreasing and be coded as 1 and 0 in LDP, respectively.

Finally, the second-order LDP is defined as the concatenation of the four directional LDPs

$$LDP^2(X\_c) = \{LDP^2\_a(X\_c)|a = 0^\ast, 45^\ast, 90^\ast, 135^\ast\}.\tag{11}$$

The ð Þ <sup>n</sup> � <sup>1</sup> th-order derivatives along 0<sup>∘</sup> , 45<sup>∘</sup> , 90<sup>∘</sup> , and 135<sup>∘</sup> directions are calculated by modifying Eqs. (5)–(8) and be re-formulated as

$$I\_{0^{\*}}^{
n-1} = I\_{0^{\*}}^{
n-2}(X\_{\mathfrak{c}}) - I\_{0^{\*}}^{
n-2}(X\_{\mathfrak{4},\mathbb{R}}) \tag{12}$$

$$I\_{45^\*}^{n-1} = I\_{45^\*}^{n-2}(X\_c) - I\_{45^\*}^{n-2}(X\_{3,R}) \tag{13}$$

$$I\_{90^\*}^{n-1} = I\_{90^\*}^{n-2}(X\_{\mathfrak{c}}) - I\_{90^\*}^{n-2}(X\_{2,\mathbb{R}}) \tag{14}$$

$$I\_{135^\*}^{n-1} = I\_{135^\*}^{n-2}(X\_c) - I\_{135^\*}^{n-2}(X\_{1,\mathbb{R}}) \tag{15}$$

Then, the nth-order LDP, LDP<sup>n</sup> <sup>α</sup>ð Þ Xc , in α derivative direction at referenced pixel Xc is expressed as

$$\text{LDP}\_{a}^{\text{\textquotedblleft}}(\text{X}\_{\text{c}}) = \left\{ f\_{2} \left( I\_{a}^{\text{\textquotedblleft}1} (\text{X}\_{\text{c}}), I\_{a}^{\text{\textquotedblright}} (\text{X}\_{\text{p},\text{R}}) \right) \right\} | p = 1, 2, \dots, P; \text{R} = 1, a = 0^{\text{\textquotedblleft}}, 45^{\text{\textquotedblleft}}, 90^{\text{\textquotedblleft}}, 135^{\text{\textquotedblright}} \right\} \tag{16}$$

An example of high-order derivative is shown in Figure 5. Figure 5(a) is the original value of image, Figure 5(b) is the first-order derivative in 0<sup>∘</sup> direction by using Eq. (5), and Figure 5(c) is the second-order derivative in 0<sup>∘</sup> direction by using Eq. (12) with the value in Figure 5(b).

Figure 6 demonstrates an example to encode the second-order LDP in 0<sup>∘</sup> direction. To encode the second-order LDP, the results of first-order derivatives are needed. Taking the bit 1 as an example, the results of first-order derivatives of referenced pixel Xc and the neighborhood X<sup>1</sup> are 7ð Þ¼ � 3 4 and 4ð Þ¼� � 9 5, respectively. The spatial relationship between neighborhood X<sup>1</sup> and referenced pixel Xc is turning 7ð Þ� � 3 ð Þ¼� 4 � 9 20 ≤ 0. Therefore, we encode the bit 1 as 1 by Eq. (10). Similarly, the spatial relationship between referenced pixel Xc and neighborhoods pixels Xp,p ¼ 4, 5, 7, 8 presents the turning and be encoded as "1". The reset of neighborhoods pixels is encoded as "0". The second-order LDP in 0<sup>∘</sup> direction, LDP<sup>2</sup> P,R¼1,α¼0<sup>∘</sup> , is encoded as "10011011". According to the same encoding process, the results of second-order LDP in 45<sup>∘</sup> , 90<sup>∘</sup> and 135<sup>∘</sup> are LDP<sup>2</sup> P,R¼1,α¼45<sup>∘</sup> <sup>¼</sup> 01110100, LDP<sup>2</sup> P,R¼1,α¼90<sup>∘</sup> <sup>¼</sup> 11100001, and LDP<sup>2</sup> P,R¼1,α¼135<sup>∘</sup> <sup>¼</sup> 10011101, respectively. Finally, LDP<sup>2</sup> P,R¼<sup>1</sup> <sup>¼</sup> <sup>1001101101110100111000011</sup> 0011101 with 32-bit is generated by concatenating the four 8-bit LDPs with various derivative directions.

Figure 7 demonstrates the spatial distribution of example of LDP in 0<sup>∘</sup> direction in onedimensional. In Figure 7, the evaluation results of LDP in 0<sup>∘</sup> direction are normalized into the region of ½ � �8 8 , the neighborhoods that are encoded as 1 are arranged on the left of 0, and the others are arranged on the right of 0. The distance is the magnitude of gradient variant between reference pixel Xc and its neighborhoods.
