Face Identification Using LBP-Based Improved Directional Wavelet Transform

*Mohd. Abdul Muqeet and Qazi Mateenuddin Hameeduddin*

### **Abstract**

Face identification is the most active area of research in computer vision and biometric authentication. Various face identification methods are developed over the time, still, numerous facial appearances are needed to cope with such as facial expression, pose, and illumination variation. Moreover, faces captured in unrestrained situations also impose immense concern in designing effective face identification methods. It is desirable to extract robust local descriptive features to effectively characterize such facial variations both in unrestrained and restrained situations. This chapter discusses such a face identification method that incorporate a popular local descriptor such as local binary patterns (LBP) based on the improved directional wavelet transform (IDW) method to extract facial features. This designed method is applied to complex face databases such as CASIA-WebFace and LFW which consists of a large number of face images collected under an unrestrained environment with extreme facial variations in expression, pose, and illumination. Experiments and comparison with various methods which include not only the local descriptive methods but also local descriptive-based multiresolution analysis (MRA) based methods demonstrate the efficacy of the LBP-based IDW method.

**Keywords:** face identification, improved directional wavelet (IDW), local binary patterns (LBP)

#### **1. Introduction**

Researchers have devoted a substantial amount of effort in studying face identification methods in the context of computer vision, image processing, and pattern recognition due the wide acceptability of face as biometric [1]. Requirements for high recognition accuracy, high computational efficiency, and invariance to variations in facial expression, illumination, pose, and occlusions are the prominent challenges in face identification. The illumination problem comes from the fact that different illuminations can cause vast changes to the image of a subject's face [2, 3]. Similarly, deviations in facial expressions along with head pose variation and occlusion can also lead to very unlike face images for the same subject. Moreover, face identification in the unrestrained environment is still a major challenge which greatly degrades the performance of various well-established methods. Additionally, there still exist challenges such as high dimensionality of feature data and intraclass variations occurring due to the effect of facial variations in restrained and unrestrained environments. A face identification method must be discriminatory

for different subjects and invariant to numerous facial variations. Researchers have been extensively utilizing MRA methods and using various off–the-shelve designs of wavelet filters [4] for the implementation of isotropic 2-D DWT for facial feature extraction. Recently implemented 2-D DWT methods such as GHWFB [5] and TWFB [6] considers the handcrafted wavelet filters with additional features compared to off-the-shelve wavelet filters. But these methods do not achieve excellent results due to limited directions orientation and non-adaptation in facial feature selection.

The 2-D IDW being isotropic method performs a separable 1-D horizontal and vertical 1-D IDW on face image with variation in prediction and update steps where seven directions are considered in an adaptive direction scheme. While performing the 1-D IDW transform if non-integer sample arrives sub-pixel interpolation is

Let x i, j ½ � be a 2-D face image which is first horizontally sub-sampled to get even subsamples xe½ �¼ i, j x 2i ½ � þ j and odd sub-samples xo½ �¼ i, j x 2i ½ � þ 1, j . Next in the prediction step odd samples xo½ � i, j are predicted from neighboring even samples with strong correlation along an optimal direction θ*:* The outcome of the prediction

Where Kp and 2Np are the length and coefficients of the prediction filter. Here, samples from six even rows are selected to conduct the prediction step [19]. Now in the update step, odd samples of H i, j ½ � along the same optimal direction θ as used in (1) are selected to modify the even samples. The update step and the generated low-

Where Ku and 2Nu are the coefficients and length of the update filter. Similarly, samples from six odd rows are selected to conduct the update step. The values of

Due to linear phase characteristics and large vanishing moments, Neville filters with the order as six are considered as the coefficients Kp and Ku [19]. The usage of

These directions are used to confirm a strong correlation among samples and to

sign nð Þ � 1 tanθ term in (1) and (3) may not always locate integer samples and may not be present on the original image sampling grid [19]. So; sub-sample interpolation is conceded to compute intensity for such non-integer samples. To maintain perfect reconstruction lifting structure [4], the integer samples required to perform sub-sample interpolation for such non-integer samples at optimal direction θ must be even sampled. If optimal direction comes across the integer samples the value is computed by the nearby even sample otherwise the value of the non-integer sample

To extract local edge details due to face variations that exist at different pixel regions, a quadtree partitioning (QTP) scheme is implemented to partition each face image into sub-blocks of distinct directional details. Each QTP sub-block will have the same direction. The improved QTP scheme provides an efficient direction

In contrast to the nine directions [17] and five directions [18], we also used

<sup>n</sup>*:*xe½ � i þ n, j þ sign nð Þ � 1 tanθ (1)

(3)

H i, j ½ �¼ gH xe½ �� i, j P x<sup>o</sup> ð Þ ð Þ½ � i, j (2)

<sup>n</sup>*:* xo½<sup>i</sup> <sup>þ</sup> n, j <sup>þ</sup> sign nð Þtanθ�� P xð Þ<sup>o</sup> ½ � i þ n, j þ sign nð Þtanθ

Θ ¼ f g θjθ ¼ 0, 22*:*5, 45, 67*:*5, 90, 112*:*5, 135 (5)

!

<sup>H</sup> ð Þ U Hð Þ<sup>o</sup> ½ � i, j � � (4)

step and the generated high-pass signal H i, j ½ � are described as [19],

*Face Identification Using LBP-Based Improved Directional Wavelet Transform*

Kp

N Xp�1

N X<sup>u</sup>�<sup>1</sup> n¼�Nu

Ku

L i, j ½ �¼ gL xo½ �þ i, j <sup>g</sup>�<sup>1</sup>

scaling factors are considered as gL ¼ 1*:*3416 and gH ¼ 0*:*7071 [17, 19].

extract directional MRA features from face images. It is to point out that

is computed from the interpolation of the two nearby even samples.

assignment while implementing the prediction and update step.

**37**

Neville filters increases the approximation ability of 2-D IDW.

seven pre-assigned directions to implement 2-D IDW [19].

n¼�Np

P x<sup>o</sup> ð Þ½ �¼ i, j

*DOI: http://dx.doi.org/10.5772/intechopen.93445*

pass signal L i, j ð Þ are given as [19],

U Hð Þ½ �¼ i, j

performed.

Various local descriptors prominently the LBP [7] and weber local descriptors (WLD) [8] have been efficiently used for facial feature extraction. The constraint of the LBP-based feature extraction method is their noise intolerance and poor discrimination capability [8]. Recently, various non-adaptive MRA methods are applied as a pre-processing step before LBP feature extraction to improve the performance. The prominent methods are local Gabor binary patterns (LGBP) [9], Steerable Pyramid Transform (SPT)-LBP [10], Curvelet Transform-LBP (Curvelet-LBP) [11], Contourlet-LBP [12], and Wavelet Transform (WT) LBP [13]. Liu et al. [14] used hierarchical multi-scale LBP to create features of sparser coefficients and performed classification using sparse coding with the application of a greedy search approach. Wang et al. [15] combined the Gabor wavelet transform (GWT) and CLBP features and carried out the SRC to perform classification.

The aforementioned LBP-based MRA methods [9–13, 15] use non-adaptive directional transform which lacks the adaptive directional selectivity based on the image description. These methods also experience various issues, for instance, selection of transformed sub-bands, complex filter design, and the large dimension of the feature vector. Maleki et al. [16] proposed adaptive direction selection and applied directional lifting within the selected optimal direction and constructed a compact representation for adaptive MRA method. Due to such inherent characteristics, significant directional details for various face variations can be approximated by the detection of edges responsible for such variations [17].

For numerous facial variations, substantial directional details can be estimated by approximating the edges [18, 19] accountable for such variations which will considerably enhance the face identification performance which decides the basis of our method. The concept has been exploited in [17–19] for face recognition applications. This work extends the design of the adaptive directional scheme presented in [19] and presents an LBP-based IDW method to capture multi-resolution directional details from the face images. Subsequently in contrast to [19] where CLBP is used, LBP is applied to the generated IDW sub-bands to extract MRA-based local descriptive features.

The Implementation of the 2-D IDW using seven directions along with the quadtree partitioning (QTP) scheme [19] is explained in Section 2. A brief theory on LBP is described in Section 3. Further, the proposed facial feature extraction method is exhibited in Section 4. In Section 5, comparative results on the CASIA-WebFace and LFW face databases are demonstrated. Conclusions are highlighted in Section 6.

### **2. Implementation of 2-D improved directional wavelet (2-D IDW)**

The fundamental concept of implementation of improved directional wavelet (IDW) is to carry out transform operations on a face image at a viable variety of possible directions while maintaining the properties of multi-resolution, localization, and isotropy intact. The authors in [19] considered a set of seven directions with a quad-tree partitioning scheme. Here we will provide a brief review of the work mentioned in [19].

*Face Identification Using LBP-Based Improved Directional Wavelet Transform DOI: http://dx.doi.org/10.5772/intechopen.93445*

The 2-D IDW being isotropic method performs a separable 1-D horizontal and vertical 1-D IDW on face image with variation in prediction and update steps where seven directions are considered in an adaptive direction scheme. While performing the 1-D IDW transform if non-integer sample arrives sub-pixel interpolation is performed.

Let x i, j ½ � be a 2-D face image which is first horizontally sub-sampled to get even subsamples xe½ �¼ i, j x 2i ½ � þ j and odd sub-samples xo½ �¼ i, j x 2i ½ � þ 1, j . Next in the prediction step odd samples xo½ � i, j are predicted from neighboring even samples with strong correlation along an optimal direction θ*:* The outcome of the prediction step and the generated high-pass signal H i, j ½ � are described as [19],

$$\mathbf{P}(\mathbf{x}^o)[\mathbf{i}, \mathbf{j}] = \sum\_{\mathbf{n} = -N\_p}^{N\_p - 1} \mathbf{K}\_{\mathbf{n}}^p \mathbf{x}\_{\mathbf{e}} [\mathbf{i} + \mathbf{n}, \mathbf{j} + \text{sign}(\mathbf{n} - \mathbf{1}) \text{tan}\theta] \tag{1}$$

$$\mathbf{H}[\mathbf{i}, \mathbf{j}] = \mathbf{g}\_{\mathbf{H}}(\mathbf{x}\_{\mathbf{e}}[\mathbf{i}, \mathbf{j}] - \mathbf{P}(\mathbf{x}^{\mathbf{o}})[\mathbf{i}, \mathbf{j}]) \tag{2}$$

Where Kp and 2Np are the length and coefficients of the prediction filter. Here, samples from six even rows are selected to conduct the prediction step [19]. Now in the update step, odd samples of H i, j ½ � along the same optimal direction θ as used in (1) are selected to modify the even samples. The update step and the generated lowpass signal L i, j ð Þ are given as [19],

$$\mathbf{U}(\mathbf{H})[\mathbf{i}, \mathbf{j}] = \sum\_{\mathbf{n} = -N\_{\mathbf{n}}}^{N\_{\mathbf{u}} - 1} \mathbf{K}\_{\mathbf{n}}^{\mathbf{u}} \begin{pmatrix} \mathbf{x}\_{\mathbf{o}}[\mathbf{i} + \mathbf{n}, \mathbf{j} + \text{sign}(\mathbf{n})\tan\theta] - \\ \mathbf{P}(\mathbf{x}\_{\mathbf{o}})[\mathbf{i} + \mathbf{n}, \mathbf{j} + \text{sign}(\mathbf{n})\tan\theta] \end{pmatrix} \tag{3}$$

$$\mathbf{L}[\mathbf{i}, \mathbf{j}] = \mathbf{g}\_{\mathbf{L}} \left( \mathbf{x}\_{\mathbf{o}}[\mathbf{i}, \mathbf{j}] + \mathbf{g}\_{\mathbf{H}}^{-1} (\mathbf{U}(\mathbf{H}\_{\mathbf{o}})[\mathbf{i}, \mathbf{j}]) \right) \tag{4}$$

Where Ku and 2Nu are the coefficients and length of the update filter. Similarly, samples from six odd rows are selected to conduct the update step. The values of scaling factors are considered as gL ¼ 1*:*3416 and gH ¼ 0*:*7071 [17, 19].

Due to linear phase characteristics and large vanishing moments, Neville filters with the order as six are considered as the coefficients Kp and Ku [19]. The usage of Neville filters increases the approximation ability of 2-D IDW.

In contrast to the nine directions [17] and five directions [18], we also used seven pre-assigned directions to implement 2-D IDW [19].

$$\Theta = \{\theta | \theta = 0, 22.5, 45, 67.5, 90, 112.5, 135\} \tag{5}$$

These directions are used to confirm a strong correlation among samples and to extract directional MRA features from face images. It is to point out that sign nð Þ � 1 tanθ term in (1) and (3) may not always locate integer samples and may not be present on the original image sampling grid [19]. So; sub-sample interpolation is conceded to compute intensity for such non-integer samples. To maintain perfect reconstruction lifting structure [4], the integer samples required to perform sub-sample interpolation for such non-integer samples at optimal direction θ must be even sampled. If optimal direction comes across the integer samples the value is computed by the nearby even sample otherwise the value of the non-integer sample is computed from the interpolation of the two nearby even samples.

To extract local edge details due to face variations that exist at different pixel regions, a quadtree partitioning (QTP) scheme is implemented to partition each face image into sub-blocks of distinct directional details. Each QTP sub-block will have the same direction. The improved QTP scheme provides an efficient direction assignment while implementing the prediction and update step.

for different subjects and invariant to numerous facial variations. Researchers have been extensively utilizing MRA methods and using various off–the-shelve designs of wavelet filters [4] for the implementation of isotropic 2-D DWT for facial feature extraction. Recently implemented 2-D DWT methods such as GHWFB [5] and TWFB [6] considers the handcrafted wavelet filters with additional features compared to off-the-shelve wavelet filters. But these methods do not achieve excellent results due to limited directions orientation and non-adaptation in facial feature

Various local descriptors prominently the LBP [7] and weber local descriptors (WLD) [8] have been efficiently used for facial feature extraction. The constraint of the LBP-based feature extraction method is their noise intolerance and poor discrimination capability [8]. Recently, various non-adaptive MRA methods are applied as a pre-processing step before LBP feature extraction to improve the performance. The prominent methods are local Gabor binary patterns (LGBP) [9], Steerable Pyramid Transform (SPT)-LBP [10], Curvelet Transform-LBP (Curvelet-LBP) [11], Contourlet-LBP [12], and Wavelet Transform (WT) LBP [13]. Liu et al. [14] used hierarchical multi-scale LBP to create features of sparser coefficients and performed classification using sparse coding with the application of a greedy search approach. Wang et al. [15] combined the Gabor wavelet transform (GWT) and

The aforementioned LBP-based MRA methods [9–13, 15] use non-adaptive directional transform which lacks the adaptive directional selectivity based on the image description. These methods also experience various issues, for instance, selection of transformed sub-bands, complex filter design, and the large dimension of the feature vector. Maleki et al. [16] proposed adaptive direction selection and applied directional lifting within the selected optimal direction and constructed a compact representation for adaptive MRA method. Due to such inherent characteristics, significant directional details for various face variations can be approximated

For numerous facial variations, substantial directional details can be estimated by approximating the edges [18, 19] accountable for such variations which will considerably enhance the face identification performance which decides the basis of our method. The concept has been exploited in [17–19] for face recognition applications. This work extends the design of the adaptive directional scheme presented in [19] and presents an LBP-based IDW method to capture multi-resolution directional details from the face images. Subsequently in contrast to [19] where CLBP is used, LBP is applied to the generated IDW sub-bands to extract MRA-based local

The Implementation of the 2-D IDW using seven directions along with the quadtree partitioning (QTP) scheme [19] is explained in Section 2. A brief theory on LBP is described in Section 3. Further, the proposed facial feature extraction method is exhibited in Section 4. In Section 5, comparative results on the CASIA-WebFace and LFW face databases are demonstrated. Conclusions are highlighted in Section 6.

**2. Implementation of 2-D improved directional wavelet (2-D IDW)**

The fundamental concept of implementation of improved directional wavelet (IDW) is to carry out transform operations on a face image at a viable variety of possible directions while maintaining the properties of multi-resolution, localization, and isotropy intact. The authors in [19] considered a set of seven directions with a quad-tree partitioning scheme. Here we will provide a brief review of the

CLBP features and carried out the SRC to perform classification.

by the detection of edges responsible for such variations [17].

selection.

*Biometric Systems*

descriptive features.

work mentioned in [19].

**36**

Let each face image x i, j ½ � is applied with QTP to obtain non-overlapping subblock xs. Also, consider the initial block size Sini, minimum block size as Smin and the Lagrangian multiplier as *α*. The energy summation of the prediction error (ESPE) for each block is computed as [19],

$$\text{ESPE}\_{\mathbf{s},\mathbf{n}} = \sum\_{\mathbf{i},\mathbf{j} \in \mathcal{R}\_{\mathbf{s},\mathbf{n}}} ||\mathbf{x}\_{\mathbf{s}}[\mathbf{i},\mathbf{j}] - \text{F}\_{\mathbf{s},\mathbf{n}}[\mathbf{i},\mathbf{j}]||\_{2}^{2} + \alpha \mathbf{B}^{\mathbf{n}} \tag{6}$$

**4. Implementation of LBP-based IDW method**

*DOI: http://dx.doi.org/10.5772/intechopen.93445*

*Face Identification Using LBP-Based Improved Directional Wavelet Transform*

f g LL, HL, LH . We used the uniform pattern LBPu2

the proposed method is presented in Algorithm 1.

**1.1.** Consider the input face image X.

**Step 2: (Computation of IDW sub-bands) for** a number of decomposition levels **do**

**for** each sub-block within the sub-band **do for** each coefficient value within the sub-block **do**

and *LHl*,*<sup>k</sup>* respectively.

**Step 4: (Dimensionality Reduction)**

vector *EFV\_test*. **Step 6**: **(Identification)**

**39**

**Input:** Test Image, Train image

**Step 1: (Preprocessing)**

blocks.

**Step 3: (LBP Computation)**

**3.2.** Compute the LBPu2

(9), and (10). **3.3.** Concatenate all such LBPu2

end for end for end for

**for** each sub-band **do**

**Algorithm:**

**Algorithm 1: Face Identification using LBP-based IDW**

**Output:** Rank-one recognition results of the feature vectors.

**1.2.** Resize the image to the resolution of 128 � 128 pixels.

**2.2.** Estimate the value of ESPE using (6) for each sub-block.

in the selected directions in the selected sub-block.

sub-band for the next decomposition level.

enhanced histogram feature vector *EFV*.

**2.1.** Quadtree partitioned the face image X into several non-overlapping sub-

**2.3.** Estimate the optimal direction which gives the least value of ESPE using (7). **2.4.** Perform the prediction and the update steps as described in (1) and (3)

**2.5.** Obtain IDW sub-bands {LL, HL, LH, HH} and proceed with the LL

**3.1.** Consider the top-level {LL, LH, HL} sub-bands and divide each subband into non-overlapping regions Rk with each of size 8 � 8 pixels.

band {LL, HL, LH} to form the histogram feature vectors *LLl*,*<sup>k</sup>*, *HLl*,*<sup>k</sup>*,

**3.4.** Concatenate all the sub-band histogram features to form the final

**6.1.** Compare test feature vector *EFV\_test* against train feature vector

**4.1.** Perform dimensionality reduction using LDA on the *EFV* feature vector. **4.2.** Save the reduced dimension train feature vector database to *EFV\_train*. **Step 5**: Repeat Step 1 to 3 and 4.1 on each test image to obtain the test feature

> *EFV\_train* using the NN classifier using Chi-Square distance measure and calculate the Rank-one results in an identification process.

8,1 histogram features from each region Rk using (8),

8,1 multi-region histograms from each sub-

We consider a resolution of 128 � 128 pixels for face images of the selected databases and face preprocessing is performed on all the face images. Thus each one of the IDW sub-bands LL, HL, LH f g is of size 32 � 32 pixels and each sub-band is divided into m ¼ 16 regions with the size of each region as x � y ¼ 8 � 8 pixels [18]. We applied LBP to each of the regions from each of the sub-bands

square distance measure. This form an enhanced feature vector or descriptor *EFV* with a combined dimension as 59 � m � 3 ¼ 2832 [18]. The algorithm representing

8,1 [7] and NN classifier with Chi-

Where Fs,n½ � i, j are the filtered responses obtained by applying the prediction filter K<sup>p</sup> along with the predefined directions θ. B<sup>n</sup> is the number of bits spent on signaling the selection of directions. When a sample is predicted from the nearest samples, each candidate direction from (5) is checked and the direction with the smallest ESPE is ultimately selected. The optimal direction which gives the least value of ESPE is selected as,

$$\Theta\_{\mathbf{s}} = \arg\min\_{\mathbf{n}} \left\{ \text{ESPE}\_{\mathbf{s},\mathbf{n}} \right\} \tag{7}$$

The value of the lagrangian multiplier α determines the complexity of the QTP scheme and its value needs to be selected sensibly. Moreover, to detect the local edge details and to suit it to the adaptability of the IDW method, a face image needs to be segmented into partitions of clear orientation bias. To resolve this problem an improved QP scheme is proposed to suit the face identification problem as mentioned in [19]. The 1-D IDW can be simply extended to the 2-D IDW where second dimension lifting is yet again performed in the horizontal direction on high-pass signal H i, j ½ � and low-pass signals L i, j ½ � to generate four sub-bands i.e. LH i, j ½ �, LL i, j ½ �, HH i, j ½ �, and HL i, j ½ �.

#### **3. Local binary patterns**

The LBP [7] is estimated with sampling points xp ∈ð Þ 0, … , P � 1 in the neighborhood of a center pixel xm ic, jc � � at a radial distance given by R [7],

$$\text{LBP}\_{\text{P,R}} = \sum\_{\mathbf{p}=\mathbf{0}}^{\mathbf{p}-1} \mathbf{t}\_{\mathbf{s}} \left(\mathbf{x}\_{\mathbf{p}} - \mathbf{x}\_{\mathbf{m}}\right) . \text{2P} \tag{8}$$

$$\mathbf{t}\_\*(\mathbf{d}) = \begin{cases} \mathbf{1}, (\mathbf{d}) \ge \mathbf{1} \\\\ \mathbf{0}, (\mathbf{d}) < \mathbf{1} \end{cases} \tag{9}$$

Where tsð Þ d is a threshold function. The sampling points which does not fit within the center of a pixel are bilinearly interpolated [7]. Another extension of LBP is the uniform patterns and it is mapped from LBPP,R to LBPu2 P,R [18], resulting in P � ð Þþ P � 1 3 feature dimension. After obtaining the LBP coded image, codes of the input image XLð Þ i, j pixels are formed into a histogram as a feature descriptor,

$$\mathbf{H}\_{\mathbf{l}} = \sum\_{\mathbf{i}, \mathbf{j}} \mathbf{F} \{ \mathbf{X} \mathbf{i} (\mathbf{i}, \mathbf{j}) = \mathbf{l} \}, \mathbf{F} \{ \mathbf{y} \} = \begin{cases} \mathbf{1}, \mathbf{i} \text{ if } \mathbf{y} \text{ is true} \\\\ \mathbf{0}, \mathbf{i} \text{ if } \mathbf{y} \text{ is false} \end{cases}, \mathbf{l} = \mathbf{0}, \mathbf{1}, \mathbf{2}, \dots, \mathbf{n} - \mathbf{1} \tag{10}$$

Where n is the number of different labels produced by the LBP operator. With the usage of LBPu2 8,1, the feature dimension is 59 [18].
