**3. Literature review and related work**

**2.** At rank stage [24]: the score values are arranged in descending order showing the possibility of the decision that at top list most preferred classes are placed and at down

**3.** Decision stage depends totally on the result value of the score stage, and final decision is taken whether the identified person is fake to reject or a unique to accept. Each clas-

• Decision is taken when a majority of the classifiers declare the same decision. To ensure

• And operator means all the classifiers give the same result whether reject or accept, and it is good when low false acceptance is required. Or operator is useful when low false

A biometric system needs to be evaluated and tested; there are some measurement concepts

• EER means both rate false accept and false reject are equal, and the more the EER, the more accurate the system is. The FRR refers to the rate of permitted users but are rejected by the

• FAR means how many people don't have permission but the system accepts them as autho-

• Failure to enroll (FTE) concerned with the rate of individuals not able to enroll in the

• Failure to capture (FTC) concerned with the biometric traits are presented correctly, but the

FRR = (number of false rejected/NAA) × 100% (1)

FAR = (number of false accepted/NIA) × 100% (2)

NIA means number of impostor attempts and NAA means number of authorized attempts. The accuracy and recognition rate and performance measurements of a biometric system can

• FRR: the number of the authorized person but falsely rejected by the system.

sifier provides a hard decision. The decisions can be combined using:

a decision is taken, we must have classifiers more than the number of classes.

• Instead of having reject or accept, we have a truth value between two values.

for evaluation as equal, false rejection and false acceptance rate [1, 26]:

list least preferred classes.

• Majority voting:

44 Machine Learning and Biometrics

• Logic operator (and, or):

reject is required.

**2.4. Performance evaluation**

rized person and falsely accepted.

be affected by some factors [26]:

system was not able to capture them correctly.

system falsely.

system.

• Fuzzy logic [25]:

Alphonse Bertillon, who firstly introduced the idea of personal identification system based on biometric, morphological, and anthropometric using color of the eyes, hair, and skin in 1896. Face recognition is lower in uniqueness and more acceptable than iris but still is user-friendly, and people are willing to use it than other techniques [27]. The soft biometric is divided into three groups as follows [28]:


Soft biometric traits also can be classified according to permanence and distinctiveness as shown in **Table 1**. The permanence of a trait shows the strength of the trait over the period of time as gender and ethnicity don't change over time. Distinctiveness refers to the ability of a trait to differentiate between individuals.


**Table 1.** Facial soft biometric traits.

In this paper, we are focusing on the head soft biometric features. As shown in **Table 2**, humans can easily be identified by their faces because they don't change over a period of time and widely. According to Lin [47], face features provide different information when resized or clipped or shown from different sides.

The related works show some of the major works presented in timeline order starting from 2000 up to 2017 as shown in **Table 2**.

Jain and Dass et al. [13] the father of soft biometric who introduced it as ancillary information, but are not able to individually authenticate the person due to the lake of distinctiveness and permanence. They propose to use demographic information (gender, ethnicity, and height) as soft biometrics to improve the primary fingerprint system. Experiments show that recognition performance of fingerprint increased 5% by using soft biometrics.

Pedro and Julian et al. [28] experimental result shows that soft biometrics can be used as a secondary information to improve the primary biometrics and they can be acquired from distance; fusion is taken at score stage. Park and Jain et al. [34] use three feature extraction techniques:


Two datasets are used to evaluate the system. They show that the use of soft biometrics (ethnicity, gender, facial marks) increases the recognition rate. Soft biometric traits can be considered as an alternative when face images are occluded or partially damaged. Gender and ethnicity of a person do not change over the lifetime, so they can be used to purge the database to narrow the search list. However, performance increased, but complexity also increased, and facial mark extraction depends on the image resolution and controlled environment needed.

Dantcheva and Velardo et al. [48] introduce two new soft biometric traits, called body weight and clothes color. Related promising results on the performance are provided. Dantcheva and Dugelay et al. [35] use eyes, skin, and hair color traits and cascade classifier; performance increased and balanced between complexity and performance. However, system suffers from illumination and poses, evaluated under one dataset and controlled environment. Soft biometric traits collected from a distance without user cooperation as shown by Denman and Fookes et al. [31] propose head and body traits and system evaluated using PETS 2006 small dataset and recognition rate decreased but the system can be used when primary data not available. Niinuma and Jain et al. [33] propose framework for continuous user authentication that uses clothing and skin colors fused with password. Soft biometric traits collected automatically every time user login with his password. Experiment results show the method effectiveness for continuous user authentication. However, system is evaluated with one dataset and suffering from illumination.

**Ref.**

[16]

•

Face

2000

• •

[10]

•

Fingerprint

160 subjects

2004

• • •

[29]

Tattoo(human, plant, flag, symbol,

• tattoo

•

Michigan State

• • •

Key point matching

A novel gait analysis in

Height, gender, and body size show a better

performance over stride/step lengths

video surveillance

Local extrema detection

Difference of Gaussian

Police Tattoo Database (MI-DB)

Image Database Web-based

•

Scale-invariant feature

Performance increased to 77.2% on (MI-DB) and 98.6%

transformer (SIFT)

on (web-based tattoo)

2008

[30]

•

Height

• •

SET HD indoor dataset

USF outdoor dataset

2008

• • •

[31]

•

Legs, head color and size

PETS 2006

Active appearance model

The error rate is decreased

2009

[11]

•

Face

• •

Mugshot

FERET

•

Morphological

System performance increased by 3 percentage to

A Survey on Soft Biometrics for Human Identification http://dx.doi.org/10.5772/intechopen.76021 47

Operators

become 94.14%

• •

Laplacian of Gaussian

AAM

2009

•

Facial marks

Stride/step

Gender

Body size

object)

Height

Ethnicity

Gender

Lip movement

Voice

**Modalities**

**Database**

• • frames

A sample of audio and video

• •

Hausdorff face location

measurement

•

Synergetic

Bayes rule

The recognition rate increased by 6%

Fourier transformation

150 persons for 3 months

•

Optical-flow technique

• rate decreased

Performance increased, and the false acceptance

**Techniques**

**Results**


In this paper, we are focusing on the head soft biometric features. As shown in **Table 2**, humans can easily be identified by their faces because they don't change over a period of time and widely. According to Lin [47], face features provide different information when resized

The related works show some of the major works presented in timeline order starting from

Jain and Dass et al. [13] the father of soft biometric who introduced it as ancillary information, but are not able to individually authenticate the person due to the lake of distinctiveness and permanence. They propose to use demographic information (gender, ethnicity, and height) as soft biometrics to improve the primary fingerprint system. Experiments show that recogni

Pedro and Julian et al. [28] experimental result shows that soft biometrics can be used as a secondary information to improve the primary biometrics and they can be acquired from distance; fusion is taken at score stage. Park and Jain et al. [34] use three feature extraction

Two datasets are used to evaluate the system. They show that the use of soft biometrics (ethnicity, gender, facial marks) increases the recognition rate. Soft biometric traits can be considered as an alternative when face images are occluded or partially damaged. Gender and ethnicity of a person do not change over the lifetime, so they can be used to purge the database to narrow the search list. However, performance increased, but complexity also increased, and facial mark extraction depends on the image resolution and controlled envi

Dantcheva and Velardo et al. [48] introduce two new soft biometric traits, called body weight and clothes color. Related promising results on the performance are provided. Dantcheva and Dugelay et al. [35] use eyes, skin, and hair color traits and cascade clas

sifier; performance increased and balanced between complexity and performance. However, system suffers from illumination and poses, evaluated under one dataset and controlled environment. Soft biometric traits collected from a distance without user coop

eration as shown by Denman and Fookes et al. [31] propose head and body traits and system evaluated using PETS 2006 small dataset and recognition rate decreased but the system can be used when primary data not available. Niinuma and Jain et al. [33] pro

pose framework for continuous user authentication that uses clothing and skin colors fused with password. Soft biometric traits collected automatically every time user login with his password. Experiment results show the method effectiveness for continuous user authentication. However, system is evaluated with one dataset and suffering from






.

tion performance of fingerprint increased 5% by using soft biometrics.

• Active appearance model for extracting facial features as nose and eyes

or clipped or shown from different sides.

2000 up to 2017 as shown in **Table 2**

46 Machine Learning and Biometrics

techniques:

• Laplacian of Gaussian

ronment needed.

illumination.

• Morphological operators


**Ref.**

[37]

•

Sunglasses

2011

[38]

Anthropometric body measures

• • • •

IMS-BHU Indian hospital

•

Principal component

Soft biometrics improve primary face biometric

performance by 6.5%

analysis (PCA)

• •

Linear discriminant

analysis (LDA) [27]

• (LBP)

•

Speeded up robust

features (SURF)

•

Wavelet

The recognition rate increased, equal error rate

decreased, and skin color highly increases the

performance

characterization

•

SVM

[40]

•

Facial measurement

Face94

2013

• •

[41]

Facial wrinkles

Well-known people from the

• • •

Modified Hausdorff

to 93%

distance (MHD)

•

Curve proximity distance (CPD)

Curves to Line segments algorithm

Bipartite graph matching algorithm

• 88% achieved by MHD

• 87 achieved by CPD

• CPD performs better than MHD

• Fusing MHD and CPD increases the recognition rate

A Survey on Soft Biometrics for Human Identification http://dx.doi.org/10.5772/intechopen.76021 49

Internet with high resolution

2013

Hair color

Skin color

Local binary pattern

Independent component analysis (ICA)

FERET dataset

NHANES dataset

Medical chimera dataset

2012

[39]

•

Gender

2012

• • •

Blood group

Weight

Height

•

Scarf

**Modalities**

**Database**

AR Face

**Techniques**

• • •

SVM

Eigenfaces

Anthropometric features improve performance in both

accuracy and recognition speed

PCA

Gabor wavelets

**Results**

Recognition rate has increased


**Ref.**

[32]

•

Height and color of:

2009

○ ○ ○

[33]

•

Face

Video frames for 20 persons

• • •

Bhattacharyya

coefficient

Haar classifier

•

Not affected by the person position

PCA (Eigen)

•

Real-time identification

2010

• • •

[34]

•

Face

• •

Mugshot

FERET

• • • •

Morphological

operators

PCA

• twins

Facial marks can help in discriminating identical

Gaussian

AAM

•

The fusion of soft biometrics is able to improve the

performance of face recognition

2010

• • •

[35]

•

Skin color

Color FERET with 646 people

• • •

Histogram-based Bayes

•

Increase system reliability

Cascade

AdaBoost

• performance

A proper balance between complexity and

2010

• • • • •

[36]

•

Facial measurement of the lips

Yale

• • •

Absolute differences

Hamming distances

Biohashing

The error rates have been decreased

2011

•

Face

and eyes

Glasses

Mustache

Beard

Eye color

Hair color

Facial marks (scars, moles, freckles)

Ethnicity

Gender

Cloth color

Body location and size

Face color

Legs

Torso

Head

**Modalities**

**Database** Subset of the PETS 2006

—

**Techniques**

**Results**

Equal error rate of 6.1% is achieved

48 Machine Learning and Biometrics

tance (CPD)


**Table 2.** List of some of these works. Asma and Souhir et al. [40] use facial measurements and skin and hair color as soft biometric traits. Support vector machine as a classifier is evaluated using one dataset. Results show equal error rate is decreased and recognition rate improved and requires no more cost since soft biomet

ric traits are collected at the time of primary biometric collection by the same sensor. However, system needs to be tested with more difficult dataset and compared with another system. On the other hand, facial measurement features are very sensitive to pose and expression variation.

Nawaf and Nixon et al. [42] consider the eyebrow measurement distance and length from crowd sourcing. System is evaluated under one dataset with one classifier. Recognition rate increases but still needs to be tested with another dataset and compared with different classifi

ers. Jain and Park et al. [11] fuse face and facial marks. Their results show system performance increased up to 94.14%, but still facial mark extraction depends on the image resolution.

Min and Hadid et al. [37] propose facial occlusions as sunglasses, scarf, eye color, beard mus

compared with different systems and neural network needs more training data.

tache, and glasses' traits. Experimental result shows that facial occlusions affect the system performance especially when user tries to use it to prevent himself from being recognized. However, they used one dataset for evaluation and did not compare it with other systems. Chen and Huang et al. [44] define new soft biometric traits to describe people based on their clothes' type, color, and pattern. RCNN body detector is used. However, they used their own dataset taken under controlled environment for training the RCNN, so the system cannot be

Jain, Dass, and Nandakumar et al. [10] combine gender, height, and ethnicity as soft biometric traits with fingerprint. The system performance increased by 6%. However, soft biometric traits did not extract automatically, and the system is evaluated by 160 subjects only. Lee, Jain, and Jin et al. [29] achieve a recognition rate of 98.6% on Web-DB with good quality taken under controlled environ

ment and 77.2% on Michigan State Police Tattoo Database (MI-DB) using scale-invariant feature transform (SIFT) feature extractor. Experiment results show scars, marks, and tattoos (SMT) are more distinctive than other demographic biometrics such as ethnicity, gender, and weight to iden

tify a person. However, tattoo dataset is collected under controlled environment at booking time.

Batool, Nazre, and Sima et al. [41] report a classification accuracy of 88% for facial wrinkles as a soft biometrics using modified Hausdorff distance (MHD) algorithm. There is no stan

dard dataset to evaluate the system and compare with the other one. However, wrinkles are extracted manually by hand, and detecting wrinkles needs high-resolution image. Velardo, Carmelo, and Jean-Luc et al. [38] present a human body measurement (anthropometry) to prune primary biometric dataset. Their own medical dataset is collected from Indian hospital used for evaluating the body measurements and FERET data for face recognition. Results

Saini and Sinha et al. [36] integrate the face and facial measurement of the lips and eyes as distance between two pupils, distance between the eyes and the lips, and length of the lips and the eyes to improve the recognition rate using hamming, absolute difference, and bio

hashing distance techniques. Experiment results on Yale dataset show error rate is decreased. However biohashing performances are poor when the tokenized random numbers are com

promised; also only one dataset is used and results are not compared with another system.

show system accuracy and recognition speed increased.


51

A Survey on Soft Biometrics for Human Identification http://dx.doi.org/10.5772/intechopen.76021








Asma and Souhir et al. [40] use facial measurements and skin and hair color as soft biometric traits. Support vector machine as a classifier is evaluated using one dataset. Results show equal error rate is decreased and recognition rate improved and requires no more cost since soft biometric traits are collected at the time of primary biometric collection by the same sensor. However, system needs to be tested with more difficult dataset and compared with another system. On the other hand, facial measurement features are very sensitive to pose and expression variation.

Nawaf and Nixon et al. [42] consider the eyebrow measurement distance and length from crowd sourcing. System is evaluated under one dataset with one classifier. Recognition rate increases but still needs to be tested with another dataset and compared with different classifiers. Jain and Park et al. [11] fuse face and facial marks. Their results show system performance increased up to 94.14%, but still facial mark extraction depends on the image resolution.

Min and Hadid et al. [37] propose facial occlusions as sunglasses, scarf, eye color, beard mustache, and glasses' traits. Experimental result shows that facial occlusions affect the system performance especially when user tries to use it to prevent himself from being recognized. However, they used one dataset for evaluation and did not compare it with other systems. Chen and Huang et al. [44] define new soft biometric traits to describe people based on their clothes' type, color, and pattern. RCNN body detector is used. However, they used their own dataset taken under controlled environment for training the RCNN, so the system cannot be compared with different systems and neural network needs more training data.

Jain, Dass, and Nandakumar et al. [10] combine gender, height, and ethnicity as soft biometric traits with fingerprint. The system performance increased by 6%. However, soft biometric traits did not extract automatically, and the system is evaluated by 160 subjects only. Lee, Jain, and Jin et al. [29] achieve a recognition rate of 98.6% on Web-DB with good quality taken under controlled environment and 77.2% on Michigan State Police Tattoo Database (MI-DB) using scale-invariant feature transform (SIFT) feature extractor. Experiment results show scars, marks, and tattoos (SMT) are more distinctive than other demographic biometrics such as ethnicity, gender, and weight to identify a person. However, tattoo dataset is collected under controlled environment at booking time.

Batool, Nazre, and Sima et al. [41] report a classification accuracy of 88% for facial wrinkles as a soft biometrics using modified Hausdorff distance (MHD) algorithm. There is no standard dataset to evaluate the system and compare with the other one. However, wrinkles are extracted manually by hand, and detecting wrinkles needs high-resolution image. Velardo, Carmelo, and Jean-Luc et al. [38] present a human body measurement (anthropometry) to prune primary biometric dataset. Their own medical dataset is collected from Indian hospital used for evaluating the body measurements and FERET data for face recognition. Results show system accuracy and recognition speed increased.

Saini and Sinha et al. [36] integrate the face and facial measurement of the lips and eyes as distance between two pupils, distance between the eyes and the lips, and length of the lips and the eyes to improve the recognition rate using hamming, absolute difference, and biohashing distance techniques. Experiment results on Yale dataset show error rate is decreased. However biohashing performances are poor when the tokenized random numbers are compromised; also only one dataset is used and results are not compared with another system.

**Ref.**

[42]

•

Eyebrow size

2014

• •

[43]

•

Clothing attribute (head, upper

•

Soton Gait database

• SVM

•

Formulation of similarity constraints

Soft-margin ranking

The identification rate increased from 78% to 95%

50 Machine Learning and Biometrics

body, lower body, foot, attached

2014

[44]

Clothe color and type

Online shopping-labeled dataset

 • • •

SV regression

Multilevel CNN

Classification rate increased by 9% than support

vector machine

RCNN detector

Deep learning based

The performance increased but more dataset for

training the module needed

2015

[45]

Body parts (clothe, hair, color)

VIPeR and GRID dataset

2015

[46]

•

Eyebrow length

Labeled faces in the wild

• •

Deformable part model

GIST descriptor

Accuracy rate increased using comparative features

Dataset

2017

• • **Table 2.**

List of some of these works.

Noise width

Eye size

to body)

Eyebrow length

Eye-to-eyebrow distance

**Modalities**

**Database** Southampton with video

recordings from more than 200

subjects

**Techniques** Viola-Jones

**Results**

The performance increased to 100% when using ten

persons only

Tiwari S, Singh A, and Singh SK et al. [39] propose an optimal framework for newborn recognition by fusing match scores from face and soft biometrics. Results on IMS-BHU Indian hospital dataset show that soft biometrics improve recognition rate by 5.6% over the primary biometric. However framework evaluated on one dataset has high-resolution image taken under controlled pose and illumination.

**Author details**

South Africa

**References**

Abdelgader Abdelwhab1

and Technology, Khartoum, Sudan

Science & Business Media; 2013

ICT. 2014;**2**(2):129-140

and Serestina Viriri2

\*Address all correspondence to: viriris@ukzn.ac.za

\*

A Survey on Soft Biometrics for Human Identification http://dx.doi.org/10.5772/intechopen.76021 53

1 College of Computer Science and Information Technology, Sudan University of Science

2 School of Maths, Statistics and Computer Science, University of KwaZulu-Natal, Durban,

[1] Zhang DD. Automated Biometrics: Technologies and Systems. Vol. 7. Berlin: Springer

[3] Clarke R. Human identification in information systems: Management challenges and

[4] Srinivasa K, Gosukonda S. Continuous multimodal user authentication: Coupling hard and soft biometrics with support vector machines to attenuate noise. CSI transactions on

[5] Franke K, Ruiz-del-Solar J, Koppen M. Soft-biometrics: Soft-computing for biometric-

[6] Lu X, Jain AK. Ethnicity identification from face images. In: Defense and Security.

[7] Jain AK, Ross A, Prabhakar S.An introduction to biometric recognition. IEEE Transactions

[8] Jain A, Verma CK. A framework based on hybrid biometrics for personal verification

[9] Kim M-G, Moon H-M, Chung Y, Pan SB. A survey and proposed framework on the soft biometrics technique for human identification in intelligent video surveillance system.

[10] Jain AK, Dass SC, Nandakumar K. Can soft biometric traits assist user recognition? In: Defense and Security. International Society for Optics and Photonics; 2004. pp. 561-572

[11] Jain AK, Park U. Facial marks: Soft biometric for face recognition. In: 2009 16th IEEE

[12] Dantcheva A, Elia P, Ross A. What else does your biometric data reveal? A survey on soft biometrics. IEEE Transactions on Information Forensics and Security. 2016;**11**(3):441-467

International Conference on Image Processing (ICIP). IEEE; 2009. pp. 37-40

[2] Shoniregun CA. The future of internet security. Ubiquity. 2002;**2002**:8

public policy issues. Information Technology & People. 1994;**7**(4):6-37

applications. International Journal of Fuzzy Systems. 2002;**4**(2):665

International Society for Optics and Photonics; 2004. pp. 114-123

on Circuits and Systems for Video Technology. 2004;**14**(1):4-20

systems. International Journal of Applied. 2012;**1**(1):55-58

BioMed Research International. 2012;**2012**

Jaha, Emad, and Mark et al. [43] show clothing traits can be used for identification of individual where clothing descriptions might be the only available feature. An, Chen, Kafai, Yang, and Bhanu et al. [49] aim to improve the re-identification performance by re-ranking the returned results based on soft biometric attributes. Experiments on challenging benchmark VIPeR dataset show that reranking improves the recognition accuracy.
