**Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics**

Hiroshi Dozono

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/52100

## **1. Introduction**

Password mechanisms are widely used for the authentication method. However, Password mechanism has many issues. For examples, Password can be stolen easily, Password may be guessed from personal information, such as birthday, families name or telephone number. Some users set unique password to different systems. If one system is hacked, all of the sys‐ tems can be accessed. Some users feel troublesome to memorize the password. For these problems, biometric authentication is one of the solutions.

Biometric authentication [1] is classified into two types. The first one is the biometric au‐ thentication with biological characteristics, such as fingerprint, Vein patterns and Iris pat‐ terns. To measure these characteristics, the additional hardware is necessary, and it costs up the computer system. And, some users may feel mentally uncomfortable to register their fin‐ gerprint to the computer system. Furthermore, static information about biological character‐ istics may be imitated by dummy. For example, the fingerprint authentication is easily hacked in the TV show.

The second type is biometric authentications with behavior characteristics, such as key‐ stroke timings [2], Signature [3], hand written pattern and mouse moving pattern. For these methods, the standard input equipments of computer are available. The dynamic informa‐ tion about behavior characteristics is hard to imitate even if it is looked by illegal one. How‐ ever, the accuracy of authentication is worse compared with that of biological characteristics. For use behavior characteristics, it is necessary to select the pattern of behav‐ ior and the features used for authentication. For this selection, Self Organizingmap(SOM)s are used for the analysis in our research.

© 2012 Dozono; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2012 Dozono; licensee InTech. This is a paper distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

SOM [4] is the architecture of neural networks, which is feedforward type and single layer network. SOM organizes the map which reflects the similarities of input vectors; thus SOM can visualize the relations among the input vector on the lower dimensional, usually 2-di‐ mensional map. SOM is often used for the visualization of the multidimensional data. SOM is also applied to the authentication with biological characteristics, such as facial recognition system.

sional map. The closest neuron is called as winner neuron. The winner neuron and the neurons in the region of the neighborhood are updated as to decrease the difference to the input vector depending on the learning rate and neighboring functions. These steps are iterated for each in‐

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

Neurons on 2 dimensional map

(Nearest neuron) Search for the winner

Update as to decrease the difference

Learning rate

Vector assigned to the neuron

Input vector

http://dx.doi.org/10.5772/52100

121

Winner

put vector with decreasing the region of the neighborhood and learning rate.

Region of the neighborhood - decreased in iterated steps

**3. Application of SOM to the authentication system using handwritten**

Recently, many mobile devices, such as Smartphones, tablet devices and small computers, are equipped with touch screen. As the authentication method for touch screen devices, the pass‐ word authentication is often used. But, on touch screen devices, password can be looked while typing on the screen. It is troublesome to enter the password using handwritten character rec‐ ognition or screen keyboard. For the touch screen devices, handwritten signature authentica‐ tion is often applied, because the touch screen is considered to be useful for signature input. However, the shape of the signature may be copied, and it is difficult to write the exact signa‐

For this problem, we propose a user authentication method using the identical symbol for all users. Using this method, the symbol which is used for authentication is displayed on the touch screen and users simply trace it. However, the pen stroke data may not be enough for user authentications. We used the pen pressure data which may have enough information for user authentication. For this purpose, We analyzed the pen stroke data and pen pressure

ture on slippery screen, especially for people who do not usually write signatures.

**Figure 1.** Schematic description of SOM algorithm

data using Self Organizing Map [5] [6].

**patterns**

At first, SOM is applied to the authentication systems of behavior biometrics of pen calligra‐ phies [5] [6] and keystroke timing [7] in this research. SOM is used for the analysis of input data to select the appropriatepattern of behavior and featureswith visualizing the input data on the map, and also for constructing an authentication system.

However, the accuracy of single behavior biometrics is not enough. For this problem, Pareto learning SOM(P-SOM) and Supervised Pareto learning SOM(SP-SOM), which can integrate multi-modal behavior biometrics [8], is proposed, and applied to the authentication system using keystroke timing and pen calligraphy [9] [10].

Furthermore, the multi-modal authentication system using keystroke timing and key typing sound, which can be obtained at the same time, is proposed [11]. Additionally, the incremental learning of the biometric data during the authentication is applied to implement the adaptive authentication system which can follow the changes of the biometrics of time [12] [13].

SP-SOM shows satisfactory performance as authentication system. However, SP-SOM needs to learn data of some users. For mobile devices, the number of users is usually one; thus the system may need dummy data. For this problem, Concurrent Full Pareto learning SOM(CFP-SOM), which uses a small map for each user, is applied. CFP-SOM can detect the unregistered user using the size of the Pareto set as index, and shows better performance than the SP-SOM [14].

In this chapter, SOM and its application to biometric authentication system are mentioned in section 2 and 3,4 respectively. In section 5 and 6, application of the SP-SOM to multi-modal authentication system and its extension to adaptive authentication system are mentioned re‐ spectively. In section 7, application of CFP-SOM to the multi-modal authentication system is mentioned.

## **2. Self Organizing Map (SOM)**

SOM [4] is a kind of neural network, which was proposed by Kohonen, and SOM can extract the feature on the multidimensional input vectors and can visualize the relations among them by unsupervised learning. SOM can integrate multi-modal input vectors and can ex‐ tract relations among them in 2-dimensional plane. SOM can be used for clustering of unla‐ beled data or classification of labeled data with labeling the output units after learning.

Figure 1 shows the basic learning algorithm of SOM. For each input vector, the neuron, which is closest to the input vector, is searched from the neurons which are arranged on the 2 dimen‐ sional map. The closest neuron is called as winner neuron. The winner neuron and the neurons in the region of the neighborhood are updated as to decrease the difference to the input vector depending on the learning rate and neighboring functions. These steps are iterated for each in‐ put vector with decreasing the region of the neighborhood and learning rate.

**Figure 1.** Schematic description of SOM algorithm

SOM [4] is the architecture of neural networks, which is feedforward type and single layer network. SOM organizes the map which reflects the similarities of input vectors; thus SOM can visualize the relations among the input vector on the lower dimensional, usually 2-di‐ mensional map. SOM is often used for the visualization of the multidimensional data. SOM is also applied to the authentication with biological characteristics, such as facial recognition

At first, SOM is applied to the authentication systems of behavior biometrics of pen calligra‐ phies [5] [6] and keystroke timing [7] in this research. SOM is used for the analysis of input data to select the appropriatepattern of behavior and featureswith visualizing the input data

However, the accuracy of single behavior biometrics is not enough. For this problem, Pareto learning SOM(P-SOM) and Supervised Pareto learning SOM(SP-SOM), which can integrate multi-modal behavior biometrics [8], is proposed, and applied to the authentication system

Furthermore, the multi-modal authentication system using keystroke timing and key typing sound, which can be obtained at the same time, is proposed [11]. Additionally, the incremental learning of the biometric data during the authentication is applied to implement the adaptive

SP-SOM shows satisfactory performance as authentication system. However, SP-SOM needs to learn data of some users. For mobile devices, the number of users is usually one; thus the system may need dummy data. For this problem, Concurrent Full Pareto learning SOM(CFP-SOM), which uses a small map for each user, is applied. CFP-SOM can detect the unregistered user using the size of the Pareto set as index, and shows better performance

In this chapter, SOM and its application to biometric authentication system are mentioned in section 2 and 3,4 respectively. In section 5 and 6, application of the SP-SOM to multi-modal authentication system and its extension to adaptive authentication system are mentioned re‐ spectively. In section 7, application of CFP-SOM to the multi-modal authentication system is

SOM [4] is a kind of neural network, which was proposed by Kohonen, and SOM can extract the feature on the multidimensional input vectors and can visualize the relations among them by unsupervised learning. SOM can integrate multi-modal input vectors and can ex‐ tract relations among them in 2-dimensional plane. SOM can be used for clustering of unla‐ beled data or classification of labeled data with labeling the output units after learning.

Figure 1 shows the basic learning algorithm of SOM. For each input vector, the neuron, which is closest to the input vector, is searched from the neurons which are arranged on the 2 dimen‐

authentication system which can follow the changes of the biometrics of time [12] [13].

on the map, and also for constructing an authentication system.

using keystroke timing and pen calligraphy [9] [10].

120 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

system.

than the SP-SOM [14].

**2. Self Organizing Map (SOM)**

mentioned.

## **3. Application of SOM to the authentication system using handwritten patterns**

Recently, many mobile devices, such as Smartphones, tablet devices and small computers, are equipped with touch screen. As the authentication method for touch screen devices, the pass‐ word authentication is often used. But, on touch screen devices, password can be looked while typing on the screen. It is troublesome to enter the password using handwritten character rec‐ ognition or screen keyboard. For the touch screen devices, handwritten signature authentica‐ tion is often applied, because the touch screen is considered to be useful for signature input. However, the shape of the signature may be copied, and it is difficult to write the exact signa‐ ture on slippery screen, especially for people who do not usually write signatures.

For this problem, we propose a user authentication method using the identical symbol for all users. Using this method, the symbol which is used for authentication is displayed on the touch screen and users simply trace it. However, the pen stroke data may not be enough for user authentications. We used the pen pressure data which may have enough information for user authentication. For this purpose, We analyzed the pen stroke data and pen pressure data using Self Organizing Map [5] [6].

1

2 3

1

2

3

**Figure 4.** Organizied map for simple symbols

**Figure 5.** Organized map for symbols spiral and complex star

and complex star were taken from each of 10 users.

data. The maps were retrained by LVQ3 algorithm.

pressure data and pen speed data.

for next experiments.

Figure 4 shows the maps of each symbol. The numbers in these figures denote the usedid. With these figures, the symbols of circle and star show better separations compared with others. We consider that the symbols comprised of oval lines or acute angle have more specific features of each user. The symbols of spiral and complex star are selected

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

123

Figure 5 shows the maps of the symbols spiral and complex star. We use the torus map for this analysis, so the upper side of the map is connected to the lower side, and the right side is connected to the left side. Both of the symbols star and spiral show better separations compared with the simple symbols. It will be possible to authenticate the user using pen

The authentication experiments using these symbols are conducted. As the authentication system, we used SOM. The settings of the experiments are as follows. 10 samples of spiral

7 samples of each person were used for training SOM map, and 3 samples were used as test

**Figure 2.** Test input screen

**Figure 3.** Pen input of symbol star

Figure 2 shows the screen of the application which measures the pen stroke data and pen pressure data of symbols square, circle triangle and star. Each symbol is written in a single stroke, and starting point is specified for all users. 5 samples are taken from each of 12 users. Figure 3 shows an example of symbol star taken from a user. This sample contains 232 points of x-axis, y-axis and pen pressure data in z-axis.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics http://dx.doi.org/10.5772/52100 123

**Figure 4.** Organizied map for simple symbols

1

2 3

1

2

3

**Figure 2.** Test input screen

122 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

**Figure 3.** Pen input of symbol star

points of x-axis, y-axis and pen pressure data in z-axis.

Figure 2 shows the screen of the application which measures the pen stroke data and pen pressure data of symbols square, circle triangle and star. Each symbol is written in a single stroke, and starting point is specified for all users. 5 samples are taken from each of 12 users. Figure 3 shows an example of symbol star taken from a user. This sample contains 232

Figure 4 shows the maps of each symbol. The numbers in these figures denote the usedid. With these figures, the symbols of circle and star show better separations compared with others. We consider that the symbols comprised of oval lines or acute angle have more specific features of each user. The symbols of spiral and complex star are selected for next experiments.

**Figure 5.** Organized map for symbols spiral and complex star

Figure 5 shows the maps of the symbols spiral and complex star. We use the torus map for this analysis, so the upper side of the map is connected to the lower side, and the right side is connected to the left side. Both of the symbols star and spiral show better separations compared with the simple symbols. It will be possible to authenticate the user using pen pressure data and pen speed data.

The authentication experiments using these symbols are conducted. As the authentication system, we used SOM. The settings of the experiments are as follows. 10 samples of spiral and complex star were taken from each of 10 users.

7 samples of each person were used for training SOM map, and 3 samples were used as test data. The maps were retrained by LVQ3 algorithm.


**4. Application of SOMto the authentication method using keystroke**

tication, is selected by the analysis using SOM [7].

It is well known that keystroke timing is usable for user authentication. We propose an au‐ thentication method which uses the keystroke timings of identical phrase for all users. Users do not need to memorize phrases. For this purpose, the phrase, which is suitable for authen‐

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

125

2 Push A Release Push B Release Push C Release Inputted phrase 3 **|------------| |-------------| |----------------| abc**

The method for taking keystroke timing is dependent on the Operating System(OS)s. The keystroke timing is the vector of intervals between pushing and releasing keys. We use this sequence as the vector of keystroke timing. The length of the vector is 2L-1, where the length

The experiments are conducted using Romaji Phrases because the examinees are always typ‐ ing Japanese using Romaji. As samples of phrases, "arigato"(Thank you in English), "kira‐ kira"(Twinkle Twinkle), "denatsu"(Voltage), "sagadai"(name of our university) and "kousatsu"(prospect) are used. The number of examinees is 10, and each examinee types

denatsup kirakita

6 6 1 1 2 3 6 3 8 8 8 1 1 8 11 11 6 6 5 5 2 6 6 6 6 6 6 1 1 9 11 4 4 6 6 5 9 9 6 5 5 6 6 6 9 9 9 4 4 4 6 6 9 9 7 7 7 6 2 2 2 9 4 4 4 4 1 9 9 9 8 7 7 8 2 2 2 8 8 4 4 4 1 1 2 8 8 8 8 8 8 2 2 2 8 4 4 4 1 5 2 2 8 10 8 8 7 7 2 2 2 4 10 4 5 5 5 10 10 10 10 7 7 7 7 2 2 10 10 4 4 5 10 10 10 10 7 7 7 7 11 11 10 10 4 4 4 7 7 10 11 11 7 7 7 7 11 11 11 10 11 11 7 7 7 7 11 11 11 7 3 3 5 11 11 11 11 11 9 7 7 7 11 11 11 10 3 3 3 5 11 11 11 3 3 5 5 3 9 9 10 10 3 3 3 9 9 11 3 3 3 5 5 3 9 9 10 10 10 3 9 9 9 9 3 3 1 1 5 3 9 3 8 10 10 3 1 9 9 9 6 1 1 1 1 3 3 3 8 8 8 1 1 1 11 11

4 Key stroke 100ms 90ms 60ms 110ms 100ms

6 Keystroke timing data for abc = (100,90,60,110,100)

9 1 2 2 11 11 11 7 7 7 3 3 8 8 6 5 9 1 2 2 11 11 11 7 7 8 3 3 3 6 6 6 9 9 2 11 11 11 11 11 8 8 8 3 3 6 6 6 9 9 9 11 11 11 11 11 3 8 8 3 3 6 6 6 10 9 10 10 11 11 11 3 3 3 8 5 5 6 6 6 10 10 10 10 11 11 11 3 3 5 5 5 5 5 6 10 10 9 10 4 10 11 11 5 5 5 5 5 5 5 9 10 4 9 4 10 10 10 10 10 5 5 5 5 5 9 9 9 4 4 4 4 10 2 7 7 10 10 10 8 8 9 9 4 4 2 4 2 2 2 7 7 10 10 10 8 8 8 1 1 1 2 2 1 2 2 7 7 10 10 10 8 8 8 1 1 2 2 1 1 1 7 7 7 7 5 5 5 4 4 1 1 4 4 2 2 7 7 7 7 7 5 5 5 4 4 4 1 4 4 2 2 2 7 7 7 7 5 5 6 8 4 1 1 1 8 2 2 2 11 11 7 7 6 6 6 8 1 1 1 5 1 1 2 11 11 11 7 7 6 6 3 8 8 1 5

**Figure 7.** Organized maps for keystroke timings "denatsu" and "kirakita"

**timings**

1

7 **Figure 6.** Keystroke timings

of the phrase is L

each phrase in 8 times.

5 timings

**Table 1.** Results of authentication experiments

Table 1 shows the result. For the learned data both of the spiral and complex star show high rate of authentication and low rate of false acceptance, but for the test data, rate of successful authentication was about 70%. From the table of spiral test data, users 6,9,10 show low rate of successful authentication, and from the table of star, same users are not authenticated at all. Both of the symbols can authenticate 7 users from 10 users. It depends of the characters of the users. The careful users tend to be authenticated better and careless users tend to be rejected falsely.

## **4. Application of SOMto the authentication method using keystroke timings**

It is well known that keystroke timing is usable for user authentication. We propose an au‐ thentication method which uses the keystroke timings of identical phrase for all users. Users do not need to memorize phrases. For this purpose, the phrase, which is suitable for authen‐ tication, is selected by the analysis using SOM [7].

> Push A Release Push B Release Push C Release Inputted phrase  **|------------| |-------------| |----------------| abc** Key stroke 100ms 90ms 60ms 110ms 100ms 5 timings Keystroke timing data for abc = (100,90,60,110,100)

#### 7 **Figure 6.** Keystroke timings

1

**Table 1.** Results of authentication experiments

124 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

rejected falsely.

Table 1 shows the result. For the learned data both of the spiral and complex star show high rate of authentication and low rate of false acceptance, but for the test data, rate of successful authentication was about 70%. From the table of spiral test data, users 6,9,10 show low rate of successful authentication, and from the table of star, same users are not authenticated at all. Both of the symbols can authenticate 7 users from 10 users. It depends of the characters of the users. The careful users tend to be authenticated better and careless users tend to be The method for taking keystroke timing is dependent on the Operating System(OS)s. The keystroke timing is the vector of intervals between pushing and releasing keys. We use this sequence as the vector of keystroke timing. The length of the vector is 2L-1, where the length of the phrase is L

The experiments are conducted using Romaji Phrases because the examinees are always typ‐ ing Japanese using Romaji. As samples of phrases, "arigato"(Thank you in English), "kira‐ kira"(Twinkle Twinkle), "denatsu"(Voltage), "sagadai"(name of our university) and "kousatsu"(prospect) are used. The number of examinees is 10, and each examinee types each phrase in 8 times.


**Figure 7.** Organized maps for keystroke timings "denatsu" and "kirakita"

Figure 7 shows the maps of the phrases "denatsu" and "kirakira". Comparing these 2 maps, the map of "kirakira" shows better clustering results of user-id. The simple phrase "kira‐ kira" is considered to be suitable for authentication because users can type unconsciously.

written symbol at login time, keystroke timing and key typing sound at login and keystroke timings and mouse moving patterns during the operating time. For the integration of multimodal biometrics, we propose the Pareto learning SOM(P-SOM). Furthermore, we propose the Supervised P-SOM(SP-SOM) which can improve the accuracy of the authentication.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

Generally speaking, the multi modal vector is the vector composed of multi-kind of vectors or attributes. For examples, in the authentication problem using key typing features, the keystroke timing vector and key typing intensity vector are composed. For face image classi‐ fication, the image vector, age, gender, jobs and other features are composed. In multi-mo‐ dal vector, each element of the vector and the attribute is described in a different unit and

Conventional SOM can be applied for integrating multi-modal vectors. For example, the

2 *ij n n*

The map is organized based on the value of the error. So, the resulting map is dominated by the largely scaled vectors and easily affected by inaccurate vector. For this problem, the con‐ catenated vector (*w*1*x*1*,w2x*2*, …, wmx*m) with weight values is often used. Then the quantiza‐

> 2 *ij nn n*

So, the resulting map heavily depends on the weight values. It is difficult to select the opti‐

Same situations occur in multi-objective optimization problem. Consider the problem, Sub‐ ject to *x* ∈ *S*, minimize multiple objective functions Fi(*x*). To solve this problem as a single

å *x m*- (1)

http://dx.doi.org/10.5772/52100

127

å *wx m*- (2)

*Fx wF x* <sup>=</sup> å (3)

simple concatenated vector (*x*1*, x*2*, …, x*m) can be used as the input vector.

*n*

*n*

objective optimization problem, the weighted sum of multi objective functions.

() () *n n n*

is minimized. However, the quality of the solution depends on the setting of weight values.

For this problem, the concept of Pareto optimal is proposed in multi-objective optimiza‐

**5.1. Pareto leatning SOM [15]**

tion error is calculated in (2).

mal value of the weights.

tion problem.

scale. Accuracy of each element may differ.

Then, the quantization error is calculated in (1).

Next, we conducted authentication experiments using the map organized by SOM. The half of the keystroke timing data is used for learning, and the remained half is used for authentication experiments.As the indexes for evaluation, FRR which is the rate of rejec‐ tion of the regular user falsely and FAR which is the rate of acceptance of the irregular user falsely, are used.


**Table 2.** Results of authentication experiments using keystroke timings

Table 2 shows the results of authentication experiments using the phrases of "denatsu" and "kirakira". As expected from the map, the phase "kirakira" shows the better result. On aver‐ age, rate of false rejections about 32%. Some users show a remarkably high rate of false rejec‐ tion. It depends on the skill of typing.

## **5. Integration of muti-modal biometrics using pareto learning SOM**

As shown before, accuracy of authentication using behavior characteristics is worse com‐ pared with biological characteristics. It is due to the variation of the behavior characteris‐ tics and noise. We consider that Integration of some behavior characteristics will improve the accuracy.

We proposed the authentication methods using the integrated information of multi-modal behavior characteristics to improve the accuracy. For example, keystroke timing and hand‐ written symbol at login time, keystroke timing and key typing sound at login and keystroke timings and mouse moving patterns during the operating time. For the integration of multimodal biometrics, we propose the Pareto learning SOM(P-SOM). Furthermore, we propose the Supervised P-SOM(SP-SOM) which can improve the accuracy of the authentication.

#### **5.1. Pareto leatning SOM [15]**

Figure 7 shows the maps of the phrases "denatsu" and "kirakira". Comparing these 2 maps, the map of "kirakira" shows better clustering results of user-id. The simple phrase "kira‐ kira" is considered to be suitable for authentication because users can type unconsciously.

Next, we conducted authentication experiments using the map organized by SOM. The half of the keystroke timing data is used for learning, and the remained half is used for authentication experiments.As the indexes for evaluation, FRR which is the rate of rejec‐ tion of the regular user falsely and FAR which is the rate of acceptance of the irregular

denatsu kirakira

Table 2 shows the results of authentication experiments using the phrases of "denatsu" and "kirakira". As expected from the map, the phase "kirakira" shows the better result. On aver‐ age, rate of false rejections about 32%. Some users show a remarkably high rate of false rejec‐

**5. Integration of muti-modal biometrics using pareto learning SOM**

As shown before, accuracy of authentication using behavior characteristics is worse com‐ pared with biological characteristics. It is due to the variation of the behavior characteris‐ tics and noise. We consider that Integration of some behavior characteristics will

We proposed the authentication methods using the integrated information of multi-modal behavior characteristics to improve the accuracy. For example, keystroke timing and hand‐

user falsely, are used.

126 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

**Table 2.** Results of authentication experiments using keystroke timings

tion. It depends on the skill of typing.

improve the accuracy.

Generally speaking, the multi modal vector is the vector composed of multi-kind of vectors or attributes. For examples, in the authentication problem using key typing features, the keystroke timing vector and key typing intensity vector are composed. For face image classi‐ fication, the image vector, age, gender, jobs and other features are composed. In multi-mo‐ dal vector, each element of the vector and the attribute is described in a different unit and scale. Accuracy of each element may differ.

Conventional SOM can be applied for integrating multi-modal vectors. For example, the simple concatenated vector (*x*1*, x*2*, …, x*m) can be used as the input vector.

Then, the quantization error is calculated in (1).

$$\sum\_{n} \left| \mathbf{x}\_{n} - m\_{n}^{\ell} \right|^{2} \tag{1}$$

The map is organized based on the value of the error. So, the resulting map is dominated by the largely scaled vectors and easily affected by inaccurate vector. For this problem, the con‐ catenated vector (*w*1*x*1*,w2x*2*, …, wmx*m) with weight values is often used. Then the quantiza‐ tion error is calculated in (2).

$$\sum\_{n} \left| w\_n x\_n - m\_n^{\psi} \right|^2 \tag{2}$$

So, the resulting map heavily depends on the weight values. It is difficult to select the opti‐ mal value of the weights.

Same situations occur in multi-objective optimization problem. Consider the problem, Sub‐ ject to *x* ∈ *S*, minimize multiple objective functions Fi(*x*). To solve this problem as a single objective optimization problem, the weighted sum of multi objective functions.

$$F(\mathbf{x}) = \sum\_{n} w\_{n} F\_{n}(\mathbf{x}) \tag{3}$$

is minimized. However, the quality of the solution depends on the setting of weight values.

For this problem, the concept of Pareto optimal is proposed in multi-objective optimiza‐ tion problem.

Figure 9 shows the learning process of SOM and Pareto learning SOM. As for SOM, only one winner is selected and the winner and its neighbors are updated. As for pareto learning SOM, pareto winner set are selected, and they are updated simultaneously. For Pareto learn‐ ing SOM, overlapped neighbors are updated more strongly, and it play an important role for integration of muti-modal vectors. In other word, conventional SOM integrates the mul‐ ti-modal vector in a unit and P-SOM integrates the multi-modal vector in the region of Pare‐

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

**1.** Initialization of the map: Initialize the vector **m**ij which are assigned to unit Uij on the map using the 1st and 2nd principal components as base vectors of 2-dimensional map.

**3.** Batch update phase: For each unit Uij update the associated vector **m**ij using the weighted average of the vectors recorded in the buffer of Uij and its neighboring units as follows.

**•** For all vectors **x** recorded in the buffer of Uij and its neighboring units in distance d ≤ Sn,

where Ui′j′s are neighbors of Uij including Uij itself, η is learning rate, fn(d) is the neighbor‐

As shown in step 2 of this algorithm, Pareto winner set for the integrated input vector **x** are searched for based on the concept of Pareto Optimality using the distance as the objective function fh(**x**) for each element xh in **x**. Thus, the multiple units become winners. The winners and their neighboring units are modified in the update process in step 3. Overlapped neigh‐ bors are updated multiply, and the overlapped region will contribute to generalization abili‐

Because Pareto learning SOM can integrate any type of vectors, the category element can be

as follows.

calculate weighted sum **S** of the updates and the sum of weight values W.

hood function which becomes 1 for d=0 and decrease with increment of d.

Repeat 2. and 3. with decreasing the size of neighbors Sn for pre-defined iterations.

<sup>p</sup> ∈ P.

p}. Uabp is an

p existing h such that eabh≤

http://dx.doi.org/10.5772/52100

129

. **•** For each vector xi, search for the pareto optimal set of the units P ={Uab

element of pareto optimal set P, if for all units Ukl∈ P−Uab

to optimal set. Algorithm of P-SOM is as follows.

**•** Clear all learning buffer of units Uij

h= | **x**<sup>i</sup>

′)

h − **m**kl h|.

to the learning buffer of all units Uab

P-SOM Algorithm

ekl

**•** Add **x**<sup>i</sup>

*S* =*S* + *ηfn*(*d*)(*x* −*m <sup>i</sup>*′ *<sup>j</sup>*

**•** Set the vector **m**ij = **m**ij + **S**/W.

ty and integration ability of P-SOM.

**5.2. Supervised paretolearning SOM (SP-SOM)**

introduced as an independent vector to each input vector *x*<sup>i</sup>

*W* = *W* + *fn*(*d*)

**2.** Batch learning phase:

h where ekl

**Figure 8.** Pareto Optimality

Suppose that the objective function f1 and f2 should be minimized under the condition f1 and f2 should be the upper right side of this line. For this problem, P1 is better than P2 con‐ cerning f1, but P2 is better than P1 concerning f2. If no priority is given to f1 and f2, these 3 points P1, P2, P3 are not inferior to others among them. These points are named as Pareto optimal set, and they are the candidates of optimal solutions.

Pareto Learning Self Organizing Map(P-SOM) uses the concept of Pareo optimality for finding winner units. The error of each element of multi-modal vector is considered as the objective functions fn(*x,*Uij)=|*x*n-*m*ij n|, where *x*=({*x*1},{*x*2},…,{*x*m}) is the input vector and*m*ij=({*m*ij 1},{*m*ij 2},…, {*m*ij m}) is the reference vector associated to the unit Uij. Pareto optimal set P(x) is the set of units Uij which are pareto optimal for the objective functions fn(*x,*Uij). So, the Pareto SOM is multiwinner SOM and all winner units and their neighbors are updated simultaneously.

**Figure 9.** Difference between conventional SOM and Pareto learning SOM

Figure 9 shows the learning process of SOM and Pareto learning SOM. As for SOM, only one winner is selected and the winner and its neighbors are updated. As for pareto learning SOM, pareto winner set are selected, and they are updated simultaneously. For Pareto learn‐ ing SOM, overlapped neighbors are updated more strongly, and it play an important role for integration of muti-modal vectors. In other word, conventional SOM integrates the mul‐ ti-modal vector in a unit and P-SOM integrates the multi-modal vector in the region of Pare‐ to optimal set. Algorithm of P-SOM is as follows.

P-SOM Algorithm

Blue points: Pareto optimal set

P1

optimal set, and they are the candidates of optimal solutions.

m}) is the reference vector associated to the unit Uij

**Figure 9.** Difference between conventional SOM and Pareto learning SOM

which are pareto optimal for the objective functions fn(*x,*Uij

winner SOM and all winner units and their neighbors are updated simultaneously.

**Figure 8.** Pareto Optimality

functions fn(*x,*Uij

*Uij*

{*m*ij

Uij

)=|*x*n-*m*ij

128 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

P2

P3

Suppose that the objective function f1 and f2 should be minimized under the condition f1 and f2 should be the upper right side of this line. For this problem, P1 is better than P2 con‐ cerning f1, but P2 is better than P1 concerning f2. If no priority is given to f1 and f2, these 3 points P1, P2, P3 are not inferior to others among them. These points are named as Pareto

Pareto Learning Self Organizing Map(P-SOM) uses the concept of Pareo optimality for finding winner units. The error of each element of multi-modal vector is considered as the objective

n|, where *x*=({*x*1},{*x*2},…,{*x*m}) is the input vector and*m*ij

*<sup>w</sup>* argmin*<sup>i</sup>*, *<sup>j</sup>* **X m***ij ij*

Conventional SOM Pareto learning SOM

=({*m*ij

). So, the Pareto SOM is multi-

. Pareto optimal set P(x) is the set of units

*kl kl h*

*ij h ij P U <sup>p</sup> h e e U P U* (*x*) | , ,

1},{*m*ij

*p*

2},…,

	- **•** Clear all learning buffer of units Uij .
	- **•** For each vector xi, search for the pareto optimal set of the units P ={Uab p}. Uabp is an element of pareto optimal set P, if for all units Ukl∈ P−Uab p existing h such that eabh≤ ekl h where ekl h= | **x**<sup>i</sup> h − **m**kl h|.
	- **•** Add **x**<sup>i</sup> to the learning buffer of all units Uab <sup>p</sup> ∈ P.

$$\mathbf{S} = \mathbf{S} + \eta f \mathbf{n}(d) (\mathbf{x} - \mathbf{m}^{i'j'} \mathbf{y}')$$

$$W = W \, \text{ + } fh(d)$$

where Ui′j′s are neighbors of Uij including Uij itself, η is learning rate, fn(d) is the neighbor‐ hood function which becomes 1 for d=0 and decrease with increment of d.

**•** Set the vector **m**ij = **m**ij + **S**/W.

Repeat 2. and 3. with decreasing the size of neighbors Sn for pre-defined iterations.

As shown in step 2 of this algorithm, Pareto winner set for the integrated input vector **x** are searched for based on the concept of Pareto Optimality using the distance as the objective function fh(**x**) for each element xh in **x**. Thus, the multiple units become winners. The winners and their neighboring units are modified in the update process in step 3. Overlapped neigh‐ bors are updated multiply, and the overlapped region will contribute to generalization abili‐ ty and integration ability of P-SOM.

#### **5.2. Supervised paretolearning SOM (SP-SOM)**

Because Pareto learning SOM can integrate any type of vectors, the category element can be introduced as an independent vector to each input vector *x*<sup>i</sup> as follows.

$$\boldsymbol{\infty}\_i^s = (\boldsymbol{\infty}\_i \boldsymbol{\mathcal{C}}\_i)$$

where

$$\mathcal{C}\_{\vec{i}} = \begin{cases} 1 & \mathfrak{x}\_{\vec{i}^\*} \in \mathcal{C}\_{\vec{i}} \\ 0 & \mathfrak{x}\_{\vec{i}} \nightsquigarrow & \mathcal{C}\_{\vec{i}^\*} \end{cases}$$

With introducing category element, it attracts the input vectors in the same category closely on the map corporately with other input vectors in the learning phase. In this meaning, P-SOM learning algorithm becomes supervised. In the recalling process, category of test vector **x**t is determined by the following equation.

1

2

4 5 6

3 Key stroke timing Pen speed and pen pressure

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

**Table 3.** Result of Authentication experiment using each of behavior biometrics

data are much worse than that of keystroke timing.

Table 3 shows the result of authentication using each data independently. In this experi‐ ment, FRR of keystroke timings is much better than that of the previous experiment because the typing skill of each user becomes much better. FRR of pen speed data and pen pressure

Table 4 shows the results of authentication experiments using integrated vectors of key‐ stroke timing, pen speed and pen pressure. The weight values for pen speed data and pen pressure data are studiously selected from many iterations of try and error for SOM. SP-SOM can achieve almost same authentication performance without tunings of parameters.

**Figure 10.** Organized map of SP-SOM with behavior biometrics data

Integration of key stroke timing, pen speed and pen

> p r e

http://dx.doi.org/10.5772/52100

131

arg max*<sup>k</sup>* { ∑ *U ij* ∈*P*(*xt*) *ck ij* }

where P(*x*<sup>t</sup> ) is the Pareto optimal set of units for **x**<sup>t</sup>

Considering the Further extension of Pareto learning SOM, anything which has its own met‐ rics can be element of multi-modal input vector for P-SOM. Structured data and vectors can be integrated as input data. For example, in bioinformatics amino acid sequence data com‐ prised of Hidden Markov Model(HMM) of the data, HMM of the structures and other fea‐ tures can be integrated. Furthermore, the map can be organized using the partial vectors which lack some attributes in the vectors.

#### **5.3. Experiment of authentication system using multi-modal behavior biometrics**

The experimental results of the authentication system using handwritten patterns and key‐ stroke timings were shown in the previous section. Another experiments using both of handwritten patterns and keystroke timings were conducted [9] [10]. The experimental set‐ tings are as follows. 6 samples are taken from each of 11 examinees. Each sample is com‐ prised of the keystroke timings and pen calligraphy data which is inputted alternately. For these experiments, tablet PC, which is equipped with keyboard and touch panel, is used.

The phrase "kirakira" which marked the best results in the previous experiments is used. The symbol Spiral is used as the handwritten symbol. SP-SOM is used for the analysis and authentication.

Figure 10 shows the map of SP-SOM using behavior biometrics data, key stroke timing, pen speed and pen pressure. Using keystroke timing, the users are clustered well except small fragmentations. Using pen speed and pen pressure, thefragmentations are increased com‐ pared with those of keystroke timings. Using the integrated biometrics of key stroke timing, pen speed and pen pressure, the fragmentations decreased compared with that of keystroke timings without affected by pen data.

The Authentication experiments are conducted using the map organized by SOM, P-SOM and SP-SOM for the comparisons. 4 of the 6 samples of each user are used to organize the map and 2 remainders are used for the test data. All of the combinations of training data and test data are examined. 6C2 x 10x2 =300 input vectors are tested.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics http://dx.doi.org/10.5772/52100 131

> p r e

5 **Figure 10.** Organized map of SP-SOM with behavior biometrics data

1

2

4

6

*xi <sup>s</sup>* =(*xi*' *ci* )

**x**t

where

*ci* ={ <sup>1</sup> *xi*' <sup>∈</sup> *Ci* 0 *xi* ∉ *Ci*

arg max*<sup>k</sup>* { ∑ *U ij* ∈*P*(*xt*) *ck ij* }

authentication.

where P(*x*<sup>t</sup>

is determined by the following equation.

130 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

which lack some attributes in the vectors.

timings without affected by pen data.

) is the Pareto optimal set of units for **x**<sup>t</sup>

With introducing category element, it attracts the input vectors in the same category closely on the map corporately with other input vectors in the learning phase. In this meaning, P-SOM learning algorithm becomes supervised. In the recalling process, category of test vector

Considering the Further extension of Pareto learning SOM, anything which has its own met‐ rics can be element of multi-modal input vector for P-SOM. Structured data and vectors can be integrated as input data. For example, in bioinformatics amino acid sequence data com‐ prised of Hidden Markov Model(HMM) of the data, HMM of the structures and other fea‐ tures can be integrated. Furthermore, the map can be organized using the partial vectors

The experimental results of the authentication system using handwritten patterns and key‐ stroke timings were shown in the previous section. Another experiments using both of handwritten patterns and keystroke timings were conducted [9] [10]. The experimental set‐ tings are as follows. 6 samples are taken from each of 11 examinees. Each sample is com‐ prised of the keystroke timings and pen calligraphy data which is inputted alternately. For these experiments, tablet PC, which is equipped with keyboard and touch panel, is used.

The phrase "kirakira" which marked the best results in the previous experiments is used. The symbol Spiral is used as the handwritten symbol. SP-SOM is used for the analysis and

Figure 10 shows the map of SP-SOM using behavior biometrics data, key stroke timing, pen speed and pen pressure. Using keystroke timing, the users are clustered well except small fragmentations. Using pen speed and pen pressure, thefragmentations are increased com‐ pared with those of keystroke timings. Using the integrated biometrics of key stroke timing, pen speed and pen pressure, the fragmentations decreased compared with that of keystroke

The Authentication experiments are conducted using the map organized by SOM, P-SOM and SP-SOM for the comparisons. 4 of the 6 samples of each user are used to organize the map and 2 remainders are used for the test data. All of the combinations of training data

and test data are examined. 6C2 x 10x2 =300 input vectors are tested.

**5.3. Experiment of authentication system using multi-modal behavior biometrics**


**Table 3.** Result of Authentication experiment using each of behavior biometrics

Table 3 shows the result of authentication using each data independently. In this experi‐ ment, FRR of keystroke timings is much better than that of the previous experiment because the typing skill of each user becomes much better. FRR of pen speed data and pen pressure data are much worse than that of keystroke timing.

Table 4 shows the results of authentication experiments using integrated vectors of key‐ stroke timing, pen speed and pen pressure. The weight values for pen speed data and pen pressure data are studiously selected from many iterations of try and error for SOM. SP-SOM can achieve almost same authentication performance without tunings of parameters.


Figure 11 shows the FRR and FAR of all user and their averages. In this experiments, aver‐ age of FRR of keystroke timing is about 0.2. With integrating the typing sound, average of FRR is improved as about 0.1. For almost users, FRR and FAR is improved by integration.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

133

As to adapt the changes of biometrics, incremental learning is introduced [15] [13]. At first, incremental learning of SP-SOM is examined using the test data. Incremental leaning is per‐ formed by updating the map after recalling process using the updating method of conven‐

where *m'ij* is the vector assigned to and *P* isPareto winner set for input vector *x*. Two types of incremental learning, which are supervised learning using the input vector with category

At first, test data is incrementally learned during the iterations of authentication. Figure 12 shows the result. Both of the FRR and FAR are improved by repeating incremental learning. Changes of behavior biometrics by time, variation of the behavior biometrics and noise af‐ fect the accuracy of authentication. The authentication system will be able to adapt changes of behavior characteristics by the time with incremental learning of the input data for au‐ thentication. However, Incremental learning of the input data with variations or noises may

Next, we conducted experiments for changes of biometrics by the time of authentication in‐ put. It will take very long time to obtain the data which changes by the time from examin‐ ees. So, simulated data is made with changing the observed data at each authentication gradually. The experimental settings are as follows. Before authentication experiments, all data are learned by SP-SOM. For each authentication, 4 elements of the keystroke timing vector are selected randomly and multiplied by 0.8 and replaced with the new values. This case is very extreme case of changing the biometric input by the time. The results of super‐ vised learning with used-id and unsupervised learning without user-id are examined.

element and unsupervised learning without category element, are examined.

**Figure 12.** Result of iterated authentication with incremental learning

affect the authentication system.

tional SOM by the following equation.

*m*′*ij* =*m*′*ij* + *η* ′ (*x* ′ −*m*′*ij* )

**Table 4.** Result of authentication experiment using integrated biometrics

## **6. Adaptive authentication system using integrated biometrics of keystroke timing and key typing sound**

In this section, the experiments on the Integration of Keystroke timings and Key Typing Sound, and adaptive authentication system is mentioned [11]. Key typing sound is used as the intensities of key typing. 10 samples are taken from each of 10 examinee. The phrase "kirakira" is used for sampling keystroke timings and typing sounds. Sampling rate of the sound is 44Khz, and the maximum amplitude for each key is used as a feature vector.

The setting of the authentication experiments is as follows. 5 of the 10 samples of each user are used to organize the map and 5 remainders are used for the test data. All of the combi‐ nations of training data and test data are examined. The results of the authentication using keystroke timing, using key typing sounds and using the integrated vector are examined.

**Figure 11.** Result of authentication experiments using keystroke timing and key typing sound

Figure 11 shows the FRR and FAR of all user and their averages. In this experiments, aver‐ age of FRR of keystroke timing is about 0.2. With integrating the typing sound, average of FRR is improved as about 0.1. For almost users, FRR and FAR is improved by integration.

As to adapt the changes of biometrics, incremental learning is introduced [15] [13]. At first, incremental learning of SP-SOM is examined using the test data. Incremental leaning is per‐ formed by updating the map after recalling process using the updating method of conven‐ tional SOM by the following equation.

*m*′*ij* =*m*′*ij* + *η* ′ (*x* ′ −*m*′*ij* )

**Table 4.** Result of authentication experiment using integrated biometrics

**keystroke timing and key typing sound**

132 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

**6. Adaptive authentication system using integrated biometrics of**

In this section, the experiments on the Integration of Keystroke timings and Key Typing Sound, and adaptive authentication system is mentioned [11]. Key typing sound is used as the intensities of key typing. 10 samples are taken from each of 10 examinee. The phrase "kirakira" is used for sampling keystroke timings and typing sounds. Sampling rate of the sound is 44Khz, and the maximum amplitude for each key is used as a feature vector.

The setting of the authentication experiments is as follows. 5 of the 10 samples of each user are used to organize the map and 5 remainders are used for the test data. All of the combi‐ nations of training data and test data are examined. The results of the authentication using keystroke timing, using key typing sounds and using the integrated vector are examined.

**Figure 11.** Result of authentication experiments using keystroke timing and key typing sound

where *m'ij* is the vector assigned to and *P* isPareto winner set for input vector *x*. Two types of incremental learning, which are supervised learning using the input vector with category element and unsupervised learning without category element, are examined.

**Figure 12.** Result of iterated authentication with incremental learning

At first, test data is incrementally learned during the iterations of authentication. Figure 12 shows the result. Both of the FRR and FAR are improved by repeating incremental learning.

Changes of behavior biometrics by time, variation of the behavior biometrics and noise af‐ fect the accuracy of authentication. The authentication system will be able to adapt changes of behavior characteristics by the time with incremental learning of the input data for au‐ thentication. However, Incremental learning of the input data with variations or noises may affect the authentication system.

Next, we conducted experiments for changes of biometrics by the time of authentication in‐ put. It will take very long time to obtain the data which changes by the time from examin‐ ees. So, simulated data is made with changing the observed data at each authentication gradually. The experimental settings are as follows. Before authentication experiments, all data are learned by SP-SOM. For each authentication, 4 elements of the keystroke timing vector are selected randomly and multiplied by 0.8 and replaced with the new values. This case is very extreme case of changing the biometric input by the time. The results of super‐ vised learning with used-id and unsupervised learning without user-id are examined.

The result for unsupervised incremental learning becomes worse by iterations. In contrast, supervised incremental learning is not affected by noises rather it becomes better than with‐

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

135

Next we conducted the experiment with changes of keystroke timings by time and adding

FRR for without incremental learning and unsupervised incremental learning becomes worse with iterations. FRR for supervised incremental learning is kept less than 0.1. In spite

As the authentication system, unregistered user must be detected. Simple SP-SOM algo‐ rithm can classify the input vector to one of learned category, and cannot detect unlearned vector. For this problem, threshold values are introduced. As the features of SP-SOM, the size of Pareto set becomes large for unregistered user(unlearned data), and the magnitude of the category value becomes small for unregistered data. Thus, unregistered users can be

Figure 16 shows the result of authentication experiment with setting the threshold for size of Pareto set and category value as 10 and 0.5 respectively. Both of the FRR and FAR of regis‐

Figure 17 shows the result with adding changes by the time and noises. In this experiment, the authentication system can also adapt to the changes by time with incremental learning.

**Figure 15.** Result of authentication experiment with changing keystroke timing by time and adding noises to key typ‐

of the noises, supervised incremental learning can adapt to changes by time.

tered users are remarkably small, and FAR of unregistered user is also small.

out incremental learning.

ing sounds (x-axis: iterations, y-axis: FRR)

the noises to key typing sounds simultaneously.

identified with setting the threshold values to these values.

**Figure 13.** Result of authentication experiment of adaptive authentication system with changing the keystroke timing by time

Figure 13 shows the result. Average of the FRR becomes worse with iterations without in‐ cremental learning because the input for biometrics changing. Average of FRR is kept al‐ most 0 with supervised incremental learning. Average of FRR becomes about 0.1 with unsupervised incremental learning. From this result, supervised incremental learning can adapt the changes by time almost perfectly.

Next, we conducted experiments for examining robustness to the noises and variations. In‐ cremental learning may affect worse using the data with noise and variations because the noisy data is added to the map. The authentication data with noises is made artificially by adding the noise to the input vector. The experimental settings are as follows. For each au‐ thentication, 5 elements of the typing sound vector are selected, and 50% random noise is added at each authentication.

**Figure 14.** Result of authentication experiment with adding the noises to key typing sounds

The result for unsupervised incremental learning becomes worse by iterations. In contrast, supervised incremental learning is not affected by noises rather it becomes better than with‐ out incremental learning.

Next we conducted the experiment with changes of keystroke timings by time and adding the noises to key typing sounds simultaneously.

FRR for without incremental learning and unsupervised incremental learning becomes worse with iterations. FRR for supervised incremental learning is kept less than 0.1. In spite of the noises, supervised incremental learning can adapt to changes by time.

As the authentication system, unregistered user must be detected. Simple SP-SOM algo‐ rithm can classify the input vector to one of learned category, and cannot detect unlearned vector. For this problem, threshold values are introduced. As the features of SP-SOM, the size of Pareto set becomes large for unregistered user(unlearned data), and the magnitude of the category value becomes small for unregistered data. Thus, unregistered users can be identified with setting the threshold values to these values.

**Figure 13.** Result of authentication experiment of adaptive authentication system with changing the keystroke timing

Figure 13 shows the result. Average of the FRR becomes worse with iterations without in‐ cremental learning because the input for biometrics changing. Average of FRR is kept al‐ most 0 with supervised incremental learning. Average of FRR becomes about 0.1 with unsupervised incremental learning. From this result, supervised incremental learning can

Next, we conducted experiments for examining robustness to the noises and variations. In‐ cremental learning may affect worse using the data with noise and variations because the noisy data is added to the map. The authentication data with noises is made artificially by adding the noise to the input vector. The experimental settings are as follows. For each au‐ thentication, 5 elements of the typing sound vector are selected, and 50% random noise is

**Figure 14.** Result of authentication experiment with adding the noises to key typing sounds

by time

adapt the changes by time almost perfectly.

134 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

added at each authentication.

Figure 16 shows the result of authentication experiment with setting the threshold for size of Pareto set and category value as 10 and 0.5 respectively. Both of the FRR and FAR of regis‐ tered users are remarkably small, and FAR of unregistered user is also small.

Figure 17 shows the result with adding changes by the time and noises. In this experiment, the authentication system can also adapt to the changes by time with incremental learning.

**Figure 15.** Result of authentication experiment with changing keystroke timing by time and adding noises to key typ‐ ing sounds (x-axis: iterations, y-axis: FRR)

1

2 3

**Figure 18.** Concurrent P-SOM

**Figure 19.** Experimental result of CP-SOM

Conventional concurrent SOM [16] uses threshold value of quantization error for classifying the learned data and unlearned data. Concurrent P-SOM uses the size of Pareto Optimal Set for detecting unregistered user.The threshold value of the size is set to TH\_T\*P\_LAST,

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

137

The authentication experiment is conducted using the keystroke timing and key typing sound data used in the previous section. Each user data is learned as the registered user on the distinct map, and the data of other users is used as the test data of unregistered users. The experiments are conducted with changing map sizes and threshold values, TH\_P. The size of Pareto sets are adaptively tuned as 1/4 and 1/2 of all units in initial learning step and last learning step respectively. The difference between registered user and unregistered

where P\_LAST is the average size of Pareto optimal set in the last phase of learning.

users becomes larger using larger size of Pareto set in the last step.

**Figure 16.** Result of authentication experiment with rejecting unregistered user

**Figure 17.** Result of authentication experiment with adding changes by time and noises using the threshold for reject‐ ing unregistered user (x-axis: iterations, y-axis: FRR and FAR)

## **7. Concurrent paretolearning SOM and its application to the authentication system using multi-modal behavior biometrics**

As mentioned in the previous section, we have proposed the authentication system using Supervised Pareto learning SOM(SP-SOM). However, SP-SOM needs the training data of multiple users for learning. For the device of single user, the authentication system, which requires the data of single user, is recommended. For this problem, Concurrent Pareto learning SOM (CP-SOM) is introduced [14]. CP-SOM is P-SOM which uses the small map for each user.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics http://dx.doi.org/10.5772/52100 137

**Figure 18.** Concurrent P-SOM

**Figure 16.** Result of authentication experiment with rejecting unregistered user

136 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

ing unregistered user (x-axis: iterations, y-axis: FRR and FAR)

small map for each user.

**Figure 17.** Result of authentication experiment with adding changes by time and noises using the threshold for reject‐

As mentioned in the previous section, we have proposed the authentication system using Supervised Pareto learning SOM(SP-SOM). However, SP-SOM needs the training data of multiple users for learning. For the device of single user, the authentication system, which requires the data of single user, is recommended. For this problem, Concurrent Pareto learning SOM (CP-SOM) is introduced [14]. CP-SOM is P-SOM which uses the

**7. Concurrent paretolearning SOM and its application to the authentication system using multi-modal behavior biometrics**

1

2 3 Conventional concurrent SOM [16] uses threshold value of quantization error for classifying the learned data and unlearned data. Concurrent P-SOM uses the size of Pareto Optimal Set for detecting unregistered user.The threshold value of the size is set to TH\_T\*P\_LAST, where P\_LAST is the average size of Pareto optimal set in the last phase of learning.

The authentication experiment is conducted using the keystroke timing and key typing sound data used in the previous section. Each user data is learned as the registered user on the distinct map, and the data of other users is used as the test data of unregistered users. The experiments are conducted with changing map sizes and threshold values, TH\_P. The size of Pareto sets are adaptively tuned as 1/4 and 1/2 of all units in initial learning step and last learning step respectively. The difference between registered user and unregistered users becomes larger using larger size of Pareto set in the last step.

**Figure 19.** Experimental result of CP-SOM

**Figure 20.** Experimental result of CFP-SOM

Figure 19 shows the experimental result of CP-SOM using the multi-modal vector composed of the 2 vectors, which are key stroke timing vector and key typing sound vector, with changing the threshold THp. The size of the map is 6x6. For typical tuning of authentication system, the threshold value is set to cross point of FRR and FAR. In this experiment, THp=1.2 and FRR=FAR=0.35 at the cross point, and this result is not adequate for authenti‐ cation system. It is because of the small difference of the size of Pareto set between regis‐ tered user and unregistered user, which are 6.22 and 7.77 respectively.

**Figure 21.** FRR and FAR of each user for experimental result of CFP-SOM

**Figure 22.** Experimental result with changing map size

ing behavior biometrics is mentioned.

In this chapter, application of Self Organizing Map(SOM)s to the authentication system us‐

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

139

In section 3 and section 4, the application of SOM to the authentication systems using pen calligraphy and keystroke timing are mentioned respectively. SOM is used to visualize the

**8. Conclusions**

To enlarge the difference of the size of Pareto set, the number of integrated vectors should be increased. For this purpose, Full Pareto learning SOM(FP-SOM) is applied. FP-SOM is P-SOM which uses each element in the vector as independent 1-dimensional vector. Concur‐ rent Full Pareto learning SOM(CFP-SOM) using multi-modal vector composed of 23 elements(15 keystroke timing and 8 typing sound) is applied to authentication system. Fig‐ ure 20 shows the result. At the cross point, FRR=FAR=0.121, and the average size of the Par‐ eto set for registered user and unregistered user are 17.95 and 27.95 respectively. FRR and FAR are much improved compared with the previous experiment because the difference of the size of Pareto set becomes larger.

Figure 20 shows the FRR and FAR of each user for experimental result of CFP-SOM. With checking each user, FARs of user 9 and user 6 are too large. Such users are called as "SHEEP" and are not adequate for this authentication method. Excluding user 6 and 9, aver‐ age of FRR=0.112, FAR=0.070, and they are better than that of SP-SOM(FRR=0.187, FAR=0.065) which are conducted in the same setting of the experiment using CFP-SOM.

Figure 21 shows the experimental result with changing map size. TH\_p is fixed to 1.4. With changing map size, FAR is not improved, however FRR is improved by enlarging map size. For map size 9x9, average of FAR=0.095, FAR=0.123, and FRR =0.090m FAR=0.061 excluding user 6 and 9.

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics http://dx.doi.org/10.5772/52100 139

**Figure 21.** FRR and FAR of each user for experimental result of CFP-SOM

**Figure 22.** Experimental result with changing map size

#### **8. Conclusions**

**Figure 20.** Experimental result of CFP-SOM

138 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

the size of Pareto set becomes larger.

user 6 and 9.

Figure 19 shows the experimental result of CP-SOM using the multi-modal vector composed of the 2 vectors, which are key stroke timing vector and key typing sound vector, with changing the threshold THp. The size of the map is 6x6. For typical tuning of authentication system, the threshold value is set to cross point of FRR and FAR. In this experiment, THp=1.2 and FRR=FAR=0.35 at the cross point, and this result is not adequate for authenti‐ cation system. It is because of the small difference of the size of Pareto set between regis‐

To enlarge the difference of the size of Pareto set, the number of integrated vectors should be increased. For this purpose, Full Pareto learning SOM(FP-SOM) is applied. FP-SOM is P-SOM which uses each element in the vector as independent 1-dimensional vector. Concur‐ rent Full Pareto learning SOM(CFP-SOM) using multi-modal vector composed of 23 elements(15 keystroke timing and 8 typing sound) is applied to authentication system. Fig‐ ure 20 shows the result. At the cross point, FRR=FAR=0.121, and the average size of the Par‐ eto set for registered user and unregistered user are 17.95 and 27.95 respectively. FRR and FAR are much improved compared with the previous experiment because the difference of

Figure 20 shows the FRR and FAR of each user for experimental result of CFP-SOM. With checking each user, FARs of user 9 and user 6 are too large. Such users are called as "SHEEP" and are not adequate for this authentication method. Excluding user 6 and 9, aver‐ age of FRR=0.112, FAR=0.070, and they are better than that of SP-SOM(FRR=0.187, FAR=0.065) which are conducted in the same setting of the experiment using CFP-SOM.

Figure 21 shows the experimental result with changing map size. TH\_p is fixed to 1.4. With changing map size, FAR is not improved, however FRR is improved by enlarging map size. For map size 9x9, average of FAR=0.095, FAR=0.123, and FRR =0.090m FAR=0.061 excluding

tered user and unregistered user, which are 6.22 and 7.77 respectively.

In this chapter, application of Self Organizing Map(SOM)s to the authentication system us‐ ing behavior biometrics is mentioned.

In section 3 and section 4, the application of SOM to the authentication systems using pen calligraphy and keystroke timing are mentioned respectively. SOM is used to visualize the relations among the biometrics data of users, and is used to select the appropriate feature, and to select the appropriate patterns of behavior for authentication. For the authentication system using pen calligraphy, the pen speed data and pen pressure data is selected as the features, and drawing spiral or star on the screen are selected as the patterns of behavior. For the authentication system using keystroke timing, keystroke timing data typing "kira‐ kira" is selected as the behavior. These systems show superior performance considering the simplicity of these methods. However, using single behavior biometrics, enough accuracy is hard to be accomplished.

**References**

2004

400-3

39-44

413-420

[1] Bolle R., Connell J., Pankanti S., Ratha N., Senior A.Guide to Biometrics, Springer;

Application of Self Organizing Maps to Multi Modal Adaptive Authentication System Using Behavior Biometrics

http://dx.doi.org/10.5772/52100

141

[2] Monrose F., Rubin A.D., Keystroke Dynamics as a Biometric for Authentication. Fu‐

[3] Brault J.J., Plamondon R. A Complexity Measure of Handwritten Curves: Modelling of Dynamic Signature Forgery. IEEE Trans. Systems, Man and Cybernetics. 1993; 23

[5] Dozono H., Nakakuni M., et al. The Analysis of Pen Pressures of Handwritten Sym‐ bols on PDA Touch Panel using Self Organizing Maps. Proceedings of the Interna‐

[6] Dozono H., Nakakuni M., et al. The Analysis of pen Inputs of Handwritten Symbols using Self Organizing Maps and its Application to User Authentication. Proceedings

[7] Dozono H., Nakakuni M., et al. The Analysis of Key Stroke Timings using Self Or‐ ganizing Maps and its Application to Authentication. Proceedings of the Internation‐

[8] Dokic S., Kulesh A., et al. An Overview of Multi-modal Biometrics for Authentica‐ tion. Proceedings of the International Conference on Security and Management; 2007:

[9] Nakakuni M., Dozono H., et al. Application of Self Organizing Maps for the Integrat‐ ed Authentication using Keystroke Timings and Handwritten Symbols. WSEAS TRANSACTIONS on INFORMATION SCIENCE & APPLICATIONS. 2006; 2-4

[10] Dozono H., Nakakuni M., et al. Application of Self Organizing Maps to User Authen‐ tication Using Combination of Key Stroke Timings and Pen Calligraphy. Proceedings of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE; 2006: 105-10

[11] Dozono H., Nakakuni M., et al. Application of the Supervised Pareto Learning Self Organizing Maps to Multi-modal Biometric Authentication. Journal of Information

[12] Dozono H., Nakakuni M., et al. Comparison of the Adaptive Authentication Systems

[13] Dozono H., Nakakuni M., et al. The Adaptive Authentication System for Behavior Biometrics Using Pareto Learning Self Organizing Maps. Neural Information Proc‐ essing Models and Applications ICONIP 2010. Springer; 2010 LNCS6444 383-90

for Behavior Biometrics using the Variations of Self Organizing Maps

of 2006 International Joint Conference on Neural Networks: 2006; 4884-9.

ture Generation Computer Systems: March;2000

[4] Kohonen T. Self Organizing Maps, Springer;ISBN 3-540-67921-9

tional Conference on Security and Management; 2005: 440-5.

al Conference on Security and Management; 2006: 100-5

Processing Society of Japan. 2008 49(9) 3028-37

In section 5, the application of SOM to the authentication system using multi-modal behav‐ ior biometrics is mentioned. As the multi-modal behavior biometrics, pen calligraphy draw‐ ing the symbols and keystroke timing which are measured on tablet PC are used. Supervised Pareto learning SOM(SP-SOM), which can integrate multi-modal vector naively with supervised learning, is proposed, and applied to the analysis of multi-modal biometric features and to the authentication system. The accuracy of authentication is improved com‐ pared with that of single behavior biometrics without disturbance from unreliable features.

In section 6, the application of SP-SOM to the authentication system using multi-modal behav‐ ior biometrics of keystroke timing and key typing sound is mentioned. These features can be measured simultaneously during typing a phrase. Additionally, the adaptive authentication system, which can follow the changes of biometrics by the time, is examined. The authentica‐ tion system shows superior accuracy with following changes of keystroke timing by the time without disturbed by noises. As the practical authentication system, the unregistered users are detected using the size of the Pareto set and category magnitude of the user inputs.

In section 7, Concurrent Full P-SOM(CFP-SOM), which uses small map for each user, is ap‐ plied to the authentication system using multi-modal behavior biometrics. CFP-SOM is suit‐ able for mobile system because it only needs the learning data of single user, in contrast, SP-SOM needs the learning data of some users. CFP-SOM classifies the registered user from unregistered users using the size of Pareto set. With adjusting the threshold of size of Pareto set, the accuracy becomes almost same or better compared with that of SP-SOM.

As the future work, the adaptive algorithm for CFP-SOM should be developed. CFP-SOM needs the adjustment of threshold to apply to authentication system, thus adaptive scheme for the threshold is required. Furthermore, P-SOM and SP-SOM is a generic extension of SOM, and it can be applied to many applications, which need both of visualization and clas‐ sification by supervised learning. P-SOM and SP-SOM can be used to integrate the multiple objects for which distance metric is defined. The novel application from this point of view should be explored.

## **Author details**

#### Hiroshi Dozono

Faculty of Science and Engineering Saga University, Honjyo Saga, Japan

## **References**

relations among the biometrics data of users, and is used to select the appropriate feature, and to select the appropriate patterns of behavior for authentication. For the authentication system using pen calligraphy, the pen speed data and pen pressure data is selected as the features, and drawing spiral or star on the screen are selected as the patterns of behavior. For the authentication system using keystroke timing, keystroke timing data typing "kira‐ kira" is selected as the behavior. These systems show superior performance considering the simplicity of these methods. However, using single behavior biometrics, enough accuracy is

In section 5, the application of SOM to the authentication system using multi-modal behav‐ ior biometrics is mentioned. As the multi-modal behavior biometrics, pen calligraphy draw‐ ing the symbols and keystroke timing which are measured on tablet PC are used. Supervised Pareto learning SOM(SP-SOM), which can integrate multi-modal vector naively with supervised learning, is proposed, and applied to the analysis of multi-modal biometric features and to the authentication system. The accuracy of authentication is improved com‐ pared with that of single behavior biometrics without disturbance from unreliable features. In section 6, the application of SP-SOM to the authentication system using multi-modal behav‐ ior biometrics of keystroke timing and key typing sound is mentioned. These features can be measured simultaneously during typing a phrase. Additionally, the adaptive authentication system, which can follow the changes of biometrics by the time, is examined. The authentica‐ tion system shows superior accuracy with following changes of keystroke timing by the time without disturbed by noises. As the practical authentication system, the unregistered users are

detected using the size of the Pareto set and category magnitude of the user inputs.

set, the accuracy becomes almost same or better compared with that of SP-SOM.

Faculty of Science and Engineering Saga University, Honjyo Saga, Japan

In section 7, Concurrent Full P-SOM(CFP-SOM), which uses small map for each user, is ap‐ plied to the authentication system using multi-modal behavior biometrics. CFP-SOM is suit‐ able for mobile system because it only needs the learning data of single user, in contrast, SP-SOM needs the learning data of some users. CFP-SOM classifies the registered user from unregistered users using the size of Pareto set. With adjusting the threshold of size of Pareto

As the future work, the adaptive algorithm for CFP-SOM should be developed. CFP-SOM needs the adjustment of threshold to apply to authentication system, thus adaptive scheme for the threshold is required. Furthermore, P-SOM and SP-SOM is a generic extension of SOM, and it can be applied to many applications, which need both of visualization and clas‐ sification by supervised learning. P-SOM and SP-SOM can be used to integrate the multiple objects for which distance metric is defined. The novel application from this point of view

hard to be accomplished.

140 Developments and Applications of Self-Organizing Maps Applications of Self-Organizing Maps

should be explored.

**Author details**

Hiroshi Dozono


[14] Dozono H., Nakakuni M., et al. The Authentication System for Multi-modal Behavior Biometrics Using Concurrent Pareto Learning SOM, Artificial Neural Networks and Machine Learning- ICANN 2011. Springer; 2011 LNCS6792 197-204

**Chapter 7**

**Quantification of Emotions for Facial Expression:**

Masaki Ishii, Toshio Shimodate, Yoichi Kageyama,

Tsuyoshi Takahashi and Makoto Nishida

http://dx.doi.org/10.5772/51136

**1. Introduction**

Additional information is available at the end of the chapter

and machines has been investigated in recent studies [1-7].

**Mapping**

**Generation of Emotional Feature Space Using Self-**

Facial expression recognition for the purpose of emotional communication between humans

The shape (static diversity) and motion (dynamic diversity) of facial components, such as the eyebrows, eyes, nose, and mouth, manifest expression. From the viewpoint of static di‐ versity, owing to the individual variation in facial configurations, it is presumed that a facial expression pattern due to the manifestation of a facial expression includes subject-specific features. In addition, from the viewpoint of dynamic diversity, because the dynamic changes in facial expressions originate from subject-specific facial expression patterns, it is presumed that the displacement vector of facial components has subject-specific features.

On the other hand, although an emotionally generated facial expression pattern of an indi‐ vidual is unique, internal emotions expressed and recognized by humans via facial expres‐ sions are considered person-independent and universal. For example, one person may express the common emotion of happiness using various facial expressions, while another person may recognize happiness from these facial expressions. Pantic et al. argued that a natural facial expression always includes various emotions, and that a pure facial expression rare‐ ly appears [1]. Furthermore, they suggested that it is not realistic to classify all facial expres‐ sions into the six basic emotion categories: anger, sadness, disgust, happiness, surprise and fear. Instead, they proposed quantitative classification into many more emotion categories.

> © 2012 Ishii et al.; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

© 2012 Ishii et al.; licensee InTech. This is a paper distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

distribution, and reproduction in any medium, provided the original work is properly cited.

