8. Neural networks for discriminating drunk persons

Let u0ð Þ x; y be the original input image and utð Þ x; y the digital image at iteration t. The discreet in time implementation of (19) is carried out by employing the four nearest neighbors and the

> X 4

g ∇u<sup>i</sup>

<sup>t</sup>ð Þ <sup>x</sup>; <sup>y</sup> � �∙∇u<sup>i</sup>

<sup>4</sup> and

<sup>t</sup>ð Þ¼ x; y utð Þ� x; y þ 1 utð Þ x; y (22)

<sup>t</sup>ð Þ¼ x; y utð Þ� x; y � 1 utð Þ x; y (23)

<sup>t</sup>ð Þ¼ x; y utð Þ� x þ 1; y utð Þ x; y (24)

<sup>t</sup>ð Þ¼ x; y utð Þ� x � 1; y utð Þ x; y (25)

<sup>t</sup>ð Þ <sup>x</sup>; <sup>y</sup> � � (21)

i¼1

The nonlinear anisotropic diffusion method was applied to all 41 faces corresponding to sober and intoxicated persons. In order the diffusion to take place only along edges the value of k

If thresholding is used on images after diffusion and top-hat transformation, the image obtained is richer for the intoxicated person compared to that of the sober person. In our experiments, the threshold was chosen to be equal to 100. In Figure 7, two images obtained

Figure 7. Binary images obtained using a threshold equal to 100. Sober left and intoxicated right. Vessels on the drunk

Laplacian operator which was used in [28]:

158 Human-Robot Interaction - Theory and Application

is the gradient of south direction,

is the gradient of north direction,

is the gradient of east direction and

is the gradient of west direction.

u<sup>t</sup>þ<sup>1</sup>ð Þ¼ x; y utð Þþ x; y λ

∇u<sup>1</sup>

∇u<sup>2</sup>

∇u<sup>3</sup>

∇u<sup>4</sup>

which affects the degree of smoothing was selected equal to 20.

person are more distinct compared to those on the sober person.

where in the experimental procedure was used 0 ≤ λ ≤ <sup>1</sup>

Neural networks have been used as a classification tool in a variety of machine vision techniques such as face recognition [29] and thermal infrared pattern recognition [30–33]. Especially, a thermo vision application for biometric recognition is addressed in [32], while neural structures are employed in [33] for recognition of facial expressions using thermal maps of the face.

This method offers a way of discriminating sober from drunk persons, using thermal infrared images and neural networks. The neural networks are employed as a black box to discriminate intoxication by means of the values of simple pixels from the thermal images of the persons' face. In this work, the neural networks were used by means of two different approaches. According to the first approach, a different neural structure is used from location to location on the thermal image of the face and the convergence capabilities of the network are monitored. A successful convergence characterizes the corresponding location of the face as being a good candidate for intoxication identification. According to the second approach, a single neural structure is trained with data from the thermal images of the whole face of a person (sober and drunk) and its capability to operate with high classification success to other persons is tested. Its generalization performance is also accessed.

In the first approach, different networks are trained on different locations of the same face. Thus, there will be a serious indication on the suitability of the specific face locations for drunk identification. Consequently, the face of each person is partitioned into a matrix of squared regions of 10 10 pixels each as the one depicted in Figure 8. There is a complete correspondence between these locations on the images of sober and drunk persons. Figure 8 is illustrated one of these square regions of 100 pixels on a pair of infrared images (sober-drunk) of a specific person.

A simple neural network is trained using the data in the two black regions as shown in Figure 8. The vectors used as input to the neural structure are of nine elements obtained when a small 3 3 window moves all over each of the two 10 10 pixels regions. In this way, 200 vectors are obtained to train a three-level neural structure of [9 30 1] neurons, for these two specific regions of 10 10 pixels. Furthermore, a larger network of [49 49 1] neurons was employed using as input vectors of 49 elements. These elements were obtained when a

window of 7 7 pixels moves around each of the two 10 10 pixels regions. The back propagation algorithm was employed for training both neural structures.

same trained network was tested with the same data and resulted in satisfactory performance. When the output was closer to zero, the pixel was declared to belong to a sober person (black), otherwise (closer to one) it was declared to represent a drunk person (white). In Figure 11, the results of this experiment are presented. The achieved performance for the pixels of the sober image is 89.22%, while for the drunk image, the performance is 87.09%, with even higher performance at the regions of the forehead and nose. In conclusion, the training of a single neural network using information from the whole face can easily point out the regions which

Intoxication Identification Using Thermal Imaging http://dx.doi.org/10.5772/intechopen.72128 161

The case of a neural network trained with the data from the whole face of a specific person (sober and drunk) and tested using the data of the rest persons is discussed also. Accordingly, if the person is sober, his face should be black, while if he is drunk, his face should be white.

Figure 10. The whole black region drawn on the face of a specific person in the above thermal image was used for

Figure 11. Discrimination results obtained by a neural structure [49 49 1], trained with the same data from the whole region. The achieved performance for the pixels of the sober image (left in black) is 89.22% while for the drunk image (right in white) the performance is 87.09%, with even higher performance at the regions of the forehead and nose.

better support drunk discrimination.

training a single network.

Successful convergence of a neural network to a minimum value in a specific location means that this face location is suitable for drunk identification. For demonstration purposes, a region for which we have high convergence of the network is given in red in Figure 9. For all participants in the experiment and especially when the large neural structures were employed, high convergence was observed mainly on the forehead, the nose, and the mouth as depicted in Figure 9. Thus, these locations of the face of a person are the most suitable to be employed for intoxication discrimination.

In the next approach, the whole face of each specific person is examined as a single area of 5000 pixels (50 100), as shown in Figure 10. Our purpose is to be able to discriminate between the sober and the drunk image of a person using a specific neural structure which has been trained with information coming from the same person.

The big region of 5000 pixels gives 5000 different vectors of 49-elements each, as a window 7 7 is moving around the image of the sober person. Another 5000 vectors are obtained from the thermal image of the drunk person. A neural structure of 3 layers with 49 neurons in the first layer, 49 neurons in the hidden layer and 1 neuron in the output layer, that is, a [49 49 1] structure, was trained with the above data. Next, the recognition procedure was tried. The

Figure 8. The same square region of 100 pixels on a pair of infrared images (sober-drunk) of a specific person.

Figure 9. Two different persons correspond to the above colored matrices. The employed neural networks with structures [49 49 1] converge at the red areas giving to the areas of the forehead, the nose and the mouth desirable drunk discrimination capabilities.

same trained network was tested with the same data and resulted in satisfactory performance. When the output was closer to zero, the pixel was declared to belong to a sober person (black), otherwise (closer to one) it was declared to represent a drunk person (white). In Figure 11, the results of this experiment are presented. The achieved performance for the pixels of the sober image is 89.22%, while for the drunk image, the performance is 87.09%, with even higher performance at the regions of the forehead and nose. In conclusion, the training of a single neural network using information from the whole face can easily point out the regions which better support drunk discrimination.

window of 7 7 pixels moves around each of the two 10 10 pixels regions. The back

Successful convergence of a neural network to a minimum value in a specific location means that this face location is suitable for drunk identification. For demonstration purposes, a region for which we have high convergence of the network is given in red in Figure 9. For all participants in the experiment and especially when the large neural structures were employed, high convergence was observed mainly on the forehead, the nose, and the mouth as depicted in Figure 9. Thus, these locations of the face of a person are the most suitable to be employed

In the next approach, the whole face of each specific person is examined as a single area of 5000 pixels (50 100), as shown in Figure 10. Our purpose is to be able to discriminate between the sober and the drunk image of a person using a specific neural structure which has been trained

The big region of 5000 pixels gives 5000 different vectors of 49-elements each, as a window 7 7 is moving around the image of the sober person. Another 5000 vectors are obtained from the thermal image of the drunk person. A neural structure of 3 layers with 49 neurons in the first layer, 49 neurons in the hidden layer and 1 neuron in the output layer, that is, a [49 49 1] structure, was trained with the above data. Next, the recognition procedure was tried. The

Figure 9. Two different persons correspond to the above colored matrices. The employed neural networks with structures [49 49 1] converge at the red areas giving to the areas of the forehead, the nose and the mouth desirable drunk

Figure 8. The same square region of 100 pixels on a pair of infrared images (sober-drunk) of a specific person.

propagation algorithm was employed for training both neural structures.

for intoxication discrimination.

160 Human-Robot Interaction - Theory and Application

discrimination capabilities.

with information coming from the same person.

The case of a neural network trained with the data from the whole face of a specific person (sober and drunk) and tested using the data of the rest persons is discussed also. Accordingly, if the person is sober, his face should be black, while if he is drunk, his face should be white.

Figure 10. The whole black region drawn on the face of a specific person in the above thermal image was used for training a single network.

Figure 11. Discrimination results obtained by a neural structure [49 49 1], trained with the same data from the whole region. The achieved performance for the pixels of the sober image (left in black) is 89.22% while for the drunk image (right in white) the performance is 87.09%, with even higher performance at the regions of the forehead and nose.

Relative results are demonstrated in Figure 12. A neural structure of [9 30 1] was trained using data from person 1 and tested using the data from person 2. At the left of Figure 12 the face should be black recognized as belonging to the sober person, while at the right it should be white since it corresponds to the drunk person. The depicted performance is satisfactory and an operator can discriminate the sober from the drunk easily. It is worth emphasizing that according to the images in Figure 12, the pixels on the forehead and the nose are correctly classified in almost all cases.

As a general conclusion of this final approach, we can say that we can decide with 90% confidence if a person is drunk or not using a small neural structure. No data records of the

Intoxication Identification Using Thermal Imaging http://dx.doi.org/10.5772/intechopen.72128 163

Temperature distribution on the eyes of sober and drunk persons is studied by means of thermal infrared images (Figure 14). It is observed that the temperature difference between the sclera and the iris is zero for the sober person and increases when somebody consumes alcohol (Figure 15). For the drunk person, iris appears darker compared to sclera which means that the sclera temperature increases. This is something expected since the sclera is full of blood vessels which present increased activity when the person consumes alcohol. Thus, in a screening procedure for drunk identification, the infrared images of the sober person are not needed. Although in most cases the sclera is brighter than the iris for the drunk persons, in case that their gray level difference is very small, histogram modification algorithms can be used to enhance this difference and show off intoxication. In order to express the confidence of the method in drunk person discrimination, the Student t-test was employed. The results gave

Specifically, for the 28 among the 41 people who participated in the experimental procedure, it is evident by a simple comparison of the thermal images that the sclera becomes hotter compared to the iris after alcohol consumption. The images in Figure 15, where the iris is darker than the sclera (right), have not undergone any kind of preprocessing. This difference between the sclera and the iris becomes evident for four more persons when a histogram equalization algorithm is applied. Initially, for all these sober persons the sclera and the iris appeared with the same gray level as being in the same temperature (Figure 15, left image). Finally, four more persons presented this difference when a histogram clipping algorithm was applied which clips all values below 0.5 and above 0.75 of the gray level range and after that stretches the remaining histogram to its full range (MATLAB imadjust ([0.5 0.75], [0 1])). For five persons who used to drink alcohol and their breathalyzer indication was below 0.4 mg/L,

Figure 14. Sclera is surrounding iris which is actually a muscle controlled part of the eye to adjust the size of the pupil.

it was not possible to show off the difference between the sclera and the iris.

inspected persons when they are sober are needed for comparison.

9. Temperature distribution on the eyes

over 99% confidence of the discrimination inference.

Sclera lies on a net of blood vessels.

Since the forehead is a very promising location on the face for intoxication identification, the above procedure was repeated only on the region of the forehead for all persons participated in the experiment. The area employed was shown in Figure 13. Accordingly, the neural structures are trained with the data from one person and tested with the data from the rest persons. The use of the forehead area for intoxication identification led to two significant conclusions: (a) The small neural structures have better identification performance since they achieve better generalization behavior during training. (b) Their success is on average 90%.

Figure 12. Results obtained when a neural structure of [9 30 1] was trained using data from person 1 and tested using the data from person 2. At the left the face should be black recognized as belonging to the sober person while at the right it should be white since it corresponds to the drunk person.

Figure 13. The black region on the forehead of the persons was employed to test the neural structures for identifying drunk persons.

As a general conclusion of this final approach, we can say that we can decide with 90% confidence if a person is drunk or not using a small neural structure. No data records of the inspected persons when they are sober are needed for comparison.
