**4. Experiment**

### **4.1 Dataset preparation**

In the dataset, the total number of images of rejuvenation treatment and eye double-fold surgery are 5585 and 36,598 pairs of the images, respectively. The data is filtered to select a good image for training. Some errors in the images are not well aligned, only one eye, etc. The reject rates are 82.15 and 59.92%, respectively.

The image was crop around the eye area. Then the data is divided into 70, 15, and 15% for training, validation, and testing, respectively.

As mentioned earlier, the output of the model is the probability of perfection. Hence, our training data include the image of before and after surgery as shown in **Figure 3**.

#### **4.2 Prediction accuracy**

We chose a thresholding value for the prediction model. If the predicted value is higher than the thresholding value, the face is predicted to be perfect. From that, we obtain accuracy of the different models as shown in **Tables 2** and **3**. We have two schemes of training in which the feature extractors are freezed (not training together with the classification model) and trainable, resulting in eight models in these two tables. When the feature extractor is freezed, we believe that it has already captured universal features such as edge and curves which is relevant to our tasks. Therefore, we want to keep the weights of the feature extract intact. However, in the second training scheme, we apply different learning rates for the feature extractor and the fully connected layers. The learning rate of the feature extractor is much smaller than that of the fully connected layer because we believe that the weights of the feature extractor is good enough for our tasks and we do not want to distort them too quickly and too much during the training of the classification model.

These above training schemes are the best common practices when finetuning deep neural networks. We tried both of them, resulting in the following. The best model for double-fold surgery and rejuvenation treatment are Models 1 and 8 (see **Tables 2** and **3** for more details), with accuracy on the test dataset of 93.1 and 88.9%, respectively. Model 1 is the model when the encoder

**49**

than 1%.

**Table 3.**

**Table 2.**

**Figure 3.**

*Testing accuracy for eye double-fold surgery.*

*Testing accuracy for rejuvenation treatment.*

**5. Conclusion**

*A Deep Learning-Based Aesthetic Surgery Recommendation System*

was freezed. However, in Model 8, the encoder was retrained. However, the accuracy differences between the best model and the second best model are less

We have presented in this chapter an interesting application of deep learning in aesthetic surgery recommendation along with its encouraging results. By using our system, the presented deep learning engine will provide a reference decision of taking either rejuvenation treatment or eye double-fold surgery or not to both the surgeon and the patient, just based on the eye photo of the patient. To this end, we trained a deep autoencoder by our in-house dataset, composing of pairs of images captured before and after the surgery. The trained encoder part learned in an unsupervised manner, a rich set of features, characterized both unattractive and beautiful facial features. We concatenate the trained encoder part to a fully connected layer to predict perfection score of an eye photo of a patient, based on which

a decision of taking treatment or not will be made.

*DOI: http://dx.doi.org/10.5772/intechopen.86411*

*Example of original and outcome of surgery.*

*A Deep Learning-Based Aesthetic Surgery Recommendation System DOI: http://dx.doi.org/10.5772/intechopen.86411*

#### **Figure 3.**

*Advanced Analytics and Artificial Intelligence Applications*

**48**

**4. Experiment**

**Table 1.**

**Figure 3**.

**4.1 Dataset preparation**

*(2.2) upsampling with transposed convolution.*

*Autoencoder model and classification model.*

**4.2 Prediction accuracy**

of the classification model.

In the dataset, the total number of images of rejuvenation treatment and eye double-fold surgery are 5585 and 36,598 pairs of the images, respectively. The data is filtered to select a good image for training. Some errors in the images are not well aligned, only one eye, etc. The reject rates are 82.15 and 59.92%, respectively. The image was crop around the eye area. Then the data is divided into 70, 15,

*conv: convolution. conv\*: convolution and max-pooling with filter size (2.2). deconv: transposed convolution. deconv\*:* 

As mentioned earlier, the output of the model is the probability of perfection. Hence, our training data include the image of before and after surgery as shown in

We chose a thresholding value for the prediction model. If the predicted value is higher than the thresholding value, the face is predicted to be perfect. From that, we obtain accuracy of the different models as shown in **Tables 2** and **3**. We have two schemes of training in which the feature extractors are freezed (not training together with the classification model) and trainable, resulting in eight models in these two tables. When the feature extractor is freezed, we believe that it has already captured universal features such as edge and curves which is relevant to our tasks. Therefore, we want to keep the weights of the feature extract intact. However, in the second training scheme, we apply different learning rates for the feature extractor and the fully connected layers. The learning rate of the feature extractor is much smaller than that of the fully connected layer because we believe that the weights of the feature extractor is good enough for our tasks and we do not want to distort them too quickly and too much during the training

These above training schemes are the best common practices when finetuning deep neural networks. We tried both of them, resulting in the following. The best model for double-fold surgery and rejuvenation treatment are Models 1 and 8 (see **Tables 2** and **3** for more details), with accuracy on the test dataset of 93.1 and 88.9%, respectively. Model 1 is the model when the encoder

and 15% for training, validation, and testing, respectively.

*Example of original and outcome of surgery.*


#### **Table 2.**

*Testing accuracy for eye double-fold surgery.*


#### **Table 3.**

*Testing accuracy for rejuvenation treatment.*

was freezed. However, in Model 8, the encoder was retrained. However, the accuracy differences between the best model and the second best model are less than 1%.
