**4.4. Results comparing different facial appearance features**

For the full recognition pipeline: The landmarks for the inner corner of the eyes and the tip of the nose are used as control points for a course registration. These points are the least effected by face morphology. An ϵ‐SVR is used for prediction of valence and arousal values [48].

Full regression results and a comparison to other state‐of‐the‐art facial appearance features are given in **Table 2**. Experiments employed a 9‐fold, leave‐one‐video‐out cross‐validation. For correlation, higher is better; for RMS lower is better. In **Table 2**, the correlation and RMS values for valence and arousal labels by the proposed method performed the best for valence and second best for arousal. Removal of background noise and then implementing LBP‐TOP provided better results. RMS values for the proposed method are also the best for arousal and second best for valence. The proposed method has the best average correlation and the lowest average RMS value. Graphs comparing the ground‐truth and predicted labels are given in **Figure 5**. It was found that frames with extreme head rotation tended to have lower correla‐ tion and higher error due to the difficulty of registering the dataset.


*Note*: The proposed method has better average correlation for valence and arousal. Bold indicates best performing feature.

**Table 2.** Correlation and RMS for prediction of valence and arousal emotion categories on the Motor Trend Magazine Best Driver's Car of the Year.

**Figure 5.** The predicted values are graphed with the values for valence and arousal.

### **5. Conclusions**

In this chapter, we proposed a system to perform facial expression recognition on a brand new dataset. This dataset is unconstrained and unique. We proposed a new feature vector that is robust to background noise and capable of capturing dynamic textures. We also proposed a novel method for fusing the output of many face detectors. Both approaches provided better results than other state‐of‐the‐art methods. In the future work, the face detection scheme will be scaled up to a 3D model to better detect the extreme out of plane head rotations.
