Evaluation of Facial Analysis Techniques and Methods

Facial expression analysis is one of the most studied problems in computer vision, motivated by numerous applications in industry, clinical research, entertainment, and marketing. Variations in head pose create a significant challenge for facial expression analysis, as expressions look significantly different from different angles. Disentangling facial expression from pose is important for improving expression recognition accuracy, as well as from an explainable AI standpoint, as one cannot reliably interpret the decisions of a facial behavior analysis system if the expression coefficients are confounded by head movements.

We study limitations of current approaches and modeling techniques (e.g., use of weak perspective camera model), as well as assessing “true” performance of state-of-the-art methods in behavioral research. To this end, we derive theoretical limitations of those methods and experimentally quantify their error bounds.

3D-to-2D mapping errors vs. rotation amount for various FOVs (30,60,90) and subject-to-camera distances (close, far). (a) yaw rotation; (b) pitch rotation. The y axis shows error rate relative to interocular distance.
The spurious expressions using (a) Basel'17 and (b) ITWMM. Each red arrow depicts the effect of the spurious expression on the corresponding landmark.

Source Codes
Assessing separability of facial pose and expression with weak perspective camera model

Sariyanidi E., Zampella C. J., Schultz R. T., Tunç B., Can Facial Pose and Expression Be Separated with Weak Perspective Camera?, IEEE Conference on Computer Vision and Pattern Recognition, Accepted for publication, 2020