Pain Facial Expression Recognition from Video Sequences Using Spatio-temporal Local Binary Patterns and Tracking Fiducial Points

Monitoring the facial expressions of patients in clinical environments is a necessity in addition to vital sign monitoring. pain monitoring of patients by facial expressions from video sequences eliminates the need for another person to accompany patients. in this paper, a novel approach is presented to monitor the expression of face and notify in case of pain using tracking fiducial points of face in video sequences and spatio-temporal local binary patterns (lbps) for eyes and eyebrows. the motion of eight fiducial points on facial features such as mouth, eyes, eyebrows are tracked by lucas-kanade algorithm and the movement angles are recorded in a feature vector which along with the spatio-temporal histogram of lbps creates a concatenated feature vector. spatio-temporal lbps boost the proposed algorithm to capture minor deformations on eyes and eyebrows. the feature vectors are then compared and classified using the chi-square similarity measure. experimental results show that leveraging spatio-temporal lbps improves the accuracy by 12% on stoic database