automatic micro-expression recognition using lbp-sipl and fr-cnn

Facial expressions are one of the most effective ways for non-verbal communications, which can be expressed as the micro-expression (me) in the high-stake situations. the mes are involuntary, rapid, subtle, and can reveal real human intentions. however, their feature extraction is very challenging due to their low intensity and very short duration. although local binary pattern on three orthogonal plane (lbp-top) feature extractor is useful for me analysis, it does not consider essential information. to address this problem, we propose a new feature extractor called local binary pattern from six intersection planes (lbp-sipl). this method extracts lbp code on six intersection planes, and then combines them. results show that the proposed method has superior performance in apex frame spotting automatically in comparison with the relevant methods on the casme i and the casme ii databases. afterwards, the apex frames are the input of the fast region-based convolutional neural network (fr-cnn) to recognize the facial expressions. simulation results show that the me has been automatically recognized in 81.56% and 96.11% on the casme i and the casme ii databases by using the proposed method, respectively.