Develop Method to Efficiently Apply Image-Based Facial Emotion Classification Models to Video Data

Abstract

The ability to recognize emotions through facial cues, in childhood, is helpful for social interactions. Image-based facial emotion recognition models need low computing power, but cannot accept sequential information from video data. Conversely, video-based facial emotion recognition models require high computational power, so it cannot be easily applied in a low computing environment. In this paper, we propose a method that classifies the emotion from facial expression video data by applying threshold using an image-based model. The proposed method improves the accuracy of 3.67%, 24.74%, and 15.13% for each video dataset by reducing the non-emotion in the video and responding more sensitively to the expressed emotion than other methods that simply select the most frequent emotion in the video. The results of the study showed the threshold method can improve the performance of emotion classification without modifying the facial emotion classification model.