emotion recognition using continuous wavelet transform and ensemble of convolutional neural networks through transfer learning from electroencephalogram signal

Emotions are integral brain states that can influence our behavior, decision-making, and functions. electroencephalogram (eeg) is an appropriate modality for emotion recognition since it has high temporal resolution and is a non-invasive and cheap technique. materials and methods: a novel approach based on ensemble pre-trained convolutional neural networks (ecnns) is proposed to recognize four emotional classes from eeg channels of individuals watching music video clips. first, scalograms are built from one-dimensional eeg signals by applying the continuous wavelet transform (cwt) method. then, these images are used to re-train five cnns: alexnet, vgg-19, inception-v1, resnet-18, and inception-v3. then, the majority voting method is applied to make the final decision about emotional classes. the 10-fold cross-validation method is used to evaluate the performance of the proposed method on eeg signals of 32 subjects from the deap database. results: the experiments showed that applying the proposed ensemble approach in combinations of scalograms of frontal and parietal regions improved results. the best accuracy, sensitivity, precision, and f-score to recognize four emotional states achieved 96.90% ± 0.52, 97.30 ± 0.55, 96.97 ± 0.62, and 96.74 ± 0.56, respectively. conclusion: so, the newly proposed model from eeg signals improves recognition of the four emotional states in the deap database.