Research on the Recognition of Psychological Emotions in Adults Using Multimodal Fusion

Abstract

Recognition of psychological emotions in adults plays a pivotal role in various fields. In this study, 110 healthy people and 110 depressed people were recruited. Gait, facial, and speech feature data were collected and screened. The obtained features were utilized as multimodal features. Subsequently, attention-bidirectional long short-term memory (BiLSTM) was developed as a method for recognizing adult psychological emotions. It was observed that better recognition results were achieved through multimodal fusion than unimodal approaches. The accuracy of the attention-BiLSTM method was 0.8182, and the F1 value was 0.8125, demonstrating higher recognition accuracy compared to methods such as the recurrent neural network and LSTM. These results verify the reliability of multimodal fusion and the attention-BiLSTM method fo recognizing psychological emotions in adults, and they can be applied in practice.