Facial Sentiment Analysis Using Convolutional Neural Network and Fuzzy Systems

Ahmed Rajab Kadhim, Raidah Salim Khudeyer, Maytham Alabbas

Abstract


 

This study provides a detailed study of a Convolutional Neural Network (CNN) model optimized for facial expression recognition with Fuzzy logic using Fuzzy2DPooling and Fuzzy Neural Networks (FNN), and discusses data augmentation in model optimization. It highlights important roles. performance. First, the effectiveness of the models in classifying emotions from FER2013, RAB-DB, and CK+ datasets was evaluated by a 5-fold cross-validation method, which showed that the accuracy varied widely among different emotion classes and was affected by overfitting. It turned out to be easy. The integration of data augmentation techniques, including random rotation, translation, and inversion, significantly improved the model's generalization capabilities. This was evidenced by higher accuracy and more consistent loss curves observed across all folds. After augmentation, the model showed significant improvement, achieving average test accuracies of 98.95% on FER2013, 99.99% on RAF-DB, and 100% on CK+ across all folds. Despite these advances, challenges specific to certain classes of emotions remain, highlighting the need for continued model refinement. This study concludes that data augmentation is an important step in developing robust facial expression recognition models and has potential benefits for a variety of applications requiring accurate emotion recognition.

 


Full Text:

PDF

References


Theodoridis, S. et al. (2010) Introduction to pattern recognition: A MATLAB approach. Amsterdam: Elsevier/Academic Press.

C. Albano et al., “Four levels of pattern recognition,” Anal. Chim. Acta, vol. 103, no. 4, pp. 429–443, 1978.

C. H. Chen, “A review of statistical pattern recognition,” in Pattern Recognition and Signal Processing, Dordrecht: Springer Netherlands, 1978, pp. 117–132.

K. S. Fu, Ed., Syntactic Pattern Recognition, Applications, 1977th ed. New York, NY: Springer, 2012.

O. Omidvar and J. Dayhoff, Neural Networks and Pattern Recognition. San Diego, CA: Academic Press, 1998.

M. Alloghani, D. Al-Jumeily, J. Mustafina, A. Hussain, and A. J. Aljaaf, “A systematic review on supervised and unsupervised machine learning algorithms for data science,” in Unsupervised and Semi-Supervised Learning, Cham: Springer International Publishing, 2020, pp. 3–21.

C. M. Travieso-Gonzalez, Ed., Applications of pattern recognition. London, England: IntechOpen, 2021.

C. Liang and J. Dong, “A survey of deep learning-based facial expression recognition research,” Frontiers in Computing and Intelligent Systems, vol. 5, no. 2, pp. 56–60, 2023.

I. M. Revina and W. R. S. Emmanuel, “A survey on human face expression recognition techniques,” J. King Saud Univ. - Comput. Inf. Sci., 2018.

S. Rajan, P. Chenniappan, S. Devaraj, and N. Madian, “Facial expression recognition techniques: a comprehensive survey,” IET Image Process., vol. 13, no. 7, pp. 1031–1040, 2019.

A. Vulpe-Grigorasi and O. Grigore, “Convolutional neural network hyperparameters optimization for facial emotion recognition,” in 2021 12th International Symposium on Advanced Topics in Electrical Engineering (ATEE), 2021.

Y. Khaireddin and Z. Chen, “Facial Emotion Recognition: state of the art performance on FER2013.,” arXiv (Cornell University), May 2021, [Online]. Available: https://arxiv.org/pdf/2105.03588

D. Zhang and Q. Tian, “A novel fuzzy optimized CNN-RNN method for facial expression recognition,” Elektronika Ir Elektrotechnika, vol. 27, no. 5, pp. 67–74, Oct. 2021, doi: 10.5755/j02.eie.29648.

Y. Yaermaimaiti, T. Kari, and G. Zhuang, “Research on facial expression recognition based on an improved fusion algorithm,” Nonlinear Eng., vol. 11, no. 1, pp. 112–122, 2022.

Four-layer Convnet to Facial Emotion Recognition With Minimal Epochs and the Signicance of Data Diversity. .

O. C. Oguine, K. J. Oguine, H. I. Bisallah, and D. Ofuani, “Hybrid Facial Expression Recognition (FER2013) model for real-time emotion classification and prediction,” arXiv [cs.CV], 2022.

S. Bobojanov, B. M. Kim, M. Arabboev, and S. Begmatov, “Comparative analysis of vision transformer models for facial emotion recognition using augmented balanced datasets,” Appl. Sci. (Basel), vol. 13, no. 22, p. 12271, 2023.

K. Wang, X. Peng, J. Yang, S. Lu, and Y. Qiao, “Suppressing uncertainties for large-scale facial expression recognition,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

Deep Learning Approaches for Classification of Emotion Recognition based on Facial Expressions.

S. Li and W. Deng, “A deeper look at facial expression dataset bias,” arXiv [cs.CV], 2019.

J. Shi, S. Zhu, and Z. Liang, “Learning to Amend facial expression representation via DE-albino and affinity,” arXiv [cs.CV], 2021.

D. Ruan, Y. Yan, S. Lai, Z. Chai, C. Shen, and H. Wang, “Feature decomposition and reconstruction learning for effective facial expression recognition,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

J.-H. Kim, B.-G. Kim, P. P. Roy, and D.-M. Jeong, “Efficient facial expression recognition algorithm based on hierarchical deep neural network structure,” IEEE Access, vol. 7, pp. 41273–41285, 2019.

Sawardekar, Sonali, and Sowmiya Raksha Naik. "Facial expression recognition using efficient LBP and CNN." Int Res J Eng Technol (IRJET) 5.6 (2018).‏

D. Lee and J.-S. Yoo, “CNN learning strategy for recognizing facial expressions,” IEEE Access, vol. 11, pp. 70865–70872, Jan. 2023, doi: 10.1109/access.2023.3294099.

S. S. Shafira, N. Ulfa, H. A. Wibawa, and Rismiyati, “Facial expression recognition using extreme learning machine,” in 2019 3rd International Conference on Informatics and Computational Sciences (ICICoS), 2019.

H. Song, “Comparison of Different Depth of Convolutional Neural Network Deep and shallow CNN comparison based on FER-2013,” Highlights in Science Engineering and Technology, vol. 41, pp. 80–86, Mar. 2023, doi: 10.54097/hset.v41i.6746.

X. Fan, Z. Deng, K. Wang, X. Peng, and Y. Qiao, “Learning discriminative representation for facial expression recognition from uncertainties,” in 2020 IEEE International Conference on Image Processing (ICIP), 2020.

S. Hershey et al., “CNN architectures for large-scale audio classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.

P. J. Woolf and Y. Wang, “A fuzzy logic approach to analyzing gene expression data,” Physiol. Genomics, vol. 3, no. 1, pp. 9–15, 2000.

M.-J. Hsu, Y.-H. Chien, W.-Y. Wang, and C.-C. Hsu, “A convolutional fuzzy neural network architecture for object classification with small training database,” Int. J. Fuzzy Syst., vol. 22, no. 1, pp. 1–10, 2020.

M. Sambare, “FER-2013.” 19-Jul-2020.

Dev-ShuvoAlok, “RAF-DB DATASET.” 20-Sep-2023.

Dev-ShuvoAlok, “CK+ dataset.” 20-Sep-2023.




DOI: https://doi.org/10.31449/inf.v48i12.6151

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.