Deep Learning-Driven Edge-Enabled Serverless Architectures for Animal Emotion Detection
Abstract
Full Text:
PDFReferences
Andresen, N., Wöllhaf, M., Hohlbaum, K., Lewejohann, L., Hellwich, O., Thöne-Reineke, C., & Belik, V. (2020). Towards a fully automated surveillance of well-being status in laboratory mice using deep learning: Starting with facial expression analysis. PloS one, 15(4), e0228059. https://doi.org/10.1371/journal.pone.0228059
Andersen, P. H., Broomé, S., Rashid, M., Lundblad, J., Ask, K., Li, Z., Hernlund, E., et al. (2021). Towards Machine Recognition of Facial Expressions of Pain in Horses. Animals, 11(6), 1643. MDPI AG. Retrieved from http://dx.doi.org/10.3390/ani11061643.
Anwar Y. (2017). Emoji fans take heart: Scientists pinpoint 27 states of emotion, https://news.berkeley.edu/2017/09/06/27-eions/, accessed in Jan. 2023.
Auer U, Kelemen Z, Engl V, Jenner F (2021) Activity Time Budgets—A Potential Tool to Monitor Equine Welfare? Animals 11:850. https://doi.org/10.3390/ani11030850.
Battini M., Agostini A., & Mattiello, S. (2019). Understanding Cows' Emotions on Farm: Are Eye White and Ear Posture Reliable Indicators?, Animals : an open access journal from MDPI, 9(8), 477. https://doi.org/10.3390/ani9080477.
Bekoff, M. (2023). Human Emotions in Animals, https://online.uwa.edu/news/empathy-in-animals/, Accessed in Jan. 2023.
Benedict, S. (2021). Performance Issues and Monitoring Mechanisms for Serverless IoT Applications—An Exploratory Study. In Smart Computing Techniques and Applications (pp. 165–174). Springer Singapore. https://doi.org/10.1007/978-981-16-0878-0_17
Benedict, S. (2022). Deep Learning Technologies for Social Impact. IOP Publishing. https://doi.org/10.1088/978-0-7503-4024-3
Blumrosen, G., Hawellek, D., & Pesaran, B. (2017). Towards Automated Recognition of Facial Expressions in Animal Models. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2810–2819. doi:10.1109/ICCVW.2017.332.
Boneh-Shitrit, T., Amir, S., Bremhorst, A., Mills, D., Riemer, S., Fried, D., & Zamansky, A. (2022). Deep Learning Models for Automated Classification of Dog Emotional States from Facial Expressions. https://doi.org/10.48550/arXiv.2206.05619
Briefer, E. F. (2012). Vocal expression of emotions in mammals: mechanisms of production and evidence. In S. Le Comber (Ed.), Journal of Zoology (Vol. 288, Issue 1, pp. 1–20). Wiley; Portico. https://doi.org/10.1111/j.1469-7998.2012.00920.x
Bremhorst, A., Sutter, N. A., Würbel, H., Mills, D. S., & Riemer, S. (2019). Differences in facial expressions during positive anticipation and frustration in dogs awaiting a reward. In Scientific Reports (Vol. 9, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-019-55714-6
Broomé, S., Gleerup, K. B., Andersen, P. H., & Kjellstrom, H. (2019). Dynamics Are Important for the Recognition of Equine Pain in Video. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2019.01295
Broomé, S., Ask, K., Rashid-Engström, M., Haubro Andersen, P., & Kjellström, H. (2022). Sharing pain: Using pain domain transfer for video recognition of low grade orthopedic pain in horses. In A. Seal (Ed.), PLOS ONE (Vol. 17, Issue 3, p. e0263854). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0263854
Caeiro, C. C., Burrows, A. M., & Waller, B. M. (2017). Development and application of CatFACS: Are human cat adopters influenced by cat facial expressions? In Applied Animal Behaviour Science (Vol. 189, pp. 66–78). Elsevier BV. https://doi.org/10.1016/j.applanim.2017.01.005
Camerlink, I., Coulange, E., Farish, M., Baxter, E. M., & Turner, S. P. (2018). Facial expression as a potential measure of both intent and emotion. In Scientific Reports (Vol. 8, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-018-35905-3
Chen, C., Zhu, W., Steibel, J., Siegford, J., Wurtz, K., Han, J., & Norton, T. (2020). Recognition of aggressive episodes of pigs based on convolutional neural network and long short-term memory. In Computers and Electronics in Agriculture (Vol. 169, p. 105166). Elsevier BV. https://doi.org/10.1016/j.compag.2019.105166
Clemins, P. J., & Johnson, M. T. (2003). Application of speech recognition to African elephant (Loxodonta africana) vocalizations. In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP ’03). International Conference on Acoustics, Speech and Signal Processing (ICASSP’03). IEEE. https://doi.org/10.1109/icassp.2003.1198823
Corujo, L. A., Kieson, E., Schloesser, T., & Gloor, P. A. (2021). Emotion Recognition in Horses with Convolutional Neural Networks. In Future Internet (Vol. 13, Issue 10, p. 250). MDPI AG. https://doi.org/10.3390/fi13100250
Critical Needs for Research in Veterinary Science. (2005). National Academies Press. https://doi.org/10.17226/11366
Dai, F., Leach, M., MacRae, A. M., Minero, M., & Dalla Costa, E. (2020). Does Thirty-Minute Standardised Training Improve the Inter-Observer Reliability of the Horse Grimace Scale (HGS)? A Case Study. In Animals (Vol. 10, Issue 5, p. 781). MDPI AG. https://doi.org/10.3390/ani10050781
Dyson, S., & Pollard, D. (2020). Application of a Ridden Horse Pain Ethogram and Its Relationship with Gait in a Convenience Sample of 60 Riding Horses. In Animals (Vol. 10, Issue 6, p. 1044). MDPI AG. https://doi.org/10.3390/ani10061044
Ede, T., Lecorps, B., von Keyserlingk, M. A. G., & Weary, D. M. (2019). Symposium review: Scientific assessment of affective states in dairy cattle. In Journal of Dairy Science (Vol. 102, Issue 11, pp. 10677–10694). American Dairy Science Association. https://doi.org/10.3168/jds.2019-16325
Ekman, P. (2023). Basic Emotions. https://www.paulekman.com/wp-content/uploads/2013/07/Basic-Emotions.pdf, Accessed Jan. 2023.
Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System. In PsycTESTS Dataset. American Psychological Association (APA). https://doi.org/10.1037/t27734-000
Evangelista, M. C., Watanabe, R., Leung, V. S. Y., Monteiro, B. P., O’Toole, E., Pang, D. S. J., & Steagall, P. V. (2019). Facial expressions of pain in cats: the development and validation of a Feline Grimace Scale. In Scientific Reports (Vol. 9, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-019-55693-8
Feighelstein, M., Shimshoni, I., Finka, L. R., Luna, S. P. L., Mills, D. S., & Zamansky, A. (2022). Automated recognition of pain in cats. In Scientific Reports (Vol. 12, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-022-13348-1
Ferres, K., Schloesser, T., & Gloor, P. A. (2022). Predicting Dog Emotions Based on Posture Analysis Using DeepLabCut. In Future Internet (Vol. 14, Issue 4, p. 97). MDPI AG. https://doi.org/10.3390/fi14040097
Finka, L. R., Luna, S. P., Brondani, J. T., Tzimiropoulos, Y., McDonagh, J., Farnworth, M. J., Ruta, M., & Mills, D. S. (2019). Geometric morphometrics for the study of facial expressions in non-human animals, using the domestic cat as an exemplar. In Scientific Reports (Vol. 9, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-019-46330-5
Franzoni, V., Milani, A., Biondi, G., & Micheli, F. (2019). A Preliminary Work on Dog Emotion Recognition. In IEEE/WIC/ACM International Conference on Web Intelligence - Companion Volume. WI ’19: IEEE/WIC/ACM International Conference on Web Intelligence. ACM. https://doi.org/10.1145/3358695.3361750
Gehlot, A., Malik, P. K., Singh, R., Akram, S. V., & Alsuwian, T. (2022). Dairy 4.0: Intelligent Communication Ecosystem for the Cattle Animal Welfare with Blockchain and IoT Enabled Technologies. In Applied Sciences (Vol. 12, Issue 14, p. 7316). MDPI AG. https://doi.org/10.3390/app12147316
Guo, S., Xu, P., Miao, Q., Shao, G., Chapman, C. A., Chen, X., He, G., Fang, D., Zhang, H., Sun, Y., Shi, Z., & Li, B. (2020). Automatic Identification of Individual Primates with Deep Learning Techniques. In iScience (Vol. 23, Issue 8, p. 101412). Elsevier BV. https://doi.org/10.1016/j.isci.2020.101412
Häger, C., Biernot, S., Buettner, M., Glage, S., Keubler, L. M., Held, N., Bleich, E. M., Otto, K., Müller, C. W., Decker, S., Talbot, S. R., & Bleich, A. (2017). The Sheep Grimace Scale as an indicator of post-operative distress and pain in laboratory sheep. In I. A. S. Olsson (Ed.), PLOS ONE (Vol. 12, Issue 4, p. e0175839). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0175839
Hummel, H. I., Pessanha, F., Salah, A. A., van Loon, T. J. P. A. M., & Veltkamp, R. C. (2020). Automatic Pain Detection on Horse and Donkey Faces. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). IEEE. https://doi.org/10.1109/fg47880.2020.00114
ICARUS, (2023). ICARUS Global Monitoring. Animals Early Warning System. https://www.icarus.mpg.de/28810/animals-warning-sensors, accessed in Jan. 2023.
Iglesias, P. M., & Camerlink, I. (2022). Tail posture and motion in relation to natural behaviour in juvenile and adult pigs. In Animal (Vol. 16, Issue 4, p. 100489). Elsevier BV. https://doi.org/10.1016/j.animal.2022.100489
Kret, M. E., Massen, J. J. M., & de Waal, F. B. M. (2022). My Fear Is Not, and Never Will Be, Your Fear: On Emotions and Feelings in Animals. In Affective Science (Vol. 3, Issue 1, pp. 182–189). Springer Science and Business Media LLC. https://doi.org/10.1007/s42761-021-00099-x
Langbauer, W. R. (2000). Elephant communication. In Zoo Biology (Vol. 19, Issue 5, pp. 425–445). Wiley. https://doi.org/10.1002/1098-2361(2000)19:5<425::aid-zoo11>3.0.co;2-a
Lencioni, G. C., de Sousa, R. V., de Souza Sardinha, E. J., Corrêa, R. R., & Zanella, A. J. (2021). Pain assessment in horses using automatic facial expression recognition through deep learning-based modeling. In H. Nisar (Ed.), PLOS ONE (Vol. 16, Issue 10, p. e0258672). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0258672
Lundblad, J., Rashid, M., Rhodin, M., & Haubro Andersen, P. (2021). Effect of transportation and social isolation on facial expressions of healthy horses. In E. Palagi (Ed.), PLOS ONE (Vol. 16, Issue 6, p. e0241532). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0241532
Li, Z., Broomé, S., Andersen, P.H., & Kjellström, H. (2021). Automated Detection of Equine Facial Action Units. ArXiv, abs/2102.08983.
Maisonpierre, I. N., Sutton, M. A., Harris, P., Menzies‐Gow, N., Weller, R., & Pfau, T. (2019). Accelerometer activity tracking in horses and the effect of pasture management on time budget. In Equine Veterinary Journal (Vol. 51, Issue 6, pp. 840–845). Wiley. https://doi.org/10.1111/evj.13130
Mendl, M., Neville, V., & Paul, E. S. (2022). Bridging the Gap: Human Emotions and Animal Emotions. In Affective Science (Vol. 3, Issue 4, pp. 703–712). Springer Science and Business Media LLC. https://doi.org/10.1007/s42761-022-00125-6
Marsot, M., Mei, J., Shan, X., Ye, L., Feng, P., Yan, X., Li, C., & Zhao, Y. (2020). An adaptive pig face recognition approach using Convolutional Neural Networks. In Computers and Electronics in Agriculture (Vol. 173, p. 105386). Elsevier BV. https://doi.org/10.1016/j.compag.2020.105386
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018). DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. In Nature Neuroscience (Vol. 21, Issue 9, pp. 1281–1289). Springer Science and Business Media LLC. https://doi.org/10.1038/s41593-018-0209-y
McLennan, K., & Mahmoud, M. (2019). Development of an Automated Pain Facial Expression Detection System for Sheep (Ovis Aries). In Animals (Vol. 9, Issue 4, p. 196). MDPI AG. https://doi.org/10.3390/ani9040196
Mogil, J. S., Pang, D. S. J., Silva Dutra, G. G., & Chambers, C. T. (2020). The development and use of facial grimace scales for pain measurement in animals. In Neuroscience; Biobehavioral Reviews (Vol. 116, pp. 480–493). Elsevier BV. https://doi.org/10.1016/j.neubiorev.2020.07.013
Mota-Rojas, D., Olmos-Hernández, A., Verduzco-Mendoza, A., Hernández, E., Martínez-Burnes, J., & Whittaker, A. L. (2020). The Utility of Grimace Scales for Practical Pain Assessment in Laboratory Animals. In Animals (Vol. 10, Issue 10, p. 1838). MDPI AG. https://doi.org/10.3390/ani10101838
Navarro, E., Mainau, E., & Manteca, X. (2020). Development of a Facial Expression Scale Using Farrowing as a Model of Pain in Sows. In Animals (Vol. 10, Issue 11, p. 2113). MDPI AG. https://doi.org/10.3390/ani10112113
Nath, T., Mathis, A., Chen, A. C., Patel, A., Bethge, M., & Mathis, M. W. (2019). Using DeepLabCut for 3D markerless pose estimation across species and behaviors. In Nature Protocols (Vol. 14, Issue 7, pp. 2152–2176). Springer Science and Business Media LLC. https://doi.org/10.1038/s41596-019-0176-0
Neethirajan, S. (2021). Happy Cow or Thinking Pig? WUR Wolf—Facial Coding Platform for Measuring Emotions in Farm Animals. In AI (Vol. 2, Issue 3, pp. 342–354). MDPI AG. https://doi.org/10.3390/ai2030021
Neethirajan, S., & Kemp, B. (2021). Digital Livestock Farming. In Sensing and Bio-Sensing Research (Vol. 32, p. 100408). Elsevier BV. https://doi.org/10.1016/j.sbsr.2021.100408
Neethirajan, S., Reimert, I., & Kemp, B. (2021). Measuring Farm Animal Emotions—Sensor-Based Approaches. In Sensors (Vol. 21, Issue 2, p. 553). MDPI AG. https://doi.org/10.3390/s21020553
Pandey, S., Kalwa, U., Kong, T., Guo, B., Gauger, P. C., Peters, D. J., & Yoon, K.-J. (2021). Behavioral Monitoring Tool for Pig Farmers: Ear Tag Sensors, Machine Intelligence, and Technology Adoption Roadmap. In Animals (Vol. 11, Issue 9, p. 2665). MDPI AG. https://doi.org/10.3390/ani11092665
Panksepp, J. (2005). Affective consciousness: Core emotional feelings in animals and humans. In Consciousness and Cognition (Vol. 14, Issue 1, pp. 30–80). Elsevier BV. https://doi.org/10.1016/j.concog.2004.10.004
Pennington, Z. T., Dong, Z., Feng, Y., Vetere, L. M., Page-Harley, L., Shuman, T., & Cai, D. J. (2019). ezTrack: An open-source video analysis pipeline for the investigation of animal behavior. In Scientific Reports (Vol. 9, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-019-56408-9
Pereira, T. D., Aldarondo, D. E., Willmore, L., Kislin, M., Wang, S. S.-H., Murthy, M., & Shaevitz, J. W. (2018). Fast animal pose estimation using deep neural networks. In Nature Methods (Vol. 16, Issue 1, pp. 117–125). Springer Science and Business Media LLC. https://doi.org/10.1038/s41592-018-0234-5
Pessanha, F., McLennan, K., & Mahmoud, M. (2020). Towards automatic monitoring of disease progression in sheep: A hierarchical model for sheep facial expressions analysis from video. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). IEEE. https://doi.org/10.1109/fg47880.2020.00107
Rashid, M., Silventoinen, A., Gleerup, K. B., & Andersen, P. H. (2020). Equine Facial Action Coding System for determination of pain-related facial responses in videos of horses. In U. G. Munderloh (Ed.), PLOS ONE (Vol. 15, Issue 11, p. e0231608). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0231608
Rashid, M., Broomé, S., Ask, K., Hernlund, E., Andersen, P.H., Kjellström, H., & Lee, Y.J. (2021). Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 152-162.
Raspa, F., Tarantola, M., Muca, E., Bergero, D., Soglia, D., Cavallini, D., Vervuert, I., Bordin, C., De Palo, P., & Valle, E. (2022). Does Feeding Management Make a Difference to Behavioural Activities and Welfare of Horses Reared for Meat Production? In Animals (Vol. 12, Issue 14, p. 1740). MDPI AG. https://doi.org/10.3390/ani12141740
Samadiani, Huang, Cai, Luo, Chi, Xiang, & He. (2019). A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor Data. In Sensors (Vol. 19, Issue 8, p. 1863). MDPI AG. https://doi.org/10.3390/s19081863
Sénèque, E., Lesimple, C., Morisset, S., & Hausberger, M. (2019). Could posture reflect welfare state? A study using geometric morphometrics in riding school horses. In J. J. Loor (Ed.), PLOS ONE (Vol. 14, Issue 2, p. e0211852). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0211852
Singh, S., & Benedict, S. (2020). Indian Semi-Acted Facial Expression (iSAFE) Dataset for Human Emotions Recognition. In Communications in Computer and Information Science (pp. 150–162). Springer Singapore. https://doi.org/10.1007/978-981-15-4828-4_13
Soltis, J., Leong, K., & Savage, A. (2005). African elephant vocal communication II: rumble variation reflects the individual identity and emotional state of callers. In Animal Behaviour (Vol. 70, Issue 3, pp. 589–599). Elsevier BV. https://doi.org/10.1016/j.anbehav.2004.11.016
Soltis, J., Blowers, T. E., & Savage, A. (2011). Measuring positive and negative affect in the voiced sounds of African elephants (Loxodonta africana). In The Journal of the Acoustical Society of America (Vol. 129, Issue 2, pp. 1059–1066). Acoustical Society of America (ASA). https://doi.org/10.1121/1.3531798
Sotocina, S. G., Sorge, R. E., Zaloum, A., Tuttle, A. H., Martin, L. J., Wieskopf, J. S., Mapplebeck, J. C., Wei, P., Zhan, S., Zhang, S., McDougall, J. J., King, O. D., & Mogil, J. S. (2011). The Rat Grimace Scale: A Partially Automated Method for Quantifying Pain in the Laboratory Rat via Facial Expressions. In Molecular Pain (Vol. 7, pp. 1744-8069-7–55). SAGE Publications. https://doi.org/10.1186/1744-8069-7-55
Statham, P., Hannuna, S., Jones, S., Campbell, N., Robert Colborne, G., Browne, W. J., Paul, E. S., & Mendl, M. (2020). Quantifying defence cascade responses as indicators of pig affect and welfare using computer vision methods. In Scientific Reports (Vol. 10, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1038/s41598-020-65954-6
Stolz, K., Heyder, T., Gloor, P. A., & Posegga, O. (2019). Measuring Human-Animal Interaction with Smartwatches: An Initial Experiment. In Studies on Entrepreneurship, Structural Change and Industrial Dynamics (pp. 165–182). Springer International Publishing. https://doi.org/10.1007/978-3-030-17238-1_10
Thangavel, S., & Shokkalingam, C. S. (2021). The IoT based embedded system for the detection and discrimination of animals to avoid human–wildlife conflict. In Journal of Ambient Intelligence and Humanized Computing (Vol. 13, Issue 6, pp. 3065–3081). Springer Science and Business Media LLC. https://doi.org/10.1007/s12652-021-03141-9
Tomkins, S. S., & McCarter, R. (1964). What and Where are the Primary Affects? Some Evidence for a Theory. In Perceptual and Motor Skills (Vol. 18, Issue 1, pp. 119–158). SAGE Publications. https://doi.org/10.2466/pms.1964.18.1.119
Tsai, M.-F., & Huang, J.-Y. (2021). Sentiment analysis of pets using deep learning technologies in artificial intelligence of things system. In Soft Computing (Vol. 25, Issue 21, pp. 13741–13752). Springer Science and Business Media LLC. https://doi.org/10.1007/s00500-021-06038-z
Waller, B.M., Caeiro, C., Peirce, K., Burrows, A.M., & Kaminski, J. (2013). DogFACS: the dog facial action coding system.
Waller, B. M., Julle-Daniere, E., & Micheletta, J. (2020). Measuring the evolution of facial ‘expression’ using multi-species FACS. In Neuroscience; Biobehavioral Reviews (Vol. 113, pp. 1–11). Elsevier BV. https://doi.org/10.1016/j.neubiorev.2020.02.031
Wathan, J., Burrows, A. M., Waller, B. M., & McComb, K. (2015). Correction: EquiFACS: The Equine Facial Action Coding System. In PLOS ONE (Vol. 10, Issue 9, p. e0137818). Public Library of Science (PLoS). https://doi.org/10.1371/journal.pone.0137818
Xu, B., Wang, W., Guo, L., Chen, G., Li, Y., Cao, Z., & Wu, S. (2022). CattleFaceNet: A cattle face identification approach based on RetinaFace and ArcFace loss. In Computers and Electronics in Agriculture (Vol. 193, p. 106675). Elsevier BV. https://doi.org/10.1016/j.compag.2021.106675
Zeppelzauer, M., & Stoeger, A. S. (2015). Establishing the fundamentals for an elephant early warning and monitoring system. In BMC Research Notes (Vol. 8, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1186/s13104-015-1370-y
Zhu, H., Salgırlı, Y., Can, P., Atılgan, D., & Salah, A. A. (2022). Video-based estimation of pain indicators in dogs. arXiv preprint arXiv:2209.13296.
DOI: https://doi.org/10.31449/inf.v49i7.6615

This work is licensed under a Creative Commons Attribution 3.0 License.