Justifying convolutional neural network with argumentation for explainability
Abstract
Convolutional neural network (CNN) has emerged as one of the most accurate methods for sentiment analysis, but it is largely uninterpretable, while case-based reasoning (CBR) is less accurate but offers interpretable outputs in the form of arguments from analogy. This paper presents an approach to combine these two methods, CNN for accuracy and CBR for explainability, using an assumption-based argumentation (ABA) framework. Our approach focuses on justifying CNN outputs using analogous sentences from CBR, while ensuring that the combined process is argumentative and hence self-explainable.
To demonstrate the proposal, we construct a CNN model M1 and a CBR model M2 for sentiment analysis using different subsets of a dataset of which the remaining part is used for testing and comparing these input models with combined models. For an input sentence, if M1 and M2 predict the same sentiment, then the analogous sentence, which M2 finds, is used to explain the sentiment. If they give conflicting sentiments, a hybrid model M3 determines which one should be followed using a system of strict rules that takes into account how assertive M1 and M2 are. Another hybrid model M4, which is implemented by an ABA framework, improves on M3 by considering the probability distribution of the set of all labels from M1, and the second (or third) most similar sentences from M2. M3 and M4 preserve the accuracy of the CNN model (specifically, 88.32% and 88.28% in comparison with 87.59% accuracy of the CNN). They justify 69.95% and 74.53% of CNN outputs, respectively.
Full Text:
PDFReferences
Bing Liu. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167, 2012.
Subhabrata Mukherjee and Pushpak Bhattacharyya. Sentiment analysis: A literature survey. arXiv preprint arXiv:1304.4520, 2013.
Ethem Alpaydin. Introduction to machine learning. MIT press, 2020.
Kitsuchart Pasupa and Thititorn Seneewong Na Ayutthaya. Thai sentiment analysis with deep learning techniques: A comparative study based on word embedding, pos-tag, and sentic features. Sustainable Cities and Society, 2019.
Neha Nandal, Rohit Tanwar, and Jyoti Pruthi. Machine learning based aspect level sentiment analysis for amazon products. Spatial Information Research, pages 1–7, 2020.
D Shubham, P Mithil, Meesala Shobharani, and S Sumathy. Aspect level sentiment analysis using machine learning. In Materials Science and Engineering Conference Series, volume 263, page 042009, 2017.
Duyu Tang, Bing Qin, and Ting Liu. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1422– 1432, 2015.
Richa Sharma, Shweta Nigam, and Rekha Jain. Opinion mining of movie reviews at document level. arXiv preprint arXiv:1408.3829, 2014.
Zhifei Zhang, Duoqian Miao, Zhihua Wei, and Lei Wang. Document-level sentiment classification based on behavior-knowledge space method. In International Conference on Advanced Data Mining and Applications, pages 330–339. Springer, 2012.
John Rothfels and Julie Tibshirani. Unsupervised sentiment classification of english movie reviews using automatic selection of positive and negative sentiment items. CS224N-Final Project, 43(2):52–56, 2010.
Informatica 45 page 501–yyy
S.H.P Oo et al.
VS Jagtap and Karishma Pawar. Analysis of different approaches to sentence-level sentiment classification. International Journal of Scientific Engineering and Technology, 2(3):164–170, 2013.
Efstratios Kontopoulos, Christos Berberidis, Theologos Dergiades, and Nick Bassiliades. Ontology-based sentiment analysis of twitter posts. Expert systems with applications, 40(10):4065–4074, 2013.
S Behdenna, Fatiha Barigou, and Ghalem Belalem. Document level sentiment analysis: A survey. EAI Endorsed Transactions on ContextAware Systems and Applications, 4(13), 2018.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
JianxinWu.Introductiontoconvolutionalneural networks. National Key Lab for Novel Software Technology. Nanjing University. China, 5:23, 2017.
Larry Medsker and Lakhmi C Jain. Recurrent neural networks: design and applications. CRC press, 1999.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
Lei Zhang, Shuai Wang, and Bing Liu. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4):e1253, 2018.
QuratTulAin,MubashirAli,AmnaRiaz,Amna Noureen, Muhammad Kamran, Babar Hayat, and A Rehman. Sentiment analysis using deep learning techniques: a review. Int J Adv Comput Sci Appl, 8(6):424, 2017.
David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong
Yang. Xai-explainable artificial intelligence. Science Robotics, 4(37), 2019.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
Henry Prakken and Rosa Ratsma. A top-level model of case-based argumentation for explanation: formalisation and experiments. Argument & Computation, (Preprint):1–36.
KristijonasCˇyras,AntonioRago,EmanueleAlbini, Pietro Baroni, and Francesca Toni. Argumentative xai: A survey. arXiv preprint arXiv:2105.11266, 2021.
Phan Minh Dung, Robert A Kowalski, and Francesca Toni. Assumption-based argumentation. In Argumentation in artificial intelligence, pages 199–218. Springer, 2009.
Ahmed Al-Ani and Mohamed Deriche. A new technique for combining multiple classifiers using the dempster-shafer theory of evidence. Journal of Artificial Intelligence Research, 17:333–361, 2002.
Lei Xu, Adam Krzyzak, and Ching Y Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE transactions on systems, man, and cybernetics, 22(3):418–435, 1992.
GalinaRogova.Combiningtheresultsofseveral neural network classifiers. In Classic Works of the Dempster-Shafer Theory of Belief Functions, pages 683–692. Springer, 2008.
David A Bell, Ji-wen W Guan, and Yaxin Bi. On combining classifier mass functions for text categorization. IEEE transactions on knowledge and data engineering, 17(10):1307–1319, 2005.
Yaxin Bi. The impact of diversity on the accuracy of evidential classifier ensembles. International Journal of Approximate Reasoning, 53(4):584–607, 2012.
Cuong Anh Le, Van-Nam Huynh, Akira Shimazu, and Yoshiteru Nakamori. Combining classifiers for word sense disambiguation based on dempster–shafer theory and owa operators. Data & Knowledge Engineering, 63(2):381– 396, 2007.
Van-Nam Huynh, Tri Thanh Nguyen, and Cuong Anh Le. Adaptively entropy-based weighting classifiers in combination using dempster–shafer theory for word sense disambiguation. Computer Speech & Language, 24(3):461–473, 2010.
Justifying convolutional neural network with argumentation for explainability. . . Informatica 45 page 501–yyy 19
Vahid Yaghoubi, Liangliang Cheng, Wim Van Paepegem, and Mathias Kersemans. A novel multi-classifier information fusion based on dempster–shafer theory: application to vibration-based fault detection. Structural Health Monitoring, page 14759217211007130, 2020.
Chenbin Zhang, Ningning Qin, and Le Yang. Optimal combination of svm and bayesian density model using dempster-shafer theory. In Proceedings of the 2020 12th International Conference on Machine Learning and Computing, pages 505–509, 2020.
Sajjad Talesh Hosseini, Omid Asghari, and Parham Pahlavani. A hybrid approach to model the dykes in sungun porphyry copper deposit using dempster–shafer theory. Arabian Journal of Geosciences, 13(24):1–20, 2020.
Sergio Peñafiel, Nelson Baloian, Horacio Sanson, and José A Pino. Applying dempster–shafer theory for developing a flexible, accurate and interpretable classifier. Expert Systems with Applications, 148:113262, 2020.
Amalendu Si, Sujit Das, and Samarjit Kar. Picture fuzzy set-based decision-making approach using dempster–shafer theory of evidence and grey relation analysis and its application in covid-19 medicine selection. Soft Computing, pages 1–15, 2021.
Thimmaiah Gudiyangada Nachappa, Sepideh Tavakkoli Piralilou, Khalil Gholamnia, Omid Ghorbanzadeh, Omid Rahmati, and Thomas Blaschke. Flood susceptibility mapping with machine learning, multi-criteria decision analysis and ensemble using dempster shafer
theory. Journal of Hydrology, page 125275, 2020.
Dai Quoc Nguyen, Dat Quoc Nguyen, Thanh Vu, and Son Bao Pham. Sentiment classification on polarity reviews: an empirical study using rating-based features. 2014.
Bernhard Lutz, Nicolas Pröllochs, and Dirk Neumann. Sentence-level sentiment analysis of financial news using distributed text representations and multi-instance learning. arXiv preprint arXiv:1901.00400, 2018.
Bishan Yang and Claire Cardie. Context-aware learning for sentence-level sentiment analysis with posterior regularization. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 325–335, 2014.
Vrushali K Bongirwar. A survey on sentence level sentiment analysis. International Journal of Computer Science Trends and Technology (IJCST), 3(3), 2015.
Kim Schouten and Flavius Frasincar. Survey on aspect-level sentiment analysis. IEEE Transactions on Knowledge and Data Engineering, 28(3):813–830, 2015.
Yu Mon Aye and Sint Sint Aung. Senti-lexicon and analysis for restaurant reviews of myanmar text. International Journal of Advanced Engineering, Management and Science, 4(5), 2018.
Anna Jurek, Maurice D Mulvenna, and Yaxin Bi. Improved lexicon-based sentiment analysis for social media analytics. Security Informatics, 4(1):1–13, 2015.
Cataldo Musto, Giovanni Semeraro, and Marco Polignano. A comparison of lexicon-based approaches for sentiment analysis of microblog posts. In DART@ AI* IA, pages 59–68, 2014.
Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. Lexiconbased methods for sentiment analysis. Computational linguistics, 37(2):267–307, 2011.
Nitika Nigam and Divakar Yadav. Lexiconbased approach to sentiment analysis of tweets using r language. In International Conference on Advances in Computing and Data Sciences, pages 154–164. Springer, 2018.
Ali Hasan, Sana Moin, Ahmad Karim, and Shahaboddin Shamshirband. Machine learningbased sentiment analysis for twitter accounts. Mathematical and Computational Applications, 23(1):11, 2018.
Deepak Kumar Gupta and Asif Ekbal. Iitp: supervised machine learning for aspect based sentiment analysis. 2014.
Informatica 45 page 501–yyy
S.H.P Oo et al.
Zhang Hailong, Gan Wenyan, and Jiang Bo. Machine learning and lexicon based methods for sentiment classification: A survey. In 2014 11th Web Information System and Application Conference, pages 262–265. IEEE, 2014.
Lei Zhang, Riddhiman Ghosh, Mohamed Dekhil, Meichun Hsu, and Bing Liu. Combining lexicon-based and learning-based methods for twitter sentiment analysis. HP Laboratories, Technical Report HPL-2011, 89, 2011.
Nipuna Upeka Pannala, Chamira Priyamanthi Nawarathna, JTK Jayakody, Lakmal Rupasinghe, and Kesavan Krishnadeva. Supervised learning based approach to aspect based sentiment analysis. In 2016 IEEE International Conference on Computer and Information Technology (CIT), pages 662–666. IEEE, 2016.
Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. Lstm neural networks for language modeling. In Thirteenth annual conference of the international speech communication association, 2012.
Asad Abdi, Siti Mariyam Shamsuddin, Shafaatunnur Hasan, and Jalil Piran. Deep learning-based sentiment classification of evaluative text based on multi-feature fusion. Information Processing & Management, 56(4):1245–1259, 2019.
Sujata Rani and Parteek Kumar. Deep learning based sentiment analysis using convolution neural network. Arabian Journal for Science and Engineering, 44(4):3305–3314, 2019.
Ashish Kumar and Aditi Sharan. Deep learning-based frameworks for aspect-based sentiment analysis. In Deep Learning-Based Approaches for Sentiment Analysis, pages 139– 158. Springer, 2020.
Usman Naseem, Imran Razzak, Shah Khalid Khan, and Mukesh Prasad. A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models. Transactions on Asian and Low-Resource Language Information Processing, 20(5):1–35, 2021.
Olivier Habimana, Yuhua Li, Ruixuan Li, Xiwu Gu, and Ge Yu. Sentiment analysis using deep
learning approaches: an overview. Science China Information Sciences, 63(1):1–36, 2020.
Aliaksei Severyn and Alessandro Moschitti. Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 959–962. ACM, 2015.
CiceroDosSantosandMairaGatti.Deepconvolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 69– 78, 2014.
Peerapon Vateekul and Thanabhat Koomsubha. A study of sentiment analysis using deep learning techniques on thai twitter data. In 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), pages 1–6. IEEE, 2016.
Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.
Hannah Kim and Young-Seob Jeong.
ment classification using convolutional neural networks. Applied Sciences, 9(11):2347, 2019.
Jan Milan Deriu and Mark Cieliebak. Sentiment analysis using convolutional neural networks with multi-task training and distant supervision on italian tweets. In Fifth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, Napoli, Italy, December 5-7, 2016. Italian Journal of Computational Linguistics, 2016.
Igor Santos, Nadia Nedjah, and Luiza de Macedo Mourelle. Sentiment analysis using convolutional neural network with fasttext embeddings. In 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pages 1–5. IEEE, 2017.
Nadia Nedjah, Igor Santos, and Luiza de Macedo Mourelle. Sentiment analysis using convolutional neural network via word embeddings. Evolutionary Intelligence, pages 1–25, 2019.
Senti-
Justifying convolutional neural network with argumentation for explainability. . . Informatica 45 page 501–yyy 21
JT Turner, Michael W Floyd, Kalyan Gupta, and Tim Oates. Nod-cc: A hybrid cbr-cnn architecture for novel object discovery. In International Conference on Case-Based Reasoning, pages 373–387. Springer, 2019.
Faliang Huang, Xuelong Li, Changan Yuan, Shichao Zhang, Jilian Zhang, and Shaojie Qiao. Attention-emotion-enhanced convolutional lstm for sentiment analysis. IEEE Transactions on Neural Networks and Learning Systems, 2021.
XiaoyanYan,FanghongJian,andBoSun.Sakgbert: Enabling language representation with knowledge graphs for chinese sentiment analysis. IEEE Access, 9:101695–101701, 2021.
Petr Berka. Sentiment analysis using rule-based and case-based reasoning. Journal of Intelligent Information Systems, pages 1–16, 2020.
Marco Lippi and Paolo Torroni. Argument mining: A machine learning perspective. In International Workshop on Theory and Applications of Formal Argumentation, pages 163–176. Springer, 2015.
Oana Cocarascu and Francesca Toni. Argumentation for machine learning: A survey. In COMMA, pages 219–230, 2016.
Oana Cocarascu and Francesca Toni. Mining bipolar argumentation frameworks from natural language text. 2017.
RihabBouslama,RaouiaAyachi,andNahlaBen Amor. Using convolutional neural network in cross-domain argumentation mining framework. In International Conference on Scalable Uncertainty Management, pages 355–367. Springer, 2019.
Vlad Niculae, Joonsuk Park, and Claire Cardie. Argument mining with structured svms and rnns. arXiv preprint arXiv:1704.06869, 2017.
Lucas Carstens and Francesca Toni. Using argumentation to improve classification in natural language problems. ACM Transactions on Internet Technology (TOIT), 17(3):30, 2017.
Kathrin Grosse, María P González, Carlos I Chesnevar, and Ana G Maguitman. Integrating
argumentation and sentiment analysis for mining opinions from twitter. AI Communications, 28(3):387–401, 2015.
OanaCocarascuandFrancescaToni.Identifying attack and support argumentative relations using deep learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1374–1379, 2017.
Oana Cocarascu and Francesca Toni. Combining deep learning and argumentative reasoning for the analysis of social media textual content using small data sets. Computational Linguistics, 44(4):833–858, 2018.
Oana Cocarascu, Kristijonas Cyras, and Francesca Toni. Explanatory predictions with artificial neural networks and argumentation. 2018.
Mark T Keane and Eoin M Kenny. How case-based reasoning explains neural networks: A theoretical analysis of xai using post-hoc explanation-by-example from a survey of anncbr twin-systems. In International Conference on Case-Based Reasoning, pages 155–171. Springer, 2019.
Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Y Lo, and Cynthia Rudin. Iaia-bl: A case-based interpretable deep learning model for classification of mass lesions in digital mammography. arXiv preprint arXiv:2103.12308, 2021.
David Leake and David Crandall. On bringing case-based reasoning methodology to deep learning. In International Conference on CaseBased Reasoning, pages 343–348. Springer, 2020.
David Leake, Xiaomeng Ye, and David J Crandall. Supporting case-based reasoning with neural networks: An illustration for case adaptation. In AAAI Spring Symposium: Combining
Informatica 45 page 501–yyy
S.H.P Oo et al.
Machine Learning with Knowledge Engineering, 2021.
Michael M Richter and Rosina O Weber. Casebased reasoning. Springer, 2016.
Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications, 7(1):39–59, 1994.
Phan Minh Dung. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and nperson games. Artificial intelligence, 77(2):321– 357, 1995.
Saung Hnin Pwint Oo, Thanaruk Theeramunkong, and Nguyen Duy Hung. Sentence sentiment classification using convolutional neural network in myanmar texts. In Proceedings of the 2020 2nd International Conference on Image, Video and Signal Processing, pages 144– 149, 2020.
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. nat mach intell. 2019; 1: 206–215.
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. Local rule-based explanations of black box decision systems. arXiv preprint arXiv:1805.10820, 2018.
Mark T Keane and Eoin M Kenny. The twinsystem approach as one generic solution for xai: An overview of ann-cbr twins for explaining deep learning. arXiv preprint arXiv:1905.08069, 2019.
Jérémie Clos, Nirmalie Wiratunga, and Stewart Massie. Towards explainable text classification by jointly learning lexicon and modifier terms. 2017.
Dino Pedreschi, Fosca Giannotti, Riccardo Guidotti, Anna Monreale, Luca Pappalardo, Salvatore Ruggieri, and Franco Turini. Open the black box data-driven explanation of black box decision systems. arXiv preprint arXiv:1806.09936, 2018.
Dino Pedreschi, Fosca Giannotti, Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, and Franco Turini. Meaningful explanations of black box ai decision systems. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 9780–9784, 2019.
Alejandro Barredo Arrieta, Natalia DíazRodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82–115, 2020.
DOI: https://doi.org/10.31449/inf.v46i9.4359
This work is licensed under a Creative Commons Attribution 3.0 License.