Explanation of Prediction Models with ExplainPrediction
Abstract
State-of-the-art prediction models are getting increasingly complex and incomprehensible for humans. This is problematic for many application areas, especially those where knowledge discovery is just as important as predictive performance, for example medicine or business consulting. As machine learning and artificial intelligence are playing an increasingly large role in the society through data based decision making, this is problematic also from broader perspective and worries general public as well as legislators. As a possible solution, several explanation methods have been recently proposed, which can explain predictions of otherwise opaque models. These methods can be divided into two main approaches: gradient based approaches limited to neural networks, and more general perturbation based approaches, which can be used with arbitrary prediction models. We present an overview of perturbation based approaches, and focus on a recently introduced implementation of two successful methods developed in Slovenia, EXPLAIN and IME. We first describe their working principles and visualizations of explanations, followed by the implementation in ExplainPrediction package for R environment.
Full Text:
PDFReferences
Charu C Aggarwal, Chen Chen, and Jiawei Han. The inverse classification problem. Journal of Computer Science and Technology, 25(3):458–468, 2010.
Leila Arras, Franziska Horn, Gregoire Montavon, Klaus-Robert Muller, and Wojciech Samek. What is relevant in a text document?: An interpretable machine learning approach. PloS ONE, 12(8):e0181142, 2017.
Sebastian Bach, Alexander Binder, Gregoire Montavon, Frederick Klauschen, Klaus-Robert Muller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS ONE, 10(7):e0130140, 2015.
David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Muller. How to explain individual classification decisions. Journal of Machine Learning Research, 11(Jun):1803–1831, 2010.
David Barbella, Sami Benzaid, JanaraMChristensen, Bret Jackson, X Victor Qin, and David R Musicant. Understanding support vector machine classifications via a recommender system-like approach. In R. Stahlbock, S. F. Crone, and S. Lessmann, editors, Proceedings of International Conference on Data Mining, pp. 305–311, 2009.
Adriano Barbosa, FV Paulovich, Afonso Paiva, Siome Goldenstein, Fabiano Petronetto, and LG Nonato. Visualizing and interacting with kernelized data. IEEE transactions on visualization and computer graphics, 22(3):1314–1325, 2016.
Marko Bohanec, Mirjana Borstnar Kljajic, and Marko Robnik-Sikonja. Explaining machine learning models in sales predictions. Expert Systems with Applications, 71:416–428, 2017.
Marko Bohanec, Marko Robnik-Sikonja, and Mirjana Kljajic Borstnar. Decision-making framework with double-loop learning through interpretable black-box machine learning models. Industrial Management & Data Systems, 117(7):1389–1406, 2017.
Zoran Bosnic, Jaka Demsar, Grega Kespret, Pedro Pereira Rodrigues, Joao Gama, and Igor Kononenko. Enhancing data stream predictions with reliability estimators and explanation. Engineering Applications of Artificial Intelligence, 34:178–192, 2014.
Jaka Demšar and Zoran Bosnić. Detecting concept drift in data streams using model explanation. Expert Systems with Applications, 92:546 – 559, 2018.
Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics, 24(1):44–65, 2015.
Lutz Hamel. Visualization of support vector machines with unsupervised learning. In Proceedings of 2006 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, 2006.
atja Hansen, David Baehrens, Timon Schroeter, Matthias Rupp, and Klaus-Robert Muller. Visual interpretation of kernel-based prediction models. Molecular Informatics, 30(9):817–826, 2011.
Aleks Jakulin, Martin Mozina, Janez Demsar, Ivan Bratko, and Blaz Zupan. Nomograms for visualizing support vector machines. In Robert Grossman, Roberto Bayardo, and Kristin P. Bennett, editors, Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 108–117. ACM, 2005.
Vincent Lemaire, Raphael Feraud, and Nicolas Voisine. Contact personalization using a score understanding method. In Proceedings of International Joint Conference on Neural Networks (IJCNN), 2008.
David Martens and Foster Provost. Explaining documents’
classifications. Technical report, Center for Digital Economy Research, New York University, Stern School of Business, 2011. Working paper CeDER-11-01.
David Meyer, Friedrich Leisch, and Kurt Hornik. The support vector machine under test. Neurocomputing, 55:169–186, 2003.
Rok Piltaver, Mitja Lustrek, Matjaz Gams, and Sandra Martincic-Ipsic. What makes classification trees comprehensible? Expert Systems with Applications, 62:
–346, 2016.
Brett Poulin, Roman Eisner, Duane Szafron, Paul Lu, Russell Greiner, David S. Wishart, Alona Fyshe, Brandon Pearcy, Cam Macdonell, and John Anvik. Visual explanation of evidence with additive classifiers. In Proceedings of AAAI’06. AAAI Press, 2006.
Marko Pregeljc, Erik Strumbelj, Miran Mihelcic, and Igor Kononenko. Learning and explaining the impact of enterprises’ organizational quality on their economic results. In R. Magdalena-Benedito, M. Martinez-Sober, J. M. Martinez-Martinez, P. Escandell-Moreno, and J. Vila-Frances, editors, Intelligent Data Analysis for Real-Life Applications: Theory and
Practice, pages 228–248. Information Science Reference, IGI Global, 2012.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. ACM, 2016.
Marko Robnik-Sikonja. Data generators for learning systems based on rbf networks. IEEE Transactions on Neural Networks and Learning Systems, 27(5):926–938, May 2016.
Marko Robnik-Sikonja. ExplainPrediction: Explanation of Predictions for Classification and Regression, 2017. URL http://cran.r-project.org/package=ExplainPrediction. R package version 1.3.0.
Marko Robnik-Sikonja and Igor Kononenko. Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering, 20(5):589–600, 2008.
Marko Robnik-Sikonja and Petr Savicky. CORElearn - classification, regression, feature evaluation and ordinal evaluation, 2017. URL http://cran.r-project.org/package=CORElearn. R package version 1.52.0.
Andrea Saltelli, Karen Chan, and E. Marian Scott. Sensitivity analysis. Wiley, Chichester; New York, 2000.
Alexander Schulz, Andrej Gisbrecht, and Barbara Hammer. Using discriminative dimensionality reduction to visualize classifiers. Neural Processing Letters, 42(1):27–54, 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
Erik Strumbelj, Zoran Bosnic, Igor Kononenko, Branko Zakotnik, and Cvetka Grasic Kuhar. Explanation and reliability of prediction models: the case of breast cancer recurrence. Knowledge and information systems, 24(2):305–324, 2010.
Erik Strumbelj and Igor Kononenko. An Efficient Explanation of Individual Classifications using Game Theory. Journal of Machine Learning Research, 11:1–18, 2010.
Erik Strumbelj, Igor Kononenko, and Marko Robnik-Sikonja. Explaining instance classifications with interactions of subsets of feature values. Data & Knowledge Engineering, 68(10):886–904, 2009.
This work is licensed under a Creative Commons Attribution 3.0 License.