Experimental Comparisons of Multi-class Classifiers

Abstract

The multi-class classification algorithms are widely used by many areas such as machine learning and
computer vision domains. Nowadays, many literatures described multi-class algorithms, however there
are few literature that introduced them with thorough theoretical analysis and experimental
comparisons. This paper discusses the principles, important parameters, application domain, runtime
performance, accuracy, and etc. of twelve multi-class algorithms: decision tree, random forests,
extremely randomized trees, multi-class adaboost classifier, stochastic gradient boosting, linear and
nonlinear support vector machines, K nearest neighbors, multi-class logistic classifier, multi-layer
perceptron and naive Bayesian classifier. The experiment tested on five data sets: SPECTF heart data
set, Ionosphere radar data set, spam junk mail filter data set, optdigits handwriting data set and scene
15 image classification data set. Our major contribution is that we study the relationships between each
classifier and impact of each parameters to classification results. The experiment shows that gradient
boosted trees, nonlinear support vector machine, K nearest neighbor reach high performance under the
circumstance of binary classification and minor data capacity; Under the condition of high dimension,
multi-class and big data, however, gradient boosted trees, linear support vector machine, multi-class
logistic classifier get good results. At last, the paper addresses the development and future of multi-class
classifier algorithms.