PSO with crossover operator applied to feature selection problem in classification

Houassi Hichem, Mahdaoui Rafik, Maarouk Toufik Mesaaoud

Abstract


In recent years, there is a large number of features in datasets used in pattern classification, which include relevant, irrelevant, and redundant features. However, irrelevant and redundant features decrease the computational time and reduce the classification performance. Feature selection is a preprocessing technique that which to choose a sub-set of relevant features to achieve a similar or even better classification performance than using all features. This paper presents two new hybrid algorithms for a feature selection called particle swarm optimization with crossover operator (denoted as PSOCO1 and PSOCO2); the algorithms are based on the integration of a particle swarm optimization (PSO) and a crossover operator (CO) of the genetic algorithms. A new relevant features vector (RFV) is introduced and used by our algorithms for execute a crossover operator between the RFV and other features vectors.  To demonstrate the effectiveness of these algorithms, we compared them with standard PSO [14], PSO4-2 [8] and HGAPSO [28] on twelve benchmark datasets. The results show that the two proposed algorithms significantly reduce the number of selected features and achieve similar or even better classification accuracy in almost all cases.

Full Text:

PDF

References


Enny I S, Sri H, Agus H, Retantyo W, Mudjosemedi M (2015) Feature Selection of the Combination of Porous Trabecular with Anthropometric Features for Osteoporosis Screening. International Journal of Electrical and Computer Engineering (IJECE) 5(1): 78--83

Zheng H, Zhang Y (2008) Feature selection for high dimensional data in astronomy. Advances of Space Research 41:1960–1964

Wang Q, Liu F, Wang X (2014) Multi-objective optimization of machining parameters considering energy consumption. International Journal of Advanced Manufacturing Technology 71:1133–1142

Hlaing T (2012) Feature Selection and Fuzzy Decision Tree for Network Intrusion Detection. International Journal of Informatics and Communication Technology 1(2):109--118

Kohavi R, John GH (1997) Wrappers for feature subset selection. Artificial Intelligence 97(1--2):273–324

Colak S, Isik C (2003) Feature subset selection for blood pressure classification using orthogonal forward selection. In: Proceedings of IEEE 29th Annual Bioengineering Conference, pp 122–123

Cotter SF, Kreutz-Delgado K., Rao BD (2001) Backward sequential elimination for sparse vector selection. Signal Processing 81:1849–1864

Xue B, Zhang M, Browne WN (2014) Particle swarm optimization for feature selection in classification: Novel initialization and updating mechanisms. Applied soft computing 18:261–276

Kanan HR, Faez K (2008) An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system. Applied Mathematics and Computation 205:716–725

Ani AA (2005) Ant colony optimization for feature subset selection. proceedings of world academy of science, engineering and technology 4:35--38

Gao HH, Yang HH, Wang XY (2005) Ant colony optimization based network intrusion feature selection and detection. Proceedings of the 4th international conference on machine learning and cyberneting. Vol. 6, , pp 3871–3875

Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimization for suppert vector machine. Expert systems with applications 31:231–240

Shuxin Z, Bin H (2013) Hybrid Feature Selection Based on Improved Genetic Algorithm. TELKOMNIKA Indonesian Journal of Electrical Engineering 11(4):1725--1730

Kennedy J, Eberhart R (1997) A discrete binary version of the particle swarm algorithm. In: IEEE International Conference on Systems, Man, and Cybernetics. Vol. 5, pp 4104–4108

Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3):528–539

Yang CS, Chuang LY, Ke CH, Yang CH (2008) Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation (CEC’08), pp 2093–2098

Lin SW, Lee ZJ, Chen SC, Tseng TY (2008) Parameter determination of support vector machine and feature selection using simulated annealing approach, Applied soft computing 8:1505–1512

Gherboudj A, Chikhi S (2011) BPSO Algorithms for Knapsack Problem. Communications in Computer and Information Science. 162:217–227

Chuang LY, Tsai SW, Yang CH (2011) Improved binary particle swarm optimization using catfish effect for feature selection. Expert system with application 38(10):12699–12707

Yang CS, Chuang LY, Ke CH (2008) Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation, pp 2093–2098

Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary PSO for feature selection using gene expression data. Computational Biology and Chemistry 32(29):29–38

Liu YN, Wang G, Chen HL, Dong H (2011) An Improved Particle Swarm Optimization for Feature Selection. Journal of Bionic Engineering 8(2):191–200

Fdhila R, Hamdani T, Alimi A (2011) Distributed MOPSO with a new population sub-division technique for the feature selection. In: IEEE 5th International Symposium on Computational Intelligence and Intelligence Informatics, pp 81–86

Hongfeng W, Dingwei W, Shengxiang Y (2007) Triggered Memory-Based Swarm Optimization in Dynamic Environments. In: Workshops on Applications of Evolutionary, pp 637–646

Sun J, Feng B, Xu W (2004) Particle Swarm Optimization with Particles Having Quantum Behavior. In: Congress on Evolutionary Computation, Portland, USA, pp 325–331

Hamed HNA, Kasabov NK, Shamsuddin (2011) SM Quantum inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks. Evolutionary algorithms 1:133–148

Mohemmed A, Zhang M, Johnston M (2009) Particle swarm optimization based AdaBoost for face detection. In: IEEE Congress on Evolutionary Computation, pp 2494–2501

Ghamissi P, Benediktsson JA (2015) Feature selection based on hybridization of genetic algorithm and particle swarm optimization. IEEE Geoscience and remote sensing letters 12(2):309–313

Frank A, Asuncion A, UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA, 2016.

Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2008) The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations Newsletter 11:10–18.




Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.