Optimized Training for Convolutional Neural Network Using Enhanced Grey Wolf Optimization Algorithm

Akram Guernine, Mohamed Tahar Kimour

Abstract


Convolutional Neural Networks (CNNs) are widely used in image classification tasks and have achieved significant performance. They have different applications with great success, especially in medical fields. The choice of architecture and hyperparameter settings of the CNN, highly influences both accuracy and its convergence speed. The empirical design and optimization of a new CNN architecture require a lot of expertise and can be very time-consuming. This paper proposes an enhanced Grey Wolf Optimization (GWO) algorithm to efficiently explore a defined space of potentially suitable CNN architectures, and simultaneously optimize their hyperparameters. Moreover, we introduce a spatial resolution reduction for a given image processing task, while taking the skin cancer detection as practical application. Through conducted experiments, we have shown that the obtained results are better than other classification methods in terms of accuracy and convergence speed.


Full Text:

PDF

References


Mirjalili, S. M., Lewis, A. Grey wolf optimizer. Advances in Engineering Software, 69, (2014). , 46–61.

G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Comput., vol. 18, no. 7, pp. 1527–1554, 2006.

Ali Bakhshi, Nasimul Noman, Zhiyong Chen, Fast Automatic Optimization of CNN Architectures for Image Classification Using Genetic Algorithm

C. Letellier, Chaos in nature, World Scientific Publishing Company, 81 (2013)

K. Price, R. Storn , Differential evolution: a simple evolution strategy for fast optimization. Dr Dobb’s J Software Tools 1997;22(4):18–24.

Coelho Ld, Ayala HV, Mariani VC. A self-adaptive chaotic differential evolution algorithm using gamma distribution for unconstrained global optimization. Appl Math Comput 2014;234:452–9.

U. Yüzgeç, M. Eser, Chaotic based differential evolution algorithm for optimization of baker’s yeast drying process, Egyptian Informatics Journal 19 (2018) 151–163

M. Suganuma, S. Shirakawa, and T. Nagao, “A genetic programming approach to designing convolutional neural network architectures,” in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 497–504, ACM, 2017.

B. Wang, Y. Sun, B. Xue, and M. Zhang, “Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification,” in 2018 IEEE Congress on Evolutionary Computation, CEC 2018 - Proceedings, 2018.

K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, “What is the best multi- stage architecture for object recognition?,” in Proceedings of the IEEE International Conference on Computer Vision, 2009.

T. Yamasaki, T. Honma, and K. Aizawa, “Efficient Optimization of Convolutional Neural Networks Using Particle Swarm Optimization,” in Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017, 2017.

J. Kennedy, R. Eberhart, "Particle swarm optimization", Proc. IEEE International Conf. on Neural Networks (Perth Australia), 1995.

S. Kumar, D. Datta, S. Kumar Singh, A. T. Azar, and S. Vaidyanathan, “Black hole algorithm and its applications,” Stud. Comput. Intell., 2015.

H. J. Kelley, “Gradient Theory of Optimal Flight Paths,” ARS J., vol. 30, no. 10, pp. 947–954, 1960.

M. Liu, J. Shi, Z. Li, C. Li, J. Zhu, and S. Liu, “Towards Better Analysis of Deep Convolutional Neural Networks,” IEEE Trans. Vis. Comput. Graph., 2017.

J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, “Understanding neural networks through deep visualization,” arXiv Prepr. arXiv1506.06579, 2015.

H. A. Ahmed, M. F. Zolkipli, and M. Ahmad, “A novel efficient substitution- box design based on firefly algorithm and discrete chaotic map,” Neural Computing and Applications, 2018.

A. A. Alzaidi, M. Ahmad, H. S. Ahmed, and E. Al Solami, “Sine-Cosine Optimization-Based Bijective Substitution-Boxes Construction Using Enhanced Dynamics of Chaotic Map,” Complexity, vol. 2018, 2018.

A. M. Taha, S.-D. Chen, and A. Mustapha, “Bat Algorithm Based Hybrid Filter- Wrapper Approach,” Adv. Oper. Res., vol. 2015, 2015.

A. M. Taha, S.-D. Chen, and A. Mustapha, “Natural Extensions: Bat Algorithm with Memory.,” J. Theor. Appl. Inf. Technol., vol. 79, no. 1, pp. 1–9, 2015.

Ogudo, K.A.; Muwawa Jean Nestor, D.; Ibrahim Khalaf, O.; Daei Kasmaei, H. A Device Performance and Data Analytics Concept for Smartphones‟ IoT Services and Machine-Type Communication in Cellular Networks. Symmetry 2019, 11, 593.

S. Q. Salih, A. A. Alsewari, B. Al- Khateeb, and M. F. Zolkipli, “Novel Multi-swarm Approach for Balancing Exploration and Exploitation in Particle Swarm Optimization,” in Recent Trends in Data Science and Soft Computing, 2019, pp. 196–206.

Z. A. Al Sudani, S. Q. Salih, Z. M. Yaseen, and others, “Development of Multivariate Adaptive Regression Spline Integrated with Differential Evolution Model for Streamflow Simulation,” J. Hydrol., pp. 1–15, 2019.

A. P. Piotrowski, J. J. Napiorkowski, and P. M. Rowinski, “How novel is the novel black hole optimization approach?,” Inf. Sci. (Ny)., 2014.

L. M. R. Rere, M. I. Fanany, and A. M. Arymurthy, “Simulated Annealing Algorithm for Deep Learning,” in Procedia Computer Science, 2015.

A. R. Syulistyo, D. M. J. Purnomo, M. F. Rachmadi, and A. Wibowo, “Particle swarm optimization (PSO) for training optimization on convolutional neural network (CNN),” J. Ilmu Komput. dan Inf., vol. 9, no. 1, pp. 52–58, 2016.

Boukaye Boubacar Traore , Bernard Kamsu-Foguem , Fana Tangara, Deep convolution neural network for image recognition, Ecological Informatics, November 2018

I. Arel, D. C. Rose, T. P. Karnowski, et al., “Deep machine learning new frontier in artificial intelligence research,” IEEE Computational Intelligence Magazine, vol. 5, no. 4, pp. 13–18, 2010.

M. Matsugu, K. Mori, Y. Mitari, and Y. Kaneda, “Subject independent facial expression recognition with robust face detection using a convolutional neural network,” Neural Networks, vol. 16, no. 5-6, pp. 555–559, 2003.

I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning, vol. 1. MIT press Cambridge, 2016.

W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, “Asurvey of deep neural network architectures and their applications,” Neurocomputing, vol. 234, pp. 11–26, 2017.

Tatsuki Serizawaa, and Hamido Fujita, Optimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization, Computer Science, ArXiv




DOI: https://doi.org/10.31449/inf.v45i5.3497

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.