Leveraging Spark-TensorFlow Distributor for Distributed Deep Convolutional Neural Networks: Accelerating large-scale COVID-19 Detection

Saliha Mezzoudj, Meriem Khelifa, Yasmina Saadna

Abstract


The COVID-19 epidemic has been a critical global challenge due to its high mortality rate and rapid spread. Initial diagnostic methods, such as chest X-rays and reverse transcriptase polymerase chain reaction (RTPCR), can be time-consuming and require enhanced efficiency, especially in scenarios where large-scale testing is necessary. Combining deep learning (DL) and big data analytics tools is crucial for developing accurate and fast systems to detect COVID-19 infections and prevent further transmission. In this study, we introduce a distributed Deep Convolutional Neural Network (DCNN) framework designed for the detection of COVID-19, utilizing a comprehensive dataset of chest X-ray images. Our approach leverages the spark-tensorflow-distributor, which integrates TensorFlow 2.x with Apache Spark 3.x, enabling distributed training within the Spark framework. The system utilizes Spark’s barrier mode to execute distributed training tasks, significantly enhancing the speed of both training and testing phases. We classify chest X-ray images into two categories: COVID-19 and non-COVID-19. The experimental setup involves processing the large-scale dataset of chest X-ray images in a parallelized manner using the Spark framework. Our results demonstrate that the proposed DCNN model achieves a classification accuracy of 95.53%, with a testing time of 0.06 seconds and a training time of 18,909 seconds. Furthermore, when compared to other state-of-the-art methods for COVID-19 classification, our approach proves to be more efficient and effective. This study highlights the potential of combining distributed computing frameworks with deep learning techniques to address largescale medical image analysis challenges, particularly in the context of COVID-19 detection.

Full Text:

PDF

References


Bharati, S., Podder, P., Mondal, M., and Prasath, V. B.: Medical imaging with deep learning for COVID-19 diagnosis: a comprehensive review. arXiv preprint arXiv:2107.09602 (2021).

Riswantini, D., Nugraheni, E., Arisal, A., Khotimah, P., H., Munandar, D., Suwarn-

ingsih, W.: Big data research in fighting COVID-19: Contributions and techniques. Big Data and Cognitive Computing 5(3), 30–89 (2021)

Bharati, S., Podder, P., Mondal, M., and Prasath, V. B.: Medical imaging with

deep learning for COVID-19 diagnosis: a comprehensive review. arXiv preprint arXiv:2107.09602 (2021)

Kaleem, S., Sohail, A., Tariq, M. U., Babar, M., and Qureshi, B.: Ensemble learn-

ing for multi-class COVID-19 detection from big data. Plos one, 18(10), e0292587 (2023).

Bobulski, J., and Kubanek, M: Big Data System for Medical Images Analysis. In

MLIS, 71–79, (2020)

Benbrahim, H., Hachimi, H., and Amine, A. (2020). Deep transfer learning with apache spark to detect COVID-19 in chest X-ray images. Romanian Journal of Information Science and Technology, 23(S, SI), S117–S129 (2020).

Ali Abbood K., Ali Noori K., Laith Hikmet M., Predictive analytics on COVID-19

data using Hive based on Hadoop cluster, 31(3), Indonesian Journal of Electrical Engineering and Computer Science, 945–956 (2023).

Rayan, R., Zafar, I., Tsagkaris, C.: Intelligent Data Analysis for COVID-19 Pan-

demic. Springer, 115–127 (2021)

Gaur, L., Bhatia, U., Jhanjhi, N. Z., Muhammad, G., and Masud, M.: Medical image-based detection of COVID-19 using deep convolution neural networks. Mul- timedia systems, 29(3), 1729–1738 (2023).

Santosh, K. C., and Ghosh, S.: Covid-19 imaging tools: How big data is big?.

Journal of Medical Systems, 45(7), 71 (2021).

Alsunaidi, S. J., Almuhaideb, A. M., Ibrahim, N. M., Shaikh, F. S., Alqudaihi, K. S., Alhaidari, F. A., and Alshahrani, M. S.: Applications of big data analytics to control COVID-19 pandemic. Sensors, 21(7), 2282 (2021).

Tahmassebi, A., Ehtemami, A., Mohebali, B., Gandomi, A. H., Pinker, K., and

Meyer-Baese: Big data analytics in medical imaging using deep learning, In Big Data: Learning, Analytics, and Applications textbf10989, 86–101(2019)

Panahi, A. H., Rafiei, A., and Rezaee, A.: FCOD: Fast COVID-19 Detector based

on deep learning techniques. Informatics in medicine unlocked, textbf22, 100506 (2021)

Ayalew, A. M., Salau, A. O., Tamyalew, Y., Abeje, B. T., and Woreta, N.: X-

Ray image-based COVID-19 detection using deep learning. Multimedia Tools and Applications, 1–19(2023).

Sarp, S., Catak, F. O., Kuzlu, M., Cali, U., Kusetogullari, H., Zhao, Y., and Guler,

O.: An XAI approach for COVID-19 detection using transfer learning with X-ray images, Heliyon, 9(4) (2023).

Sakr, A. S., P˚Aawiak, P., Tadeusiewicz, R., P˚Aawiak, J., Sakr, M., and Ham-

mad, M: ECG-COVID: An end-to-end deep model based on electrocardiogram for COVID-19 detection, Information Sciences, 619, 324–339 (2023).

Siddique, A. A., Talha, S. U., Aamir, M., Algarni, A. D., Soliman, N. F., and

El-Shafai, W.: COVID-19 Classification from X-Ray Images: An Approach to Im- plement Federated Learning on Decentralized Dataset, Computers, Materials and Continua, 3883–3901(2023).

Heymann, D. L., and Shindo, N: COVID-19: what is next for public health?. The lancet, ,395(10224), 542–545 (2020).

Sohrabi, C., Alsafi, Z., O’neill, N., Khan, M., Kerwan, A., Al-Jabir, A., and Agha, R.: World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19). International journal of surgery, 76, 71–76 (2020).

Huang, P., Liu, T., Huang, L., Liu, H., Lei, M., Xu, W., ... and Liu, B.: Use of chest CT in combination with negative RT-PCR assay for the 2019 novel coronavirus but high clinical suspicion, Radiology, 295(1), 22-23(2020).

Ng, M. Y., Lee, E. Y., Yang, J., Yang, F., Li, X., Wang, H., ... and Kuo, M. D.: Imaging profile of the COVID-19 infection: radiologic findings and literature review. Radiology: Cardiothoracic Imaging, 2(1), e200034(2020).

Tian, X., Wang, J., Du, D., Li, S., Han, C., Zhu, G., ... and Lei, M.: Medical imaging and diagnosis of subpatellar vertebrae based on improved Laplacian image enhancement algorithm, Computer Methods and Programs in Biomedicine, 187, 105082(2020).

Solomon, M. D., McNulty, E. J., Rana, J. S., Leong, T. K., Lee, C., Sung, S. H.,

... and Go, A. S.: The Covid-19 pandemic and the incidence of acute myocardial infarction, New England Journal of Medicine, 383(7), 691–693 (2020).

Daniel, S. J. . Education and the COVID-19 pandemic. Prospects, textbf49(1), 91–96(2020).

Spinelli, A., and Pellino, G.: COVID-19 pandemic: perspectives on an unfolding crisis, Journal of British Surgery, 107(7), 785–787(2020).

Saliha, M., Ali, B., and Rachid, S.: Towards large-scale face-based race classifi- cation on spark framework. Multimedia Tools and Applications, 78(18), 26729– 26746(2019).

Mezzoudj, S., Behloul, A., Seghir, R., and Saadna, Y.: A parallel content-based image retrieval system using spark and tachyon frameworks. Journal of King Saud University-Computer and Information Sciences, 33(2), 141–149 (2021).

Narin, A., Kaya, C., and Pamuk, Z.: Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks. Pattern Anal- ysis and Applications, 24, 1207–1220 (2021)

Viceconti, M., Hunter, P., and Hose, R.: Big data, big knowledge: big data for personalized healthcare. IEEE journal of biomedical and health informatics, 19(4), 1209–1215 (2015).

Benbrahim, H., Hachimi, H., and Amine, A.: Deep transfer learning with apache spark to detect COVID-19 in chest X-ray images. Romanian Journal of Information Science and Technology, 23(S, SI), S117–S129 (2020).

Gupta, V. K., Gupta, A., Kumar, D., and Sardana, A.: Prediction of COVID-19 confirmed, death, and cured cases in India using random forest model, Big Data Mining and Analytics, 4(2), 116–123 (2021).

Wang, L., Lin, Z. Q., and Wong, A. (2020). Covid-net: A tailored deep convolu- tional neural network design for detection of covid-19 cases from chest x-ray images. Scientific reports, 10(1), 19549 (2020).

Agbehadji, I. E., Awuzie, B. O., Ngowi, A. B., and Millham, R. C.: Review of big data analytics, artificial intelligence and nature-inspired computing models towards accurate detection of COVID-19 pandemic cases and contact tracing, International journal of environmental research and public health, 17(15), 5330(2020).

Basu, S., Mitra, S., and Saha, N.: Deep learning for screening covid-19 using chest x-ray images, In 2020 IEEE symposium series on computational intelligence (SSCI), 2521–2527, IEEE(2020).

Hemdan, E. E. D., Shouman, M. A., and Karar, M. E.: Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images, arXiv preprint arXiv:2003.11055 (2020).

Krizhevsky, A., Sutskever, I., and Hinton, G. E., Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems, 25 (2012).

Dean, J., and Ghemawat, S.: MapReduce: simplified data processing on large clus- ters. Communications of the ACM, 51(1), 107–113 (2008).

Duran-Lopez, L., Dominguez-Morales, J. P., Corral-Jaime, J., Vicente-Diaz, S., and Linares-Barranco, A.: COVID-XNet: A custom deep learning system to diagnose and locate COVID-19 in chest X-ray images. Applied Sciences, 10(16), 5683(2020).

Lapez-Cabrera, J. D., Orozco-Morales, R., Portal-Diaz, J. A.,

and al.: Current limitations to identify COVID-19 using ar- tificial intelligence with chest X-ray imaging, Health and Technology, 11(2), 411- 424(2021).

Karau, H., Konwinski, A., Wendell, P., and Zaharia, M.: Learning spark: lightning- fast big data analysis. ” O’Reilly Media, Inc.”(2015).

Hemdan, E. E. D., Shouman, M. A., and Karar, M. E.: Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images, arXiv preprint arXiv:2003.11055 (2020).

Xu, X., Tian, H., Zhang, X., Qi, L., He, Q., Dou, W.: DisCOV: Distributed COVID- 19 detection on X-ray images with edge-cloud collaboration. IEEE Transactions on Services Computing, 15(3), 1206–1219 (2022).

Saadna, Yassmina, Ali Behloul, and Saliha Mezzoudj :Speed limit sign detection and recognition system using SVM and MNIST datasets.” Neural Computing and Applications 31.9, 5005–5015(2019).

Mezzoudj S.: Towards large scale image retrieval system using parallel frameworks. InMultimedia Information Retrieval, Dec 23. IntechOpen (2020).

Mezzoudj, S., Melkemi, K. E. (2021). A hybrid approach for shape retrieval using genetic algorithms and approximate distance. In Research Anthology on Multi- Industry Uses of Genetic Programming and Algorithms, IGI Global, 205-222(2021).

Borthakur, D.: HDFS architecture guide. Hadoop apache project, 53(1-13), 2 (2008).

Tartaglione Enzo, Alberto Barbano Carlo, Berzovini Claudio, Calandri Marco, Grangetto Marco. Unveiling covid-19 from chest x-ray with deep learning: a hurdles race with small data. arXiv preprint arXiv:2004.05405 (2020).

Afshar Parnian, Heidarian Shahin, Naderkhani Farnoosh, Oikonomou Anastasia, Plataniotis Konstantinos N, Mohammadi Arash. Covid-caps: a capsule network- based framework for identification of covid-19 cases from x-ray images, (2020).

Vaid Shashank, Kalantar Reza, Bhandari Mohit. Deep learning covid-19 detection- bias: accuracy through artificial intelligence. Int Orthop (2020).

WAHEED, Abdul, GOYAL, Muskan, GUPTA, Deepak, et al. Covidgan: data aug- mentation using auxiliary classifier gan for improved covid-19 detection, Ieee Access, 8, 91916—91923. (2020).

Ali Narin, Kaya Ceren, Pamuk Ziynet: Automatic detection of coronavirus disease(covid-19) using x-ray images and deep convolutional neural networks, 10849, (2020).

HEMDAN, Ezz El-Din, SHOUMAN, Marwa A., et KARAR, Mohamed Esmail. Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images. arXiv preprint arXiv:2003, 11055(2020).

Narin, Ali, Ceren Kaya, and Ziynet Pamuk: Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks. Pat- tern Analysis and Applications 24: 1207–1220(2021).

Ismael, A. M., sengur, A. :Deep Learning Approaches for COVID-19 Detection Based on Chest X- ray Images. Expert Systems with Applications, 114054(2020).

Dhiman, G., Chang, V., Kant Singh, K., Shankar, A. :“Adopt: automatic deep learning andoptimization-based approach for detection of novel coronavirus covid- 19 disease using x-ray images, Journal of biomolecular structure and dynamics, 1–13(2021).

Khelifa, Meriem, and Dalila Boughaci. :A variable neighborhood search method for solving the traveling tournaments problem, Electronic Notes in Discrete Math- ematics 47, 157–164(2015).

Khelifa, Meriem, Dalila Boughaci, and Esma A¨ımeur. : An enhanced genetic algo- rithm with a new crossover operator for the traveling tournament problem.4th Inter- national Conference on Control, Decision and Information Technologies (CoDIT). IEEE, (2017).

Khelifa, M., Boughaci, D.: Hybrid harmony search combined with variable neigh- borhood search for the traveling tournament problem. In Computational Collective Intelligence: 8th International Conference, ICCCI 2016, Halkidiki, Greece, Springer International Publishing, 28-30, 520–530(2016).

Khelifa, M., Boughaci, D., A¨ımeur, E. : A new approach based on graph match- ing and evolutionary approach for sport scheduling problem. Intelligent Decision Technologies, 14(4), 565–580(2020).

Sarp, S., Catak, F. O., Kuzlu, M., Cali, U., Kusetogullari, H., Zhao, Y. & Guler, O. : An XAI approach for COVID-19 detection using transfer learning with X-ray images. Heliyon, 9(4), (2023).

Kaleem, S., Sohail, A., Tariq, M. U., Babar, M., & Qureshi, B.: Ensemble learning for multi-class COVID-19 detection from big data, 18(10), (2023).

Salau AO, Jain S: Adaptive diagnostic machine learning technique for classification of cell decisions for AKT protein. In




DOI: https://doi.org/10.31449/inf.v49i17.7505

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.