Performance Evaluation of Machine Learning Models for Cyber Threat Detection and Prevention in Mobile Money Services
Abstract
In this paper, investigation was made to evaluate the effectiveness of the different classifiers suitable to predict the probability of cyber threat or fraudulent intent applicant during Mobile Money Service on-boarding or service activation process with the goal of determining the best machine learning model for the predictive model solution. Experimental work was carried out by formulating cyber threat predictive models, using six (6) supervised machine learning algorithms, Logistic regression, Naïve Bayes, Shallow Neural Network (SNN), Deep Neural Network (DNN), Classification and Regression Trees (CART) and Random Forest (RF) of different configurations. Each model was simulated with Synthetic Minority Operation Techniques (SMOTE) and without SMOTE (No-SMOTE) application on 25,000 dataset records of mobile money applicants. Twenty-four (24) different configurations of the formulated predictive models were simulated and evaluated using Python programming language. Simulation results of the predictive models proved that the Random Forest model multiclass configurations with SMOTE dataset outperformed all other configurations. The results also showed that the multiclass experiments with SMOTE has a better performance than the binary configuration without SMOTE of the predictive models. The study concluded that using the Random Forest based predictive machine learning model will increase the security level of Mobile Money solution by detecting and preventing anomalous customer registrations during mobile money for the unbanked on-boarding process.
Full Text:
PDFReferences
Castle, S., Pervaiz, F., Weld, G., Roesner, F. and Anderson, R. (2016). Let's Talk Money: Evaluating the Security Challenges of Mobile Money in the Developing World. In Proceedings of the 7th Annual Symposium on Computing for Development, Pages 1-10, https://doi.org/10.1145/3001913.3001919
Bara, A. (2013). Mobile money for financial inclusion: policy and regulatory perspective in Zimbabwe, African Journal of Science, Technology, Innovation and Development, Vol. 5 No. 5, pp. 345-354
Suárez, S.L. (2016). Poor people’s money: the politics of mobile money in Mexico and Kenya, Telecommunications Policy, Vol. 40 Nos 10/11, pp. 945-955.
Kanobe, F., Alexander, P.M. and Bwalya, K.J. (2017). Policies, regulations and procedures and their effects on mobile money systems in Uganda, The Electronic Journal of Information Systems in Developing Countries, Vol. 83 No. 1, pp. 1-15.
Akomea-Frimpong, I., Andoh, C., Akomea-Frimpong, A. and Dwomoh-Okudzeto, Y. (2019), Control of fraud on mobile money services in Ghana: an exploratory study, Journal of Money Laundering Control, Vol. 22 No. 2, pp. 300-317. https://doi.org/10.1108/JMLC-03-2018-0023
Célerier, C., Matray, A. (2019). Bank-branch supply, financial inclusion, and wealth accumulation. Rev. Finan. Stud. 32(12), 4767–4809.
Ahamed, M.M., Mallick, S.K. (2019). Is financial inclusion good for bank stability? International evidence. J. Econ. Behav. Organ. 157(1), 403–427.
Koi-Akrofi J. (2022). Mobile Money Adoption in Africa: A Literature-Based Analysis, Texila International Journal of Management, Vol. 8, No. 2, https://doi.org/10.21522/TIJMG.2015.08.02.Art014
Chatain, P.L., Zerzan, A., Noor, W., Dannaoui, N. and De Koker, L. (2011). Protecting Mobile Money against Financial Crimes: Global Policy Challenges and Solutions, World Bank Publications, New York, NY
Buku M. W. and Mazer R. (2017). Fraud in Mobile Financial Services: Protecting Consumers, Providers, and the System. World bank publications. https://documents1.worldbank.org/curated/en/249151504766545101/pdf/119208-BRI-PUBLIC-Brief-Fraud-in-Mobile-Financial-Services-April-2017.pdf
Pallangyo H. J., (2022). Cyber Security Challenges, its Emerging Trends on Latest Information and Communication Technology and Cyber Crime in Mobile Money Transaction Services, Tanzania Journal of Engineering and Technology, Vol. 41, No. 2, pp. 189-204, https://doi.org/10.52339/tjet.v41i2.79.
Azamuke D., Katarahweire M., Bainomugisha E., (2022).
Scenario-based Synthetic Dataset Generation for Mobile Money Transactions. In Proceedings of the Federated Africa and Middle East Conference on Software Engineering, pp. 64–72, https://doi.org/10.1145/3531056.3542774
Sanni M.L., Akinyemi B.O., Olalere D.A., Olajubu E.A. and Aderounmu G.A. (2023), A Predictive Cyber Threat Model for Mobile Money Service. Annals of Emerging Technologies in Computing, 7(1): 40-60, https://doi.org/10.33166/AETiC.2023.01.004
Ambore, S., Richardson, C., Dogan, H., Apeh, E. and Osselton, D. (2017). A resilient cybersecurity framework for Mobile Financial Services (MFS). Journal of Cyber Security Technology, 1(3-4): 202-224.
Zhdanova M., Repp J., Rieke R., Gaber C., and Hemery B. (2014). No Smurfs: Revealing Fraud Chains in Mobile Money Transfers. In proceedings of the International Conference on Availability, Reliability and Security (ARES), Switzerland. pp.10.
Botchey, F. E., Zhen Q., and Hughes-Lartey K. (2020). Mobile Money Fraud Prediction—A Cross-Case Analysis on the Efficiency of Support Vector Machines, Gradient Boosted Decision Trees, and Naïve Bayes Algorithms. Information, 11, no. 8, pp. 383. https://doi.org/10.3390/info11080383.
Eweoya, I., and Adebiyi A. A., Azeta, A, and Okesola, O. J. (2019). Fraud Prediction in Bank Credit Administration: A Systematic Literature Review. Journal of Theoretical and Applied Information Technology, 97 (11). pp. 3147-3169.
Akinnuwesi, B.A., Fashoto, S.G., Metfula, A.S., Akinnuwesi, A.N. (2020). Experimental Application of Machine Learning on Financial Inclusion Data for Governance in Eswatini. In: Hattingh, M., Matthee, M., Smuts, H., Pappas, I., Dwivedi, Y.K., Mäntymäki, M. (eds) Responsible Design, Implementation and Use of Information and Communication Technology. I3E 2020. Lecture Notes in Computer Science, vol 12067. Springer, Cham. https://doi.org/10.1007/978-3-030-45002-1_36
Botchey F. E., Qin Z., Hughes-Lartey K. and Ampomah K.E., (2021). Predicting Fraud in Mobile Money Transactions using Machine Learning: The Effects of Sampling Techniques on the Imbalanced Dataset. Informatica, 45, pp. 45–56. https://doi.org/10.31449/inf.v45i7.3179.
AlSaif S. A., and Hidri A., (2021). Impact of data balancing during training for best predictions, Informatica, 45(2) 223–230, vol. 45, no. 2, https://doi.org/10.31449/inf.v45i2.3479.
Benchaji, S. D. and ElOuahidi B., (2018). Using Genetic Algorithm to Improve Classification of Imbalanced Datasets for Credit Card Fraud Detection, in Proceedings of the 2018 2nd Cyber Security in Networking Conference (CSNet), Paris, France, pp. 1-5, https://doi.org/10.1109/CSNET.2018.8602972.
Bashir S., and Ghous H., (2021). Detecting Mobile Money Laundering Using Genetic Algorithm as Feature Selection Method with Classification Method. LC International Journal of STEM, 1(4), pp. 121-129, https://doi.org/10.5281/zenodo.5149794.
Bashir, S., Jabin S., Jabin S. (2021). Detecting Mobile Money Laundering Using KPCA as Feature Selection Method. LC International Journal of STEM, 2(3), pp.1-8. https://doi.org/10.5281/zenodo.5751721
Awoyemi J. O., Adetunmbi A. O., and Oluwadare S. A., (2017). Credit card fraud detection using machine learning techniques: A comparative analysis, in proceedings of the IEEE 2017 International Conference on Computing Networking and Informatics (ICCNI). pp. 1–9, https://doi.org/10.1109/iccni.2017.8123782.
Varmedja, D., Karanovic, M, . Sladojevic, S., Arsenovic, M., and Anderla, A. (2019). Credit card fraud detection-machine learning methods, In proceedings of the IEEE 2019 18th International Symposium INFOTEH-JAHORINA (INFOTEH), pp. 1–5, https://doi.org/10.1109/infoteh.2019.8717766
Gyamfi N. K. and Abdulai J. D., (2018). Bank Fraud Detection Using Support Vector Machine, in proceedings of the 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, pp. 37-41, https://doi.org/10.1109/IEMCON.2018.8614994.
Zhang, D., Bhandari, B. and Black, D. (2020). Credit Card Fraud Detection Using Weighted Support Vector Machine. Applied Mathematics, 11, 1275-1291, https://doi.org/10.4236/am.2020.1112087.
Adepoju O., Wosowei J., lawte S. and Jaiman H., (2019). Comparative Evaluation of Credit Card Fraud Detection Using Machine Learning Techniques, In proceedings of the 2019 Global Conference for Advancement in Technology (GCAT), Bangalore, India, pp. 1-6, https://doi.org/10.1109/GCAT47503.2019.8978372.
Khatri S., Arora A. and Agrawal A. P., (2020). Supervised Machine Learning Algorithms for Credit Card Fraud Detection: A Comparison, In proceedings of the 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, pp. 680-683, https://doi.org/10.1109/Confluence47617.2020.9057851.
Adedoyin, A., Kapetanakis, S., Samakovitis, G. and Petridis, M. (2017). Predicting fraud in mobile money transfer using case-based reasoning. In proceedings of the Artificial Intelligence XXXIV: 37th SGAI International Conference on Artificial Intelligence, AI 2017, Cambridge, UK, December 12-14, 2017, https://doi.org/10.1007/978-3-319-71078-5_28.
Delecourt, S., and Guo L. (2019). Building a Robust Mobile Payment Fraud Detection System with Adversarial Examples. In proceedings of the 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), 03-05 June 2019, https://doi.org/10.1109/AIKE.2019.00026.
Mambina, S., Ndibwile J. D., and Michael, K. F. (2022). Classifying Swahili Smishing Attacks for Mobile Money Users: A Machine-Learning Approach, IEEE Access, vol. 10, pp. 83061-83074, https://doi.org/10.1109/ACCESS.2022.3196464.
AnkitaKamle, M., MeherBhawnani, M., and AnkitaKamle M. (2021). Implementation and Design on Fraud Detection and Prediction of Mobile Money Transaction Using ML Techniques. Annals of the Romanian Society for Cell Biology, Vol. 24, No. 2, pp. 261 – 269.
Novikova E., and Kotenko I., (2014). Visual Analytics for Detecting Anomalous Activity in Mobile Money Transfer Services. In Proceedings of the International Cross-Domain Conference and Workshop on Availability, Reliability, and Security (CD-ARES), Sep, Fribourg, Switzerland. pp.63-78, https://doi.org/ff10.1007/978-3-319-10975-6_5ff.
Khan, M. R., and Blumenstock J. E., (2016). Predictors without borders: behavioral modeling of product adoption in three developing countries. In Proceedings of the ACM 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.145–154.
Centellegher, S., Miritello, G., Villatoro, D., Parameshwar, D., Lepri B., and Oliver N. (2018). In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. Vol.2, No. 7, pp 1–18, https://doi.org/10.1145/3287035.
DOI: https://doi.org/10.31449/inf.v47i6.4691
This work is licensed under a Creative Commons Attribution 3.0 License.