Hyperparameter Optimization for Malicious URL Detection: Leveraging Optuna and Random Search in Machine Learning and Deep Learning Models

Miloud Khaldi, Zohra Alilat, Hana Bendoubba, Nadir Mahammed

Abstract


Uniform Resource Locators (URLs) are critical indicators for identifying malicious online activities such as malware distribution, phishing attacks, and website defacement. This research presents a robust approach for detecting these threats using both deep learning (DL) and machine learning (ML) techniques. We emphasize hyperparameter optimization, employing Optuna—a Bayesian optimization framework— and Random Search to systematically enhance model performance. Unlike many prior studies, which often overlook thorough hyperparameter tuning, our approach demonstrates improvements over state of the art methods. Our Bidirectional Encoder Representations from Transformers (BERT) model achieved an accuracy of 98.84%, with an F1 score of 99.02%, while the Light Gradient Boosting Machine (LightGBM) attained an accuracy of 98.46% and an F1 score of 98.45%

Full Text:

PDF

References


A. Basit, M. Zafar, X. Liu, A. R. Javed, Z. Jalil, and K. Kifayat, “A comprehensive survey of ai-enabled phishing attacks detection techniques,” Springer Science, 2020. DOI: https : / / doi . org / 10 .1007/s11235-020-00733-2.

Phishing activity trends reports, https://apwg.org/trendsreports, 2024.

Fbi releases internet crime report, https : / /www . fbi . gov / contact - us / field -offices / sanfrancisco / news / fbi -releases-internet-crime-report, April

, 2024.

Cryptocurrency exchange says it was victim of 1.5billionhack, https : / /

apnews . com / article / bybit -

exchange - crypto - hack -88256366c723a9de8327ef3d4071057e,

February 22, 2025.

S. Sheikhi, “An effective fake news detection method using woa-xgbtree algorithm and contentbased features,” Elsevier, 2021. DOI: https://doi.org/10.1016/j.asoc.2021.107559.

S. Sheikhi and P. Kostakos, “Safeguarding cyberspace:Enhancing malicious website detection with pso–optimized xgboost and firefly-based feature

selection,” Computers Security, 2024. DOI:10.1016/j.cose.2024.103885.

M.-Y. Su and K.-L. Su, “Bert-based approaches to identifying malicious urls,” Sensors, vol. 23, p. 8499, 2023. DOI: 10.3390/s23208499. [Online]. Available: https://www.mdpi.com/journal/sensors.

M. Nanda, M. Saraswat, and P. K. Sharma, “Enhancing cybersecurity: A review and comparative analysis of convolutional neural network approaches

for detecting url-based phishing attacks,” Elsevier eprime, April 2024. DOI:10.1016/j.prime.2024 . 100533. [Online]. Available: https :/ / www . sciencedirect . com / science /article/pii/S2772671124001153.

Z. Wang, X. Ren, S. Li, B. Wang, J. Zhang, and T. Yang, “A malicious url detection model based on convolutional neural network,” Hindawi Security

and Communication Networks, vol. 2021, p. 12, 2021. DOI: 10.1155/2021/5518528.

B. B. Gupta, K. Yadav, I. Razzak, K. Psannis, A. Castiglione, and X. Chang, “A novel approach for phishing urls detection using lexical-based machine learning in a real-time environment,” Computer Communications, 2021. DOI:10.1016/j.comcom.2021.04.023.

F. Ullah, A. Alsirhani, M. M. Alshahrani, A. Alomari, H. Naeem, and S. A. Shah, “Explainable malware detection system using transformers-based transfer learning and multi-model visual representation,” Sensors, 2022. DOI: https://doi.org/10.3390/s22186766.

T. Wu, M. Wang, Y. Xi, and Z. Zhao, “Malicious url detection model based on bidirectional gated recurrent unit and attention mechanism,” applied Science MDPI, December 2022. DOI: 10 . 3390 /app122312367. [Online]. Available: https://www.mdpi.com/2076-3417/12/23/12367.

Z. Alshingiti, R. Alaqel, J. AlMuhtadi, Q. E. U. Haq, K. Saleem, and M. H. Faheem, “A deep learningbased phishing detection system using cnn,lstm, and lstm-cnn,” electronics MDPI, January 2023. DOI:10.3390/electronics12010232. [Online]. Available: https://www.mdpi.com/2079-9292/12/1/232.

R. Indu, M. Bhavya, V. Pardhasaradhi, Y. S. Ram, and Y. Suresh, “Malicious url detection,” International Journal of Creative Research Thoughts

(IJCRT), Apr. 2023, ISSN: 2320-2882. [Online]. Available: https :/ /ijcrt .org / papers /IJCRT2304563.pdf.

M. Alsaedi, F. A. Ghaleb, F. Saeed, J. Ahmad, and M. Alasli, “Cyber threat intelligence-based malicious url detection model using ensemble learning,”

Sensors, 2022. DOI: 10.3390/s22093373.

B. Yu, F. Tang, D. Ergu, R. Zeng, B. Ma, and F. Liu, “Efficient classification of malicious urls: M-bert—a modified bert variant for enhanced semantic understanding,” IEEE Access, 2024. DOI: 10 . 1109 /ACCESS . 2024 . 3357095. [Online]. Available: https : / / www . researchgate . net /

publication / 377618560 _ Efficient _Classification_of_Malicious_URLs_M - BERT_ - _A _ Modified _ BERT _Variant _ for _ Enhanced _ Semantic _

Understanding.

P. Maneriker, J. W. Stokes, E. G. Lazo, D. Carutasu, F. Tajaddodianfar, and A. Gururajan, “Urltran: Improving phishing url detection using transformers,” arXiv, 2021. DOI: https://arxiv.org/abs/2106.05256.

Malicious urls dataset, https://www.kaggle.com/datasets/sid321axn/maliciousurls-dataset, Accessed: 26 August 2023.

H. M. J. Khan, “A machine learning based web service for malicious url detection in a browser,” M.S. thesis, Electrical and Computer Engineering Department Hammond, Indiana, 2019.

K. Clark, U. Khandelwal, O. Levy, and C. D. Manning, “What does bert look at? an analysis of bert’s attention,” arXiv preprint arXiv:1906.04341, 2019. [Online]. Available: https : / / arxiv . org /abs/1906.04341.

G. Ke, Q. Meng, T. Finley, et al., “Lightgbm: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems, vol. 30, 2017. [Online]. Available:https : / / papers . neurips . cc /

paper _ files / paper / 2017 / hash /6449f44a102fde848669bdd9eb6b76fa -

Abstract.html.

J.-P. Lai, Y.-L. Lin, H.-C. Lin, C.-Y. Shih, Y.-P. Wang, and P.-F. Pai, “Tree-based machine learning models with optuna in predicting impedance values for circuit analysis,” Micromachines, vol. 14, no. 2, 2023, ISSN: 2072-666X. DOI: 10 . 3390 /mi14020265. [Online]. Available: https : / /www.mdpi.com/2072-666X/14/2/265.

S. Hanifi, A. Cammarono, and H. Zare-Behtash, “Advanced hyperparameter optimization of deep learning models for wind power prediction,” Renewable Energy, vol. 221, p. 119 700, 2024,

ISSN: 0960-1481. DOI: https : / / doi .org / 10 . 1016 / j . renene . 2023 .

[Online]. Available: https : / / www.sciencedirect.com/science/article/

pii/S0960148123016154.

T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” CoRR,

vol. abs/1907.10902, 2019. DOI: https : / /doi.org/10.4855/arXiv.1907.10902.

arXiv: 1907 . 10902. [Online]. Available:http://arxiv.org/abs/1907.10902.

D. A. Anggoro and S. S. Mukti, “Performance comparison of grid search and random search methods for hyperparameter tuning in extreme gradient boosting algorithm to predict chronic kidney failure.,”International Journal of Intelligent Engineering & Systems, vol. 14, no. 6, 2021. DOI:10.22266/ijies2021.1231.19. [Online]. Available: https://inass.org/wp-content/uploads/2021/10/2021123119.pdf.




DOI: https://doi.org/10.31449/inf.v49i27.9106

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.