Dynamic Routing via Reinforcement Learning for Network Traffic Optimization

Jian Ma, Chaoyong Zhu, Yuntao Fu, Haichao Zhang, Wenjing Xiong

Abstract


With the rapid development of the Internet, network traffic has shown explosive growth, which puts forward higher requirements for the network routing system. Traditional static routing methods are no longer able to meet the needs of today's complex and ever-changing network environment, as they cannot be flexibly adjusted according to real-time network conditions. In order to address this challenge, this paper proposes an innovative dynamic routing method. This method is based on reinforcement learning, especially Q-learning algorithm, which realizes the dynamic adjustment of routing decisions through continuous learning and adaptation to changes in the network environment. Our goal is to minimize root mean square error (RMSE) to improve routing accuracy, while at the same time improving load balancing efficiency to ensure that network resources are fully utilized. In order to verify the effectiveness of this method, we conducted detailed simulation experiments. Experimental results show that compared with the baseline method, our dynamic routing method significantly improves the throughput of the network, which increases by 30%, effectively reduces the delay, and reduces 25%. These positive results not only prove the effectiveness of our method in network traffic optimization, but also provide new ideas for the development of network routing system in the future

Full Text:

PDF

References


Li, Y., Song, L., Chen, Y., Li, Z., Zhang, X., Wang, X., & Sun, J. (2020). Learning dynamic routing for semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8553-8562.

Rios, B. H. O., Xavier, E. C., Miyazawa, F. K., Amorim, P., Curcio, E., & Santos, M. J. (2021). Recent dynamic vehicle routing problems: A survey. Computers & Industrial Engineering, 160, 107604.

Li, C., Wang, G., Wang, B., Liang, X., Li, Z., & Chang, X. (2021). Dynamic slimmable network. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 8607-8617.

Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y. (2021). Dynamic neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 7436-7456.

Mor, A., & Speranza, M. G. (2022). Vehicle routing problems over time: a survey. Annals of Operations Research, 314(1), 255-275.

Srilakshmi, U., Veeraiah, N., Alotaibi, Y., Alghamdi, S. A., Khalaf, O. I., & Subbayamma, B. V. (2021). An improved hybrid secure multipath routing protocol for MANET. IEEE Access, 9, 163043-163053.

Fu, X., Fortino, G., Pace, P., Aloi, G., & Li, W. (2020). Environment-fusion multipath routing protocol for wireless sensor networks. Information Fusion, 53, 4-19.

Mittal, A., Kumar, D., Mittal, M., Saba, T., Abunadi, I., Rehman, A., & Roy, S. (2020). Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors, 20(4), 1068.

Bhardwaj, A., & El-Ocla, H. (2020). Multipath routing protocol using genetic algorithm in mobile ad hoc networks. IEEE Access, 8, 177534-177548.

Luo, J., Chen, Y., Wu, M., & Yang, Y. (2021). A survey of routing protocols for underwater wireless sensor networks. IEEE Communications Surveys & Tutorials, 23(1), 137-160.

Rani, S., Ahmed, S. H., & Rastogi, R. (2020). Dynamic clustering approach based on wireless sensor networks genetic algorithm for IoT applications. Wireless Networks, 26(4), 2307-2316.

Zhao, D., Li, Y., Zeng, Y., Wang, J., & Zhang, Q. (2022). Spiking capsnet: A spiking neural network with a biologically plausible routing rule between capsules. Information Sciences, 610, 1-13.

Khudayer, B. H., Anbar, M., Hanshi, S. M., & Wan, T. C. (2020). Efficient route discovery and link failure detection mechanisms for source routing protocol in mobile ad-hoc networks. IEEE Access, 8, 24019-24032.

Zhou, X., Yang, X., Ma, J., Kevin, I., & Wang, K. (2021). Energy-efficient smart routing based on link correlation mining for wireless edge computing in IoT. IEEE Internet of Things Journal, 9(16), 14988-14997.

Lakew, D. S., Sa’ad, U., Dao, N. N., Na, W., & Cho, S. (2020). Routing in flying ad hoc networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 22(2), 1071-1120.

Hong, L., Guo, H., Liu, J., & Zhang, Y. (2020). Toward swarm coordination: Topology-aware inter-UAV routing optimization. IEEE Transactions on Vehicular Technology, 69(9), 10177-10187.

Khan, I. U., Qureshi, I. M., Aziz, M. A., Cheema, T. A., & Shah, S. B. H. (2020). Smart IoT control-based nature inspired energy efficient routing protocol for flying ad hoc network (FANET). IEEE Access, 8, 56371-56378.

Zis, T. P., Psaraftis, H. N., & Ding, L. (2020). Ship weather routing: A taxonomy and survey. Ocean Engineering, 213, 107697.

Daanoune, I., Abdennaceur, B., & Ballouk, A. (2021). A comprehensive survey on LEACH-based clustering routing protocols in Wireless Sensor Networks. Ad Hoc Networks, 114, 102409.

Gao, H., Liu, C., Li, Y., & Yang, X. (2020). V2VR: reliable hybrid-network-oriented V2V data transmission and routing considering RSUs and connectivity probability. IEEE Transactions on Intelligent Transportation Systems, 22(6), 3533-3546.

Sun, Z., Zhao, G., Scherer, R., Wei, W., & Woźniak, M. (2022). Overview of capsule neural networks. Journal of Internet Technology, 23(1), 33-44.

Deebak, B. D., & Al-Turjman, F. (2020). A hybrid secure routing and monitoring mechanism in IoT-based wireless sensor networks. Ad Hoc Networks, 97, 102022.

Xin, L., Song, W., Cao, Z., & Zhang, J. (2021). Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(13), 12042-12049).

Zhu, L., Wang, X., Ke, Z., Zhang, W., & Lau, R. W. (2023). Biformer: Vision transformer with bi-level routing attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10323-10333.

Chen, X., Tang, J., & Lao, S. (2020). Review of unmanned aerial vehicle swarm communication architectures and routing protocols. Applied Sciences, 10(10), 3661.

Zhang, K., He, F., Zhang, Z., Lin, X., & Li, M. (2020). Multi-vehicle routing problems with soft time windows: A multi-agent reinforcement learning approach. Transportation Research Part C: Emerging Technologies, 121, 102861.

Kim, J., Jang, S., Park, E., & Choi, S. (2020). Text classification using capsules. Neurocomputing, 376, 214-221.

Wu, H., Alay, Ö., Brunstrom, A., Ferlin, S., & Caso, G. (2020). Peekaboo: Learning-based multipath scheduling for dynamic heterogeneous environments. IEEE Journal on Selected Areas in Communications, 38(10), 2295-2310.

Pessoa, A., Sadykov, R., Uchoa, E., & Vanderbeck, F. (2020). A generic exact solver for vehicle routing and related problems. Mathematical Programming, 183, 483-523.

Sumathi, J., & Velusamy, R. L. (2021). A review on distributed cluster based routing approaches in mobile wireless sensor networks. Journal of Ambient Intelligence and Humanized Computing, 12(1), 835-849.




DOI: https://doi.org/10.31449/inf.v49i8.7126

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.