FD3QN: A Federated Deep Reinforcement Learning Approach for Cross-Domain Resource Cooperative Scheduling in Hybrid Cloud Architecture

Abstract

To address the challenge of insufficient computing power in multi-access edge computing (MEC) servers caused by highly dynamic service requests and uneven service distribution in vehicular networks, this paper proposes a hybrid multi-server MEC architecture that leverages both fixed road-side units (RSUs) and mobile unmanned aerial vehicles (UAVs). We introduce the FD3QN algorithm, which integrates federated learning and deep reinforcement learning, to minimize the weighted sum of service latency and energy consumption. Specifically, the MATD3 algorithm is employed for safe and efficient UAV trajectory planning in the offloading decision process. For resource allocation, we embed vertical federated learning into the D3QN network to enable cross-domain resource cooperative scheduling. A decentralized federated aggregation framework is utilized to maintain a global model for optimizing resource allocation in a collaborative and privacy-preserving manner. The proposed algorithm jointly optimizes transmission power, computing, and storage resources. Extensive simulations are conducted to evaluate the performance of FD3QN in a realistic vehicular network environment with varying numbers of vehicles and task arrivals. The results demonstrate that FD3QN outperforms benchmark algorithms, achieving an 11.37% and 12.06% reduction in system cost compared to the FDDQN algorithm in scenarios with 8 and 12 vehicles, respectively. Moreover, FD3QN exhibits a 25% decrease in average service latency and a 15% improvement in energy efficiency compared to traditional deep reinforcement learning approaches. The proposed algorithm also maintains a high task completion rate of over 98% under dynamic network conditions. These findings validate the strong model generalization ability of FD3QN in the dynamic vehicular networking environment and highlight its practicality for real-world deployment. This study provides novel insights into the development of intelligent transportation systems and edge computing paradigms.