Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges

Wenchao He

doi:10.47297/taposatWSP2633-456915.20260706

PDF
导出
分享
收藏
专辑

Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges

7卷第6期

浏览量: 0 Downloads: 0

Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges
Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges
School of Intelligent Engineering, Yanching Institute of Technology, Langfang, Hebei, 065000, China
2026年7卷第6期页码：69-74
网络首发：2026-08-18，

纸质出版：2026-06-30
DOI：10.47297/taposatWSP2633-456915.20260706
Accepted：

Scan QR Code

Wenchao He. (2026). Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges. 科技理论与实践（英文版）, 7(6), 69-74.

He Wenchao.Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges[J].Theory and Practice of Science and Technology,2026,07(06):69-74.
Wenchao He. (2026). Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges. 科技理论与实践（英文版）, 7(6), 69-74. DOI： 10.47297/taposatWSP2633-456915.20260706.

He Wenchao.Multi-Agent Deep Reinforcement Learning in UAV Swarm Formation Control: A Review and Future Challenges[J].Theory and Practice of Science and Technology,2026,07(06):69-74. DOI： 10.47297/taposatWSP2633-456915.20260706.

摘要

Abstract

Due to constraints on payload and detection range

a UAV struggles to perform complex dynamic tasks

making swarm coordination a mainstream area of research. Traditional formation control relies on precise dynamic modeling

which limits its adaptability in dynamic environments. Deep Reinforcement Learning (DRL) and Multi-Agent Reinforcement Learning (MARL) possess model-free self-learning capabilities

offering new avenues for intelligent swarm formation.This paper systematically traces the evolution of formation control from traditional model-driven approaches and single-agent DRL to MARL. It compares the applicable scenarios and performance differences among mainstream MARL frameworks (MAPPO

MADDPG

DQN

DDPG

and Dec-MDP) and summarizes progress in integrating MARL with traditional methods such as leader-follower

consistency

and artificial potential fields. It analyzes current core challenges

such as slow convergence

difficulties in scaling

insufficient generalization from simulation to real-world environments

and limitations in on-board deployment

and outlines future trends from the perspectives of engineering implementation

heterogeneous swarms

robustness in complex environments

and algorithm lightweighting. This paper serves as a reference for the algorithmic design and engineering applications of drone swarm formation control.

关键词

Keywords

references

Zong, Q., Wang, D., Shao, S., et al. (2017) Multi-UAV cooperative formation flight control: Current status and development. J. Harbin Inst. Technol. , 49(3): 1–14.

Desai, J.P., Ostrowski, J.P., Kumar, V. (2001) Modeling and control of formations of nonholonomic mobile robots. IEEE Trans. Robot. Autom. , 17(6): 905–908.

Olfati-Saber, R. (2006) Flocking for multi-agent dynamic systems: Algorithms and theory. IEEE Trans. Autom. Control , 51(3): 401–420.

Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015) Human-level control through deep reinforcement learning. Nature , 518(7540): 529–533.

Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al. (2020) Continuous control with deep reinforcement learning. U.S. Patent 10,776,692.

Lowe, R., Wu, Y.I., Tamar, A., et al. (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems. Long Beach, CA. pp. 6379–6390.

Reynolds, C.W. (1987) Flocks, herds and schools: A distributed behavioral model. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques . New York. pp. 25–34.

Lewis, M.A., Tan, K.H. (1997) High precision formation control of mobile robots using virtual structures. Auton. Robots , 4(4): 387–403.

Schulman, J., Wolski, F., Dhariwal, P., et al. (2017) Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347 .

Yu, C., Velu, A., Vinitsky, E., et al. (2022) The surprising effectiveness of PPO in cooperative multi-agent games. In: Advances in Neural Information Processing Systems, Vol. 35 . New Orleans. pp. 24611–24624.

Sun, S., Jin, B., Yu, H., et al. (2025) Graph neural network enhanced multi-agent reinforcement learning for UAV formation control. In: 2025 3rd International Conference on Algorithm, Image Processing and Machine Vision (AIPMV) . Singapore. pp. 1–6.

Sheng, Y., Xie, X., Liu, H., et al. (2026) Dynamic target assignment and cooperative decision-making for UAV swarms based on multi-agent reinforcement learning. IEEE Internet Things J. , 13(8): 12345–12360.

Zhang, Y., Wu, Z., Ma, Y., et al. (2022) Research on autonomous formation of multi-UAV based on MADDPG algorithm. In: 2022 IEEE 17th International Conference on Control & Automation (ICCA) . Naples. pp. 249–254.

Wu, J., Li, D., Yu, Y., et al. (2024) An attention mechanism and adaptive accuracy triple-dependent MADDPG formation control method for hybrid UAVs. IEEE Trans. Intell. Transp. Syst. , 25(9): 11858–11870.

Xing, X., Zhou, Z., Li, Y., et al. (2024) Multi-UAV adaptive cooperative formation trajectory planning based on an improved MATD3 algorithm of deep reinforcement learning. IEEE Trans. Veh. Technol. , 73(9): 12484–12499.

Xu, Z., Wang, Q., Kong, F., et al. (2022) Ga-DQN: A gravity-aware DQN based UAV path planning algorithm. In: 2022 IEEE International Conference on Unmanned Systems (ICUS) . Guangzhou. pp. 1215–1220.

Salimi, M., Pasquier, P. (2021) Deep reinforcement learning for flocking control of UAVs in complex environments. In: 2021 6th International Conference on Robotics and Automation Engineering (ICRAE) . Sin gapore. pp. 344–352.

Azam, M.A., Mittelmann, H.D., Ragi, S. (2021) UAV formation shape control via decentralized Markov decision processes. Algorithms , 14(3): 91.

文章被引用时，请邮件提醒。

Submit

A Review of the Integrated Application of Sensor Technology in Safety Control Systems for New Energy Vehicles

A Review of the Applications and Innovative Practices of Artificial Intelligence Algorithms in Gaming

Review of Combined Braking Systems for Electric Vehicles

Research on Perception and Decision-Making Methods for Autonomous Driving

Research on intelligent maintenance technology of eindustrial robot claw based on Internet of Things and deep reinforcement learning

Related Author

Yanan Li School of Electronic and Information Engineering, Taiyuan University of Science and Technology

Yihan Liu Anhui University

Boyu Wang Jinan Xinhang Experimental Foreign Language School

Qinwen Qiu School of Management and engineering, Nanjing University, Industrial Intelligence and System Integration

Jundan Wang University of Science and Technology Liaoning

Aobo Zhang University of Science and Technology Liaoning

Jikai Xin University of Science and Technology Liaoning

Baowei Yu University of Science and Technology Liaoning

Related Institution

School of Electronic and Information Engineering, Taiyuan University of Science and Technology

Anhui University

Jinan Xinhang Experimental Foreign Language School

School of Management and engineering, Nanjing University, Industrial Intelligence and System Integration

University of Science and Technology Liaoning

⁰