Multi-agent Control of UAVs Using Deep Reinforcement Learning
DOI:
https://doi.org/10.18372/1990-5548.84.20187Keywords:
unmanned aerial vehicle swarm, deep reinforcement learning, multi-agent systems, multi-agent deep reinforcement learning, drone coordination, centralized training with decentralized execution, obstacle avoidanceAbstract
This paper presents a novel control framework for managing a group of unmanned aerial vehicles using multi-agent deep reinforcement learning. The approach leverages actor–critic architectures, centralized training with decentralized execution, and shared experience replay to enable autonomous coordination in dynamic environments. Simulation results confirm improved tracking accuracy, reduced collision rates, and increased coverage efficiency. The study also compares the proposed system against baseline methods and outlines future work for real-world adaptation. The novelty lies in applying multi-agent deep reinforcement learning to a continuous unmanned aerial vehicle control task in cluttered environments with limited sensing.
References
A. Hussain, H. Khan, S. Nazir, I. Ullah, and T. Hussain, “Taking FANET to Next Level: The Contrast Evaluation of Moth-and-Ant with Bee Ad-hoc Routing Protocols for Flying Ad-hoc Networks,” ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, vol. 10, pp. 321–337, 2022. https://doi.org/10.14201/ADCAIJ2021104321337.
F. L. Lewis, D. Vrabie, and V. L. Syrmos, Optimal Control, Wiley, 2012, 552 p. ttps://doi.org/10.1002/9781118122631
H. X. Pham, H. M. La, D. Feil-Seifer, and A. Nefian, “Cooperative and Distributed Reinforcement Learning of Drones for Field Coverage,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), 2018, pp. 1–6. https://doi.org/10.1109/SSRR.2018.8468611
D. D. Fan, E. Theodorou, J. Reeder, “Model-Based Stochastic Search for Large Scale Optimization of Multi-Agent UAV Swarms,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), 2018, pp. 1–6. https://doi.org/10.1109/SSCI.2018.8628677
D. Baldazo, J. Parras, and S. Zazo, “Decentralized Multi-Agent Deep Reinforcement Learning in Swarms of Drones for Flood Monitoring,” Proc. of the 27th European Signal Processing Conf. (EUSIPCO), 2019. https://doi.org/10.23919/EUSIPCO.2019.8903067
G. Venturini, F. Mason, F. Pase, A. Testolin, A. Zanella, and M. Zorzi, “Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control,” Proc. Cognitive IoT Tech. Conf., 2020, pp. 1–6. https://doi.org/10.1145/3396864.3399701
A. Shamsoshoara, M. Khaledi, F. Afghah, A. Razi, J. Ashdown, “Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning,” Proc. Int. Conf. on Computing, Networking and Communications (ICNC), 2018, pp. 124–129.
K. Gundy-Burlet, “Neural Flight Control System,” NASA Technical Memorandum, NASA Ames Research Center, 2003, TM-2003-212408.
M. P. Musiyenko and I. M. Zhuravska, “Route Planning Algorithms for UAVs Using Hopfield Neural Networks,” Visnyk Cherkaskoho Derzhavnoho Tekhnolohichnoho Universytetu, no. 1, pp. 20–27, 2016.
N. Thumiger and M. Deghat, “A Multi-Agent Deep Reinforcement Learning Approach for Practical Decentralized UAV Collision Avoidance, IEEE Control Systems Letters, vol. 6, 2021, pp. 1–5. https://doi.org/10.1109/LCSYS.2021.3138941
S. Batra, Z. Huang, A. Petrenko, et al. “Decentralized Control of Quadrotor Swarms with End-to-End Deep Reinforcement Learning, Proc. of the 5th Conf. on Robot Learning (CoRL), 2022, pp. 576–586.
J. Kocic, N. Jovicic, V. Drndarevic, “An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms,” Sensors, vol. 19(2), pp. 1–19. 2019. https://doi.org/10.3390/s19092064
A. Hussain, H. Khan, S. Nazir, I. Ullah, and T. Hussain, “Taking FANET to Next Level: The Contrast Evaluation of Moth-and-Ant with Bee Ad-hoc Routing Protocols for Flying Ad-hoc Networks,” ADCAIJ, vol. 10, pp. 321–337, 2022. https://doi.org/10.14201/ADCAIJ2021104321337
J. Z. Chang, “Training Neural Networks to Pilot Autonomous Vehicles: Scaled Self-Driving Car,” Senior Projects Spring, no. 402, 2018.
A. A. Khalil, A. J. Byrne, M. A. Rahman, end M. H. Manshaei, “Efficient UAV Trajectory-Planning Using Economic Reinforcement Learning,” Proc. Int. Conf. on Advanced Information Networking and Applications (AINA), 2020, pp. 233–243.
L. Bellone, B. Galkin, E. Traversi, end E. Natalizio, “Deep Reinforcement Learning for Combined Coverage and Resource Allocation in UAV-aided RAN-slicing,” IEEE Trans. on Mobile Computing, 2022, pp. 1–10. https://doi.org/10.1109/DCOSS-IoT58021.2023.00106
Sarder Fakhrul Abedin, Md. Shirajum Munir, Nguyen H. Tran, Zhu Han, and Choong Seon Hong, “Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach,” IEEE Internet of Things Journal, vol. 8(6), pp. 4514–4529. 2021. https://doi.org/10.48550/arXiv.2003.04816
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
