Compared with terrestrial networks, unmanned aerial vehicles (UAVs) have the characteristics of flexible deployment and strong adaptability, which are an important supplement to intelligent transportation systems (ITS). In this paper, we focus on the multi-UAV network area coverage problem (ACP) which require intelligent UAVs long-term trajectory decisions in the complex and scalable network environment. Multi-agent deep reinforcement learning (DRL) has recently emerged as an effective tool for solving long-term decisions problems. However, since the input dimension of multi-layer perceptron (MLP)-based deep neural network (DNN) is fixed, it is difficult for standard DNN to adapt to a variable number of UAVs and network users. Therefore, we combine Transformer with DRL to meet the scalability of the network and propose a Transformer-based deep multi-agent reinforcement learning (T-MARL) algorithm. Transformer can adapt to variable input dimensions and extract important information from complex network states by attention module. In our research, we find that random initialization of Transformer may cause DRL training failure, so we propose a baseline-assisted pre-training scheme. This scheme can quickly provide an initial policy model for UAVs based on imitation learning, and use the temporal-difference(1) algorithm to initialize policy evaluation network. Finally, based on parameter sharing, T-MARL is applicable to any standard DRL algorithm and supports expansion on networks of different sizes. Experimental results show that T-MARL can make UAVs have cooperative behaviors and perform outstandingly on ACP.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Transformer-Based Reinforcement Learning for Scalable Multi-UAV Area Coverage


    Beteiligte:
    Chen, Dezhi (Autor:in) / Qi, Qi (Autor:in) / Fu, Qianlong (Autor:in) / Wang, Jingyu (Autor:in) / Liao, Jianxin (Autor:in) / Han, Zhu (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.08.2024


    Format / Umfang :

    16711991 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Multi-Agent Dynamic Area Coverage Based on Reinforcement Learning with Connected Agents

    ÇETİN, AYDIN / Aydemir, Fatih | BASE | 2023

    Freier Zugriff

    Disaster Area Coverage Optimisation Using Reinforcement Learning

    Gruffeille, Ciaran / Perrusquia, Adolfo / Tsourdos, Antonios et al. | IEEE | 2024


    Design of Reinforcement Learning Environment for Multiple UAV Area Coverage Search

    Sun, Yinjiang / Zhang, Rui / Liu, Xiaoheng et al. | Springer Verlag | 2022


    Design of Reinforcement Learning Environment for Multiple UAV Area Coverage Search

    Sun, Yinjiang / Zhang, Rui / Liu, Xiaoheng et al. | TIBKAT | 2022