Compared with terrestrial networks, unmanned aerial vehicles (UAVs) have the characteristics of flexible deployment and strong adaptability, which are an important supplement to intelligent transportation systems (ITS). In this paper, we focus on the multi-UAV network area coverage problem (ACP) which require intelligent UAVs long-term trajectory decisions in the complex and scalable network environment. Multi-agent deep reinforcement learning (DRL) has recently emerged as an effective tool for solving long-term decisions problems. However, since the input dimension of multi-layer perceptron (MLP)-based deep neural network (DNN) is fixed, it is difficult for standard DNN to adapt to a variable number of UAVs and network users. Therefore, we combine Transformer with DRL to meet the scalability of the network and propose a Transformer-based deep multi-agent reinforcement learning (T-MARL) algorithm. Transformer can adapt to variable input dimensions and extract important information from complex network states by attention module. In our research, we find that random initialization of Transformer may cause DRL training failure, so we propose a baseline-assisted pre-training scheme. This scheme can quickly provide an initial policy model for UAVs based on imitation learning, and use the temporal-difference(1) algorithm to initialize policy evaluation network. Finally, based on parameter sharing, T-MARL is applicable to any standard DRL algorithm and supports expansion on networks of different sizes. Experimental results show that T-MARL can make UAVs have cooperative behaviors and perform outstandingly on ACP.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Transformer-Based Reinforcement Learning for Scalable Multi-UAV Area Coverage


    Contributors:
    Chen, Dezhi (author) / Qi, Qi (author) / Fu, Qianlong (author) / Wang, Jingyu (author) / Liao, Jianxin (author) / Han, Zhu (author)


    Publication date :

    2024-08-01


    Size :

    16711991 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    Disaster Area Coverage Optimisation Using Reinforcement Learning

    Gruffeille, Ciaran / Perrusquia, Adolfo / Tsourdos, Antonios et al. | IEEE | 2024


    Design of Reinforcement Learning Environment for Multiple UAV Area Coverage Search

    Sun, Yinjiang / Zhang, Rui / Liu, Xiaoheng et al. | Springer Verlag | 2022


    Design of Reinforcement Learning Environment for Multiple UAV Area Coverage Search

    Sun, Yinjiang / Zhang, Rui / Liu, Xiaoheng et al. | TIBKAT | 2022