With the development of unmanned aerial vehicle (UAV) technology, UAV swarm confrontation has attracted many researchers’ attention. However, the situation faced by the UAV swarm has substantial uncertainty and dynamic variability. The state space and action space increase exponentially with the number of UAVs, so that autonomous decision-making becomes a difficult problem in the confrontation environment. In this paper, a multiagent reinforcement learning method with macro action and human expertise is proposed for autonomous decision-making of UAVs. In the proposed approach, UAV swarm is modeled as a large multiagent system (MAS) with an individual UAV as an agent, and the sequential decision-making problem in swarm confrontation is modeled as a Markov decision process. Agents in the proposed method are trained based on the macro actions, where sparse and delayed rewards, large state space, and action space are effectively overcome. The key to the success of this method is the generation of the macro actions that allow the high-level policy to find a near-optimal solution. In this paper, we further leverage human expertise to design a set of good macro actions. Extensive empirical experiments in our constructed swarm confrontation environment show that our method performs better than the other algorithms.


    Access

    Download


    Export, share and cite



    Title :

    UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning


    Contributors:
    Baolai Wang (author) / Shengang Li (author) / Xianzhong Gao (author) / Tao Xie (author)


    Publication date :

    2021




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown






    Multiagent AGV dispatching system based on hierarchical reinforcement learning

    Li, X.-m. / Yang, Y.-p. / Xu, X.-m. | British Library Online Contents | 2002


    2v2 Air Combat Confrontation Strategy Based on Reinforcement Learning

    Wang, Jinlin / Zhu, Longtao / Yang, Hongyu et al. | Springer Verlag | 2023


    Autonomous Bus Fleet Control Using Multiagent Reinforcement Learning

    Sung-Jung Wang / S. K. Jason Chang | DOAJ | 2021

    Free access