Aiming at the problem that unmanned combat aerial vehicle (UCAV) is difficult to quickly and accurately perceive situation information and make maneuvering decision autonomously in modern air combat, which is easily affected by complex factors, a maneuvering decision algorithm of UCAV combined with deep reinforcement learning and game theory is proposed in this paper. Firstly, through the UCAV dynamics model and maneuver library, a reasonable air combat situation assessment model and advantage reward function are established, and the sample data of situation assessment indicators are constructed using the structure entropy weight method. Secondly, the convolutional neural network (CNN) is used to process the high-dimensional continuous situation features of UCAV in air combat, eliminate the correlation and redundancy between situation features, and train the neural network to approximate the action-value function. Then, the double deep Q network (DDQN) algorithm in reinforcement learning (RL) is introduced to train the agent by the interaction with the environment and combined with Minimax algorithm in stochastic game theory to solve the optimal value function in each specific state, and the optimal maneuver decision of UCAV is obtained. Air combat simulation results show that UCAV can choose maneuvers autonomously under different situations and occupy a dominant position quickly by this method, which greatly improves the combat effectiveness of UCAV.


    Access

    Download


    Export, share and cite



    Title :

    Autonomous Maneuver Decision of UCAV Air Combat Based on Double Deep Q Network Algorithm and Stochastic Game Theory


    Contributors:
    Yuan Cao (author) / Ying-Xin Kou (author) / Zhan-Wu Li (author) / An Xu (author)


    Publication date :

    2023




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    An Air Combat UCAV Autonomous Maneuver Decision Method Based on LSTM Network and MCDTS

    Dang, Fangyuan / Zhu, Huaguang / Ning, Xin | Springer Verlag | 2023


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | British Library Conference Proceedings | 2022


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | Springer Verlag | 2022


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | TIBKAT | 2022


    Maneuver Strategy Generation of Multi-UCAV for Air Combat Based on Fuzzy Theory

    Zhang, Xiaoyan / Huang, Hanqiao / Zhang, Di et al. | Springer Verlag | 2022