In response to the challenges of estimating missile launch timing during close-range unmanned autonomous air combat in the future, this article proposes an autonomous attack decision-making method based on hierarchical virtual Bayesian reinforcement learning (HVBRL). First, a six-degree-of-freedom (6-DOF) high-fidelity aircraft dynamics model, along with missile dynamics and guidance rate models, is constructed. Second, the HVBRL algorithm is introduced, where the low-level algorithm outputs control parameters and the high-level algorithm generates control commands. Given that the number of missile hits on a target under specific conditions follows a binomial distribution, a simple prior knowledge can be introduced through its conjugate prior, the Beta distribution, to avoid prolonged exploration of ineffective areas. Moreover, carrying only a limited number of missiles and predicting the number of hits by multiple virtual missiles in specific states through a neural network circumvent the computational complexity issue associated with carrying an excessive number of missiles. Finally, this article presents the low-level training algorithm, the high-level training algorithm, and the high-level self-play training algorithm. Experimental results show that our method significantly reduces the simulation computational complexity. Compared with the Monte Carlo method carrying 1000 missiles, the simulation speed of the high-level training algorithm is increased by 32.75 times, and that of the high-level self-play algorithm is increased by 23 times. Moreover, the estimated missile hit probability with bias can effectively guide the timing of missile launches in close-range air combat, which has significant implications for intelligent autonomous air combat decision-making and operational analysis.


    Access

    Download


    Export, share and cite



    Title :

    An Autonomous Attack Decision-Making Method Based on Hierarchical Virtual Bayesian Reinforcement Learning


    Contributors:


    Publication date :

    2024-10-01


    Size :

    2888181 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    AUTONOMOUS DECISION-MAKING METHOD OF LEARNING MAIN BODY BASED ON DEEP REINFORCEMENT LEARNING

    LIU DONGSHENG / LIU YANNI / WANG LIMING et al. | European Patent Office | 2024

    Free access

    A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning

    Zhang, Mingheng / Wan, Xing / Lv, Xinfei et al. | British Library Conference Proceedings | 2020


    Autonomous Decision-Making of Drone Swarm Based on Deep Reinforcement Learning

    Zhang, Na / Chen, Shuhan / Xiong, Shixun et al. | Springer Verlag | 2025


    A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning

    Zhang, Mingheng / Wan, Xing / Lv, Xinfei et al. | British Library Conference Proceedings | 2020