In response to the challenges of estimating missile launch timing during close-range unmanned autonomous air combat in the future, this article proposes an autonomous attack decision-making method based on hierarchical virtual Bayesian reinforcement learning (HVBRL). First, a six-degree-of-freedom (6-DOF) high-fidelity aircraft dynamics model, along with missile dynamics and guidance rate models, is constructed. Second, the HVBRL algorithm is introduced, where the low-level algorithm outputs control parameters and the high-level algorithm generates control commands. Given that the number of missile hits on a target under specific conditions follows a binomial distribution, a simple prior knowledge can be introduced through its conjugate prior, the Beta distribution, to avoid prolonged exploration of ineffective areas. Moreover, carrying only a limited number of missiles and predicting the number of hits by multiple virtual missiles in specific states through a neural network circumvent the computational complexity issue associated with carrying an excessive number of missiles. Finally, this article presents the low-level training algorithm, the high-level training algorithm, and the high-level self-play training algorithm. Experimental results show that our method significantly reduces the simulation computational complexity. Compared with the Monte Carlo method carrying 1000 missiles, the simulation speed of the high-level training algorithm is increased by 32.75 times, and that of the high-level self-play algorithm is increased by 23 times. Moreover, the estimated missile hit probability with bias can effectively guide the timing of missile launches in close-range air combat, which has significant implications for intelligent autonomous air combat decision-making and operational analysis.
An Autonomous Attack Decision-Making Method Based on Hierarchical Virtual Bayesian Reinforcement Learning
IEEE Transactions on Aerospace and Electronic Systems ; 60 , 5 ; 7075-7088
2024-10-01
2888181 byte
Article (Journal)
Electronic Resource
English
AUTONOMOUS DECISION-MAKING METHOD OF LEARNING MAIN BODY BASED ON DEEP REINFORCEMENT LEARNING
European Patent Office | 2024
|A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning
British Library Conference Proceedings | 2020
|Autonomous Decision-Making of Drone Swarm Based on Deep Reinforcement Learning
Springer Verlag | 2025
|A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning
British Library Conference Proceedings | 2020
|