Multi-agent decision-making faces many challenges such as non-stationarity and sparse rewards, while the complexity and randomness of the real environment further complicate policy development. This paper addresses the high-dimensional policy optimization problems of unmanned aerial vehicle (UAV) swarms. By modeling the problem scenario as a Markov decision process, a real-time policy optimization algorithm based on evolution strategy (ES) pre-training is proposed. This approach combines decision-time planning with background planning to evaluate and integrate different sets of policy parameters in a temporal context. In the experimental phase, the policy network is trained using both ES and REINFORCE algorithms on a constructed simulation platform. Comparative experiments demonstrate the effectiveness of using ES for policy pre-training. Finally, the proposed real-time policy optimization algorithm further improves the performance of the swarm by approximately 10% in simulations, offering a feasible solution for adversarial games between swarms and extending the research scope of evolutionary algorithms.


    Access

    Download


    Export, share and cite



    Title :

    Real-Time Policy Optimization for UAV Swarms Based on Evolution Strategies


    Contributors:
    Zeyu Chen (author) / Haiying Liu (author) / Guohua Liu (author)


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Atmospheric Sampling Strategies with UAS Swarms

    Cooper, Jared K. / De Wekker, Stephan / Bateman, Alec J. | AIAA | 2024


    Multizone accretional evolution of planetesimal swarms

    Spaute, D. / Davis, D. R. / Weidenschilling, S. J. | NTRS | 1990


    Multizone accretional evolution of planetesimal swarms

    Spaute, D. / Davis, D.R. / Weidenschilling, S.J. | Elsevier | 1989



    Cultural evolution of probabilistic aggregation in synthetic swarms

    Cambier, Nicolas / Albani, Dario / Frémont, Vincent et al. | BASE | 2021

    Free access