The invention belongs to the technical field of unmanned aerial vehicle control, and particularly relates to an unmanned aerial vehicle charging scheduling strategy based on deep reinforcement learning. The charging recommendation problem of the unmanned aerial vehicles needs to consider the potential resource competition between the unmanned aerial vehicles and the occupation condition of the charging pile in the charging process. MAPPO is an algorithm for multi-agent reinforcement learning, shows excellent performance on some models, but also has certain limitation, namely, the stability of the algorithm is relatively weak, and the algorithm is easy to fall into local optimum. In order to solve the problems, the invention proposes that a Traley PPO algorithm is applied to multi-agent reinforcement learning, the defects of a multi-agent near-end optimization algorithm MAPPO are overcome, meanwhile, an attention level system is introduced into a centralized Critic network, the optimization direction is adaptively guided through a dynamic gradient weighting strategy, and the problem that the state action space is too large is avoided. According to the method provided by the invention, the unmanned aerial vehicle can be efficiently distributed to the most suitable charging station.

    本发明属于无人机控制技术领域,特别涉及一种基于基于深度强化学习的无人机充电调度策略。无人机充电推荐问题需要考虑无人机之间潜在的资源竞争及充电过程中对充电桩占用情况。MAPPO是多智能体强化学习的一种算法,在一些模型上表现出了优异的性能,但它也有一定的局限性,即算法的稳定性较弱,也容易陷入局部最优。针对这些问题,本发明提出了将Truly PPO算法应用于多智能体强化学习,解决了多智能体近端优化算法MAPPO的缺点,同时将注意力级制引入到集中式Critic网络中,通过动态梯度加权策略自适应引导优化方向,避免了状态作用空间过大的问题。本发明提出的方法,可以将无人机高效的分配到最合适的充电站。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Unmanned aerial vehicle charging scheduling method based on deep reinforcement learning


    Weitere Titelangaben:

    一种基于深度强化学习的无人机充电调度方法


    Beteiligte:
    ZHAI XIANGPING (Autor:in) / JIN JING (Autor:in) / LI YURAN (Autor:in)

    Erscheinungsdatum :

    11.04.2025


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    B64U / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06Q Datenverarbeitungssysteme oder -verfahren, besonders angepasst an verwaltungstechnische, geschäftliche, finanzielle oder betriebswirtschaftliche Zwecke, sowie an geschäftsbezogene Überwachungs- oder Voraussagezwecke , DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES / G05D SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES , Systeme zum Steuern oder Regeln nichtelektrischer veränderlicher Größen





    CHARGING UNMANNED AERIAL VEHICLE, FUNCTIONAL UNMANNED AERIAL VEHICLE AND AIR-CHARGING UNMANNED AERIAL VEHICLE SET THEREOF

    LIANG CHEN-YI / CHANG YUNG CHEN / CHANG SHIH-YUAN | Europäisches Patentamt | 2019

    Freier Zugriff

    Charging unmanned aerial vehicle, functional unmanned aerial vehicle and air-charging unmanned aerial vehicle set thereof

    LIANG CHEN-YI / CHANG YUNG CHEN / CHANG SHIH-YUAN | Europäisches Patentamt | 2019

    Freier Zugriff