The invention relates to an automatic driving decision-making method based on off-line and on-line fusion reinforcement learning, which relates to the field of automatic driving and artificial intelligence and comprises two stages of off-line training and on-line strategy expansion. And initializing offline and online reinforcement learning strategies and an offline value network. In the off-line training stage, data in the static data set are put into an off-line experience pool, static data are obtained through random sampling from the off-line experience pool, and an off-line strategy and an off-line value network are updated. In the online strategy expansion stage, firstly freezing the offline strategies obtained in the offline training stage, combining the offline strategies with the online strategies to form an expansion strategy set, selecting strategies from the expansion strategy set according to a certain probability to interact with the environment, and putting interaction data into an online experience pool; and randomly sampling from the online experience pool to obtain static data, and updating an online strategy and an online value network. According to the method, the decision-making agent can benefit from further online learning while the safety of offline reinforcement learning is kept.

    本发明涉及一种基于离线在线融合强化学习的自动驾驶决策方法,涉及自动驾驶和人工智能领域,包括离线训练和在线策略扩展两个阶段。初始化离线和在线强化学习的策略、离线值网络。在离线训练阶段,将静态数据集中的数据放入离线经验池,从离线经验池中随机采样得到静态数据并更新离线策略和离线值网络。在线策略扩展阶段,先冻结离线训练阶段得到的离线策略并将其与在线策略组合成扩展策略集,以一定概率从扩展策略集中选择策略与环境交互并将交互数据放入在线经验池,从在线经验池中随机采样得到静态数据并更新在线策略和在线值网络。本发明在保留离线强化学习安全性的同时,让决策智能体能够从进一步的在线学习中受益。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving decision-making method based on offline and online fusion reinforcement learning


    Additional title:

    一种基于离线在线融合强化学习的自动驾驶决策方法


    Contributors:
    ZHANG KAIXIN (author) / YU SIJIU (author)

    Publication date :

    2025-03-21


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion



    Automatic driving decision-making method based on deep reinforcement learning

    LIU CHENGQI / LIU SHAOWEIHUA / ZHANG YUJIE et al. | European Patent Office | 2024

    Free access

    Automatic driving vehicle microscopic decision-making method based on reinforcement learning

    ZHENG KAN / LIU JIE / ZHAO LONG | European Patent Office | 2020

    Free access

    Automatic driving behavior decision-making method based on deep reinforcement learning

    YANG MINGZHU / LIU XIANGWEI / LI ZHUOLUO | European Patent Office | 2020

    Free access

    Automatic driving behavior decision-making method based on deep reinforcement learning

    ZOU JUNYI / WANG HAIXIN | European Patent Office | 2024

    Free access

    Automatic driving decision-making system based on deep reinforcement learning

    ZHENG XIAOYAO / YAO QINGHE / ZHANG JIANPENG et al. | European Patent Office | 2024

    Free access