The invention relates to an automatic driving decision-making method based on off-line and on-line fusion reinforcement learning, which relates to the field of automatic driving and artificial intelligence and comprises two stages of off-line training and on-line strategy expansion. And initializing offline and online reinforcement learning strategies and an offline value network. In the off-line training stage, data in the static data set are put into an off-line experience pool, static data are obtained through random sampling from the off-line experience pool, and an off-line strategy and an off-line value network are updated. In the online strategy expansion stage, firstly freezing the offline strategies obtained in the offline training stage, combining the offline strategies with the online strategies to form an expansion strategy set, selecting strategies from the expansion strategy set according to a certain probability to interact with the environment, and putting interaction data into an online experience pool; and randomly sampling from the online experience pool to obtain static data, and updating an online strategy and an online value network. According to the method, the decision-making agent can benefit from further online learning while the safety of offline reinforcement learning is kept.
本发明涉及一种基于离线在线融合强化学习的自动驾驶决策方法,涉及自动驾驶和人工智能领域,包括离线训练和在线策略扩展两个阶段。初始化离线和在线强化学习的策略、离线值网络。在离线训练阶段,将静态数据集中的数据放入离线经验池,从离线经验池中随机采样得到静态数据并更新离线策略和离线值网络。在线策略扩展阶段,先冻结离线训练阶段得到的离线策略并将其与在线策略组合成扩展策略集,以一定概率从扩展策略集中选择策略与环境交互并将交互数据放入在线经验池,从在线经验池中随机采样得到静态数据并更新在线策略和在线值网络。本发明在保留离线强化学习安全性的同时,让决策智能体能够从进一步的在线学习中受益。
Automatic driving decision-making method based on offline and online fusion reinforcement learning
一种基于离线在线融合强化学习的自动驾驶决策方法
2025-03-21
Patent
Electronic Resource
Chinese
IPC: | G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion |
Automatic driving decision-making method based on deep reinforcement learning
European Patent Office | 2024
|Automatic driving vehicle microscopic decision-making method based on reinforcement learning
European Patent Office | 2020
|Automatic driving behavior decision-making method based on deep reinforcement learning
European Patent Office | 2020
|Automatic driving behavior decision-making method based on deep reinforcement learning
European Patent Office | 2024
|Automatic driving decision-making system based on deep reinforcement learning
European Patent Office | 2024
|