The invention provides an automatic driving vehicle control method and system based on multi-modal data fusion. The method comprises the following steps: acquiring a multi-modal image set including a three-view camera image, a long-distance camera image and a laser radar image in a vehicle driving environment; inputting the multi-modal image set into a data fusion network to obtain a vehicle driving control instruction; the data fusion network comprises a signal lamp sensing module, a scene sensing module and a trajectory prediction module; inputting the three-view camera image and the long-distance camera image into a signal lamp sensing module, and extracting signal lamp sensing features; inputting the three-view camera image and the laser radar image into a scene sensing module, and extracting scene sensing features; and fusing the signal lamp sensing features and the scene sensing features into multi-model fusion features, inputting the multi-model fusion features into a trajectory prediction module to predict a vehicle displacement vector, and obtaining a vehicle driving control instruction based on the displacement vector. By enhancing the perception of the state of the traffic signal lamp, the perception and understanding ability of the automatic driving system is improved.
本发明提出一种基于多模态数据融合的自动驾驶车辆控制方法及系统,方法为:获取车辆行驶环境中包括三视角相机图像、远距相机图像和激光雷达图像的多模态图像集;将多模态图像集输入数据融合网络,得到车辆行驶控制指令;所述数据融合网络包括信号灯感知模块、场景感知模块和轨迹预测模块;将三视角相机图像和远距相机图像输入信号灯感知模块,提取信号灯感知特征;将三视角相机图像和激光雷达图像输入场景感知模块,提取场景感知特征;将信号灯感知特征与场景感知特征进行融合为多模型融合特征,输入轨迹预测模块预测车辆位移向量,基于位移向量得到车辆行驶控制指令。通过增强对交通信号灯状态的感知,提升自动驾驶系统的感知理解能力。
Automatic driving vehicle control method and system based on multi-modal data fusion
基于多模态数据融合的自动驾驶车辆控制方法及系统
2025-05-23
Patent
Electronic Resource
Chinese
IPC: | B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion |
Automatic driving method and system based on multi-view multi-modal fusion
European Patent Office | 2024
|Automatic driving vehicle safety control method and device based on multi-modal data
European Patent Office | 2025
End-to-end automatic driving multi-modal data fusion method and system
European Patent Office | 2025
|Automatic driving safety assisting method and system based on multi-modal fusion
European Patent Office | 2025
|Vehicle automatic driving control method and system based on fusion
European Patent Office | 2022
|