The invention discloses a multi-modal road perception method and system, terminal equipment and a storage medium, and the method comprises the steps: mapping the obtained visual information, thermal imaging information, traffic information and three-dimensional space information to a same coordinate system, and carrying out the coordinate system alignment, target visual information, target thermal imaging information, target traffic information and target three-dimensional space information are obtained; performing feature extraction and classification to obtain a first perception result corresponding to the target visual information, a second perception result corresponding to the target thermal imaging information, a third perception result corresponding to the target traffic information and a fourth perception result corresponding to the target three-dimensional space information; according to the first perception result, the second perception result, the third perception result and the fourth perception result, the perception result of the road to be perceived is obtained through calculation, coordinate system alignment is conducted on the multi-modal road information, then integration is conducted, the comprehensiveness of the road perception information is improved, and multi-modal road information perception is achieved.
本发明公开了一种多模态道路感知方法、系统、终端设备及存储介质,方法包括:将获取的视觉信息、热成像信息、交通信息以及三维空间信息,映射到同一坐标系下进行坐标系对齐,得到目标视觉信息、目标热成像信息、目标交通信息以及目标三维空间信息;然后进行特征提取并分类,得到目标视觉信息所对应的第一感知结果、目标热成像信息所对应的第二感知结果、目标交通信息所对应的第三感知结果以及目标三维空间信息所对应的第四感知结果;根据第一感知结果、第二感知结果、第三感知结果以及第四感知结果,计算得到待感知道路的感知结果,对多模态道路信息进行坐标系对齐后整合,提高道路感知信息的全面性,实现多模态道路信息感知。
Multi-modal road sensing method and system, terminal equipment and storage medium
一种多模态道路感知方法、系统、终端设备及存储介质
2024-08-30
Patent
Electronic Resource
Chinese
IPC: | G06V / G01D MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE , Anzeigen oder Aufzeichnen in Verbindung mit Messen allgemein / G01J Messen der Intensität, der Geschwindigkeit, der spektralen Zusammensetzung, der Polarisation, der Phase oder der Pulscharakteristik von infrarotem, sichtbarem oder ultraviolettem Licht , MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT / G01S RADIO DIRECTION-FINDING , Funkpeilung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS |
Multi-modal sensing road anomaly identification method and system
European Patent Office | 2024
|Road driving environment intelligent sensing method and system, terminal and storage medium
European Patent Office | 2024
|Road violation snapshot method and system, terminal equipment and storage medium
European Patent Office | 2020
|Road equipment control method and device, terminal and readable storage medium
European Patent Office | 2022
|Road condition prediction method and device, terminal equipment and storage medium
European Patent Office | 2023
|