The invention discloses a multi-modal road perception method and system, terminal equipment and a storage medium, and the method comprises the steps: mapping the obtained visual information, thermal imaging information, traffic information and three-dimensional space information to a same coordinate system, and carrying out the coordinate system alignment, target visual information, target thermal imaging information, target traffic information and target three-dimensional space information are obtained; performing feature extraction and classification to obtain a first perception result corresponding to the target visual information, a second perception result corresponding to the target thermal imaging information, a third perception result corresponding to the target traffic information and a fourth perception result corresponding to the target three-dimensional space information; according to the first perception result, the second perception result, the third perception result and the fourth perception result, the perception result of the road to be perceived is obtained through calculation, coordinate system alignment is conducted on the multi-modal road information, then integration is conducted, the comprehensiveness of the road perception information is improved, and multi-modal road information perception is achieved.

    本发明公开了一种多模态道路感知方法、系统、终端设备及存储介质,方法包括:将获取的视觉信息、热成像信息、交通信息以及三维空间信息,映射到同一坐标系下进行坐标系对齐,得到目标视觉信息、目标热成像信息、目标交通信息以及目标三维空间信息;然后进行特征提取并分类,得到目标视觉信息所对应的第一感知结果、目标热成像信息所对应的第二感知结果、目标交通信息所对应的第三感知结果以及目标三维空间信息所对应的第四感知结果;根据第一感知结果、第二感知结果、第三感知结果以及第四感知结果,计算得到待感知道路的感知结果,对多模态道路信息进行坐标系对齐后整合,提高道路感知信息的全面性,实现多模态道路信息感知。


    Access

    Download


    Export, share and cite



    Title :

    Multi-modal road sensing method and system, terminal equipment and storage medium


    Additional title:

    一种多模态道路感知方法、系统、终端设备及存储介质


    Contributors:
    ZHANG ZHITING (author) / LI LIMING (author) / ZHENG JIANFENG (author) / HUANG XIAOCHUAN (author) / ZHOU YUE (author) / FU TIAN (author) / LIU XIAOFENG (author)

    Publication date :

    2024-08-30


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06V / G01D MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE , Anzeigen oder Aufzeichnen in Verbindung mit Messen allgemein / G01J Messen der Intensität, der Geschwindigkeit, der spektralen Zusammensetzung, der Polarisation, der Phase oder der Pulscharakteristik von infrarotem, sichtbarem oder ultraviolettem Licht , MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT / G01S RADIO DIRECTION-FINDING , Funkpeilung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS



    Multi-modal sensing road anomaly identification method and system

    YAN JUN / HUO JIANJIE | European Patent Office | 2024

    Free access

    Road driving environment intelligent sensing method and system, terminal and storage medium

    JI LIWEI / JING HAOXIN / CHEN MENG | European Patent Office | 2024

    Free access

    Road violation snapshot method and system, terminal equipment and storage medium

    GUO LIN / WAN YAN | European Patent Office | 2020

    Free access

    Road equipment control method and device, terminal and readable storage medium

    LIU WEIGANG / YUE RUIPING / TAN LONGJIANG | European Patent Office | 2022

    Free access

    Road condition prediction method and device, terminal equipment and storage medium

    BAO YU / FAN YANG / PENG XIANGXIONG | European Patent Office | 2023

    Free access