The invention relates to an automatic driving safety assisting method and system based on multi-modal fusion, and belongs to the field of automatic driving. The system is composed of a multi-mode sensor, a preprocessing unit, a feature encoder, a time sequence merging module, a context attention module, a classification module, a data storage unit and an output unit. The method comprises the following steps: S1, collecting visual image data and non-visual image data; s2, preprocessing the visual image data and the non-visual image data; s3, performing feature extraction by using a feature encoder; s4, key event division is carried out by using a time sequence merging module; S5, feature fusion is carried out by using a context attention module; s6, inputting the fusion features into a classification module, and predicting pedestrian behavior types; and S7, storing a pedestrian behavior type prediction result, and feeding back the pedestrian behavior type prediction result to the automobile terminal system. According to the method, key events related to pedestrian behaviors can be effectively identified and aggregated, and the accuracy and efficiency of pedestrian behavior intention prediction are improved.

    本发明涉及基于多模态融合的自动驾驶安全性辅助方法和系统,属于自动驾驶领域。该系统由多模态传感器、预处理单元、特征编码器、时序合并模块、上下文注意力模块、分类模块、数据存储单元、输出单元构成。该方法包含以下步骤:S1:采集视觉图像数据和非视觉图像数据;S2:对视觉图像数据和非视觉图像数据进行预处理;S3:利用特征编码器进行特征提取;S4:利用时序合并模块进行关键事件划分:S5:利用上下文注意力模块对进行特征融合;S6:将融合特征输入分类模块中,预测行人行为类型;S7:将预测行人行为类型结果存储,并反馈给汽车终端系统。本发明方法能够有效地识别和聚合与行人行为相关的关键事件,提高行人行为意图预测的准确性和效率。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Automatic driving safety assisting method and system based on multi-modal fusion


    Weitere Titelangaben:

    基于多模态融合的自动驾驶安全性辅助方法和系统


    Beteiligte:
    CHEN LIN (Autor:in) / LIANG HONGBIN (Autor:in) / SHANG MINGSHENG (Autor:in) / GONG ZHIGANG (Autor:in)

    Erscheinungsdatum :

    21.02.2025


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G06V / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung



    AUTOMATIC DRIVING ASSISTING SYSTEM, AUTOMATIC DRIVING ASSISTING METHOD, AND COMPUTER PROGRAM

    KONDO YOSHITO / SATO YUJI / ISHIKAWA TAKESHI | Europäisches Patentamt | 2015

    Freier Zugriff

    AUTOMATIC DRIVING ASSISTING SYSTEM, AUTOMATIC DRIVING ASSISTING METHOD, AND COMPUTER PROGRAM

    GOTO HIROHIKO / SATO YUJI / TANAKA KUNIAKI et al. | Europäisches Patentamt | 2016

    Freier Zugriff

    Automatic driving method and system based on multi-view multi-modal fusion

    LIU CHUNSHENG / SUN JINYING / CHANG FALIANG et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    Driving safety assisting method and system

    HUANG XUNWEN | Europäisches Patentamt | 2023

    Freier Zugriff

    End-to-end automatic driving multi-modal data fusion method and system

    SU XIAOCONG / LONG KUN / CHEN JUEXUAN et al. | Europäisches Patentamt | 2025

    Freier Zugriff