The invention discloses a fatigue driving detection method and system based on multi-modal feature fusion, and the method is explained by taking a DROZY data set as an embodiment. Through preprocessing a data set, image data and PSG time sequence data which are in one-to-one correspondence in time are extracted, and the image data and the time sequence data are input and used by a subsequent system. Feature information of shallow, medium and deep networks is extracted from the two types of modal data; and after the dimensions of the three feature blocks of the two modes are adjusted to be consistent, feature information is optimized through a residual convolution attention module. And obtaining weight information of different layers of the single-mode data through a weight calculation method, and carrying out multi-scale weight fusion on the weight information and an original feature block to obtain a feature vector extraction result of the image data and the time series data. And fusing the feature vector data of the two modes, and outputting a classification result through a classifier according to the fused features to realize fatigue driving detection.

    本发明公开了一种基于多模态特征融合的疲劳驾驶检测方法及系统,该方法通过DROZY数据集作为实施例,对该发明方法进行说明。通过对数据集进行预处理,提取出时间上一一对应的图像数据与PSG时序数据,图像与时序数据供后续系统输入使用。对两种模态数据提取浅、中、深层网络的特征信息。得到两种模态的3个特征块维数调整一致后,通过残差卷积注意力模块优化特征信息。再经过权重计算方法得到单模态数据不同层的权重信息,再由权重信息与原特征块进行多尺度权重融合,得到图像数据与时序数据的特征向量提取结果。两模态的特征向量数据进行融合,由融合特征通过分类器输出分类结果,实现疲劳驾驶检测。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Fatigue driving detection method and system based on multi-modal feature fusion


    Weitere Titelangaben:

    一种基于多模态特征融合的疲劳驾驶检测方法及系统


    Beteiligte:
    YE LUOQI (Autor:in) / CHEN XI (Autor:in) / FENG PEIHUA (Autor:in)

    Erscheinungsdatum :

    04.04.2025


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G06V / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung



    Fatigue Driving Detection Based on Multi Feature Fusion

    Deng, Wanghua / Zhan, Zeyan / Yu, Yi et al. | IEEE | 2019


    Fatigue driving state detection for tanker truck drivers based on multi-feature fusion

    Zhang, Ning / Zhang, Ziyi / Ma, Chuanyi et al. | SPIE | 2024


    Fatigue driving detection method based on multi-information fusion

    MENG SIHONG | Europäisches Patentamt | 2020

    Freier Zugriff

    Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning

    Li, Xu / Hong, Lin / Wang, Jian-chun et al. | IET | 2019

    Freier Zugriff

    Train driving detection method and system based on multi-feature fusion

    YU DAN / LAN YUQING / LI YIJUN et al. | Europäisches Patentamt | 2023

    Freier Zugriff