The invention discloses a fatigue driving detection method and system based on multi-modal feature fusion, and the method is explained by taking a DROZY data set as an embodiment. Through preprocessing a data set, image data and PSG time sequence data which are in one-to-one correspondence in time are extracted, and the image data and the time sequence data are input and used by a subsequent system. Feature information of shallow, medium and deep networks is extracted from the two types of modal data; and after the dimensions of the three feature blocks of the two modes are adjusted to be consistent, feature information is optimized through a residual convolution attention module. And obtaining weight information of different layers of the single-mode data through a weight calculation method, and carrying out multi-scale weight fusion on the weight information and an original feature block to obtain a feature vector extraction result of the image data and the time series data. And fusing the feature vector data of the two modes, and outputting a classification result through a classifier according to the fused features to realize fatigue driving detection.
本发明公开了一种基于多模态特征融合的疲劳驾驶检测方法及系统,该方法通过DROZY数据集作为实施例,对该发明方法进行说明。通过对数据集进行预处理,提取出时间上一一对应的图像数据与PSG时序数据,图像与时序数据供后续系统输入使用。对两种模态数据提取浅、中、深层网络的特征信息。得到两种模态的3个特征块维数调整一致后,通过残差卷积注意力模块优化特征信息。再经过权重计算方法得到单模态数据不同层的权重信息,再由权重信息与原特征块进行多尺度权重融合,得到图像数据与时序数据的特征向量提取结果。两模态的特征向量数据进行融合,由融合特征通过分类器输出分类结果,实现疲劳驾驶检测。
Fatigue driving detection method and system based on multi-modal feature fusion
一种基于多模态特征融合的疲劳驾驶检测方法及系统
04.04.2025
Patent
Elektronische Ressource
Chinesisch
Fatigue Driving Detection Based on Multi Feature Fusion
IEEE | 2019
|Fatigue driving detection method based on multi-information fusion
Europäisches Patentamt | 2020
|Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning
IET | 2019
|Train driving detection method and system based on multi-feature fusion
Europäisches Patentamt | 2023
|