Real-time recognition of driver emotions can greatly improve traffic safety. With the rapid development of communication technology, it becomes possible to process large amounts of video data and identify the driver’s emotions in real time. To effectively recognize driver’s emotions, this paper proposes a new deep learning framework called Convolution Bidirectional Long Short-term Memory Neural Network (CBLNN). This method predicts the driver’s emotion based on the geometric features extracted from facial skin information and the heart rate extracted from changes in RGB components. The facial geometry features obtained by using Convolutional Neural Network (CNN) are intermediate variables for the heart rate analysis of Bidirectional Long Short Term Memory (Bi-LSTM). Subsequently, the output of Bi-LSTM is used as input to the CNN module to extract the hear rate features. CBLNN uses Multi-modal factorized bilinear pooling (MFB) to fuse the extracted information and classifies it into five common emotions: happiness, anger, sadness, fear and neutrality. Our emotion recognition method was tested, proving that it can be used to quickly and steadily recognize emotions in real time.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Convolution Bidirectional Long Short-Term Memory Neural Network for Driver Emotion Recognition


    Contributors:


    Publication date :

    2021-07-01


    Size :

    2795734 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    A modified bidirectional long short-term memory neural network for rail vehicle suspension fault detection

    Chen, Yuejian / Niu, Gang / Li, Yifan et al. | Taylor & Francis Verlag | 2023




    Long Short Term Memory for Driver Intent Prediction

    Zyner, Alex / Worrall, Stewart / Ward, James Robert et al. | British Library Conference Proceedings | 2017