Yawning detection is a key means in driver fatigue detection, which suffers difficulties including head poses, facial expressions, illumination variations, occlusions, etc. Yet, most previous methods mainly focus on frontal faces, and are deficient to deal with different facial actions under arbitrary poses in the actual driving environment. In this study, we propose a novel Joint Head Pose and Facial Action Network (JHPFA-Net) for driver yawning detection across arbitrary poses in videos, with three main parts including a Geometric-based Key-frame Selection Module (GK-Module), a Face Frontalization with Warp Attention Module (FF-Module) and a dual-channel classifier for Head Pose & Facial Action Fusion Module (HF-Module). Firstly, the GK-Module is proposed to extract geometric vectors and to construct a two-stage judgment mechanism, with the purpose of dealing with frame redundancy and improving the efficiency of JHPFA-Net structure. Secondly, distinguished with existing methods, the FF-Module is proposed to synthesize photo-realistic frontal faces, which can be used for capturing the facial actions under arbitrary poses. Finally, the HF-Module is proposed to fuse head pose attributes and facial modalities together, for the purpose of achieving pose-invariant detection and improving accuracy. Extensive experiments show that the proposed JHPFA-Net achieves state-of-the-art results comparing with some representative methods on the public YawDD benchmark, and it performs well in real-time application.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    JHPFA-Net: Joint Head Pose and Facial Action Network for Driver Yawning Detection Across Arbitrary Poses in Videos


    Contributors:
    Lu, Yansha (author) / Liu, Chunsheng (author) / Chang, Faliang (author) / Liu, Hui (author) / Huan, Hengqiang (author)

    Published in:

    Publication date :

    2023-11-01


    Size :

    3168486 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Yawning detection for monitoring driver fatigue based on two cameras

    Li, Lingling / Chen, Yangzhou / Li, Zhenlong | IEEE | 2009


    Driver Head Pose Detection From Naturalistic Driving Data

    Chai, Weiheng / Chen, Jiajing / Wang, Jiyang et al. | IEEE | 2023


    DD-POSE - A LARGE-SCALE DRIVER HEAD POSE BENCHMARK

    Roth, Markus / Gavrila, Dariu M. | British Library Conference Proceedings | 2019


    DD-Pose - A large-scale Driver Head Pose Benchmark

    Roth, Markus / Gavrila, Dariu M. | IEEE | 2019


    Automatic fatigue detection of drivers through yawning analysis

    Azim, Tayyaba / Jaffar, M. Arfan / Ramzan, M. et al. | Tema Archive | 2009