Touchless in-car devices controlled by single and continuous finger gestures can provide comfort and safety on driving while manipulating secondary devices. Recognition of finger gestures is a challenging task due to (i) similarities between gesture and non-gesture frames, and (ii) the difficulty in identifying the temporal boundaries of continuous gestures. In addition, (iii) the intraclass variability of gestures’ duration is a critical issue for recognizing finger gestures intended to control in-car devices. To address difficulties (i) and (ii), we propose a gesture spotting method where continuous gestures are segmented by detecting boundary frames and evaluating hand similarities between the start and end boundaries of each gesture. Subsequently, we introduce a gesture recognition based on a temporal normalization of features extracted from the set of spotted frames, which overcomes difficulty (iii). This normalization enables the representation of any gesture with the same limited number of features. We ensure real-time performance by proposing an approach based on compact deep neural networks. Moreover, we demonstrate the effectiveness of our proposal with a second approach based on hand-crafted features performing in real-time, even without GPU requirements. Furthermore, we present a realistic driving setup to capture a dataset of continuous finger gestures, which includes more than 2,800 instances on untrimmed videos covering safety driving requirements. With this dataset, our both approaches can run at 53 fps and 28 fps on GPU and CPU, respectively, around 13 fps faster than previous works, while achieving better performance (at least 5% higher mean tIoU).


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Continuous Finger Gesture Spotting and Recognition Based on Similarities Between Start and End Frames


    Contributors:


    Publication date :

    2022-01-01


    Size :

    4527327 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    A Fuzzy Framework for Real-Time Gesture Spotting and Recognition

    Bakheet, S. | British Library Online Contents | 2017


    Robust Player Gesture Spotting and Recognition in Low-Resolution Sports Video

    Roh, M.-C. / Christmas, B. / Kittler, J. et al. | British Library Conference Proceedings | 2006


    An Efficient Finger Gesture Recognition System Using Image

    Kumar, M. Arun / Jayachithra, S. / Aravindh, G. et al. | IEEE | 2020


    Continuous Gesture Sequences Recognition Based on Few-Shot Learning

    Zhe Liu / Cao Pan / Hongyuan Wang | DOAJ | 2022

    Free access