While deep Convolutional Neural Networks(CNNs) have become front-runners in the field of driver observation, they are often perceived as black boxes due to their end-to-end nature. Interpretability of such models is vital for building trust and is a serious concern for the integration of CNNs in real-life systems. In this paper, we implement a diagnostic framework for analyzing such models internally and shed light on the learned spatiotemporal representations in a comprehensive study. We examine prominent driver monitoring models from three points of view: (1) visually explaining the prediction by combining the gradient with respect to the intermediate features and the corresponding activation maps, (2) looking at what the network has learned by clustering the internal representations and discovering, how individual classes relate at the feature-level, and (3) conducting a detailed failure analysis with multiple metrics and evaluation settings (e.g. common versus rare behaviors). Among our findings, we show that most of the mistakes can be traced back to learning an object- or a specific movement bias, strong semantic similarity between classes (e.g. preparing food and eating) and underrepresentation in the training set. Besides, we demonstrate the advantages of the Inflated 3D Net compared to other CNNs as it results in more discriminative embedding clusters and in the highest recognition rates based on all metrics.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    CNN-based Driver Activity Understanding: Shedding Light on Deep Spatiotemporal Representations


    Contributors:


    Publication date :

    2020-09-20


    Size :

    1634893 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Shedding new light

    Online Contents | 2005


    Shedding Light on Stereoscopic Segmentation

    Jin, H. / Cremers, D. / Yezzi, A. et al. | British Library Conference Proceedings | 2004


    Shedding More Light on Transit

    Online Contents | 1995


    Shedding light on stereoscopic segmentation

    Hailin Jin, / Cremers, D. / Yezzi, A.J. et al. | IEEE | 2004


    Shedding light on the weather

    Narasimhan, S.G. / Nayar, S.K. | IEEE | 2003