Action recognition has been used in a wide range of applications such as human-computer interaction, intelligent video surveillance systems, video summarization, and robotics. Recognizing action is important for intelligent agents to understand, learn and interact with the environment. The recent technology that allows the acquisition of RGB+D and 3D skeleton data and a deep learning model's development significantly increases the action recognition model's performance. In this research, hierarchical Long Sort-Term Memory is proposed to recognize action based on 3D skeleton joints from Kinect sensor. The model uses the 3D axis of skeleton joints and groups each joint in the axis into parts, namely, spine, left and right arm, left and right hand, and left and right leg. To fit the hierarchically structured layers of LSTM, the parts are concatenated into spine, arms, hands, and legs and then concatenated into the body. The model crosses the body in each axis into a single final body and fed to the final layer to classify the action. The performance is measured using cross-view and cross-subject evaluation and achieves accuracy 0.854 and 0.837, respectively, from the 10 action classes of the NTU RGB+D dataset.


    Access

    Download


    Export, share and cite



    Title :

    Hierarchical long short-term memory for action recognition based on 3D skeleton joints from Kinect sensor


    Contributors:

    Publication date :

    2021-02-24


    Remarks:

    doi:10.26555/jifo.v15i1.a20106
    Jurnal Informatika; Vol 15, No 1 (2021): January 2021; 17-27 ; 2528-6374 ; 1978-0524



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629




    Pedestrian Activity Recognition from 3D Skeleton Data using Long Short Term Memory Units

    Jan, Qazi Hamza / Baddela, Yogitha Sai / Berns, Karsten | TIBKAT | 2022

    Free access

    Robust gait identification using Kinect dynamic skeleton data

    Gianaria, Elena / Grangetto, Marco | BASE | 2019

    Free access

    Recognition method of traffic police and their command action based on kinect

    Song, Wenjie / Fu, Mengyin / Yang, Yi et al. | IEEE | 2014



    Long short-term memory networks for vehicle sensor fusion

    Gandy, Jonah T. / Ball, John E. | British Library Conference Proceedings | 2022