Recognition of human hand gestures in industrial environments is gaining popularity, especially in the context of assistance systems, thanks to advancements in deep learning-based vision methods. Also, head-worn devices with cameras are becoming more popular especially for smart assistance using Extended Reality (XR) technology, even for industrial use cases. Employing sensors from head-worn devices such as HoloLens enhance the communication between human and robot hereby providing interaction using ego-centric vision. This study delves into human-robot interaction by investigating ego-centered hand gesture recognition for commanding robots. A pipeline is developed for collecting these HoloLens video frames and to detect hand landmark labels on them using MediaPipe library by Google. Then, a Long Short-Term Memory Network (LSTM) model for hand-gesture recognition was developed that classifies the hand-gesture from the given hand landmarks in near real-time, which can then be translated into robot commands. We also present results for our network’s performance and implementation pipeline.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Human-Robot Interaction Through Egocentric Hand Gesture Recognition


    Weitere Titelangaben:

    Lect.Notes Mechanical Engineering


    Beteiligte:
    Alexopoulos, Kosmas (Herausgeber:in) / Makris, Sotiris (Herausgeber:in) / Stavropoulos, Panagiotis (Herausgeber:in) / Walunj, Snehal (Autor:in) / Mashhaditafreshi, Nazanin (Autor:in) / Pahlevannejad, Parsha (Autor:in) / Wagner, Achim (Autor:in) / Ruskowski, Martin (Autor:in)

    Kongress:

    European Symposium on Artificial Intelligence in Manufacturing ; 2024 ; Athens, Greece October 16, 2024 - October 16, 2024



    Erscheinungsdatum :

    22.03.2025


    Format / Umfang :

    9 pages





    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Real-Time Hand Gesture Recognition for Robot Hand Interface

    Lv, X. / Xu, Y. / Wang, M. | British Library Conference Proceedings | 2014


    Real-Time Hand Gesture Recognition for Robot Hand Interface

    Lv, Xiaomeng / Xu, Yulin / Wang, Ming | Springer Verlag | 2014


    Human-Computer Interaction for Drone Control through Hand Gesture Recognition with MediaPipe Integration

    Kumar, G Hemanth / K N, Vidya Sagar / Patil, Prajwal et al. | IEEE | 2024


    An integrative framework of human hand gesture segmentation for human robot interaction

    Ju, Zhaojie / Ji, Xiaofei / Li, Jing et al. | BASE | 2015

    Freier Zugriff

    Console Free Robot Chassis Using Hand Gesture Recognition

    P, Shanmuga Priya / T, Swetha / D, Swetha et al. | IEEE | 2023