The concept of Industry 4.0 brings the change of industry manufacturing patterns that become more efficient and more flexible. In response to this tendency, an efficient robot teaching approach without complex programming has become a popular research direction. Therefore, we propose an interactive finger-touch based robot teaching schema using a multimodal 3D image (color (RGB), thermal (T) and point cloud (3D)) processing. Here, the resulting heat trace touching the object surface will be analyzed on multimodal data, in order to precisely identify the true hand/object contact points. These identified contact points are used to calculate the robot path directly. To optimize the identification of the contact points we propose a calculation scheme using a number of anchor points which are first predicted by hand/object point cloud segmentation. Subsequently a probability density function is defined to calculate the prior probability distribution of true finger trace. The temperature in the neighborhood of each anchor point is then dynamically analyzed to calculate the likelihood. Experiments show that the trajectories estimated by our multimodal method have significantly better accuracy and smoothness than only by analyzing point cloud and static temperature distribution.


    Access

    Download


    Export, share and cite



    Title :

    Interactive robot teaching based on finger trajectory using multimodal RGB-D-T-data


    Contributors:

    Publication date :

    2023-03-16



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    004 / 629




    Interactive Multimodal Mobile Robot for Musical Performance

    Suzuki, K. / Ohashi, T. / Hashimoto, S. | British Library Conference Proceedings | 1999


    MULTIMODAL TRAJECTORY PREDICTIONS BASED ON GEOMETRIC ANCHORING

    ABDULHAMID ADAM / KURUTACH THANARD | European Patent Office | 2024

    Free access

    TRAJECTORY-BASED MULTIMODAL TRANSPORT MANAGEMENT FOR RESILIENT TRANSPORTATION

    Evelin ENGLER / Stefan GEWIES / Paweł BANYŚ et al. | DOAJ | 2018

    Free access

    Visual Interactive Trajectory Design

    Mathur, Ravishankar | TIBKAT | 2019