PURPOSE: Concentric tube robots are composed of multiple concentric, pre-curved, super-elastic, telescopic tubes that are compliant and have a small diameter suitable for interventions that must be minimally invasive like fetal surgery. Combinations of rotation and extension of the tubes can alter the robot's shape but the inverse kinematics are complex to model due to the challenge of incorporating friction and other tube interactions or manufacturing imperfections. We propose a model-free reinforcement learning approach to form the inverse kinematics solution and directly obtain a control policy. METHOD: Three exploration strategies are shown for deep deterministic policy gradient with hindsight experience replay for concentric tube robots in simulation environments. The aim is to overcome the joint to Cartesian sampling bias and be scalable with the number of robotic tubes. To compare strategies, evaluation of the trained policy network to selected Cartesian goals and associated errors are analyzed. The learned control policy is demonstrated with trajectory following tasks. RESULTS: Separation of extension and rotation joints for Gaussian exploration is required to overcome Cartesian sampling bias. Parameter noise and Ornstein-Uhlenbeck were found to be optimal strategies with less than 1 mm error in all simulation environments. Various trajectories can be followed with the optimal exploration strategy learned policy at high joint extension values. Our inverse kinematics solver in evaluation has 0.44 mm extension and [Formula: see text] rotation error. CONCLUSION: We demonstrate the feasibility of effective model-free control for concentric tube robots. Directly using the control policy, arbitrary trajectories can be followed and this is an important step towards overcoming the challenge of concentric tube robot control for clinical use in minimally invasive interventions.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Investigating exploration for deep reinforcement learning of concentric tube robot control


    Beteiligte:
    Iyengar, K (Autor:in) / Dwyer, G (Autor:in) / Stoyanov, D (Autor:in)

    Erscheinungsdatum :

    06.06.2020


    Anmerkungen:

    International Journal of Computer Assisted Radiology and Surgery , 15 pp. 1157-1165. (2020)


    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629



    Deep Reinforcement Learning for Concentric Tube Robot Path Following

    Iyengar, Keshav / Spurgeon, Sarah / Stoyanov, Danail | BASE | 2023

    Freier Zugriff


    A Deep Reinforcement Learning Approach for Inverse Kinematics of Concentric Tube Robots

    Iyengar, K / Dwyer, G / Stoyanov, D | BASE | 2019

    Freier Zugriff

    Sim2Real Transfer of Reinforcement Learning for Concentric Tube Robots

    Iyengar, Keshav / Sadati, SM Hadi / Bergeles, Christos et al. | BASE | 2023

    Freier Zugriff

    Coordinated Multi-Robot Exploration using Reinforcement Learning

    Mete, Atharva / Mouhoub, Malek / Farid, Ali Moltajaei | IEEE | 2023