Large Language Models (LLMs) have the potential to revolutionize the field of robotics by enabling advanced natural language understanding and reasoning capabilities, making them ideal for robot control and human–robot interaction. This chapter explores the role of LLMs in LLM-controlled robot systems, focusing on human–robot interaction and instruction–action alignment. Various literature studies have reported that by pretraining on vast amounts of text data across different domains and fine-tuning with human demonstration data, LLMs can interpret high-level natural language commands and translate them into actionable instructions for robots to execute. This approach offers several advantages over traditional robot control methods, such as improved accuracy, flexibility, robustness, and user-friendliness. The integration of LLMs into closed-loop robot control systems has led to the development of various real-world applications, including robots for interactive healthcare, manufacturing workflow, and home assistance. During the pre-training and fine-tuning, it is vital to leverage advanced machine learning techniques such as multimodal learning, semantic context analysis, real-time feedback, and interactive learning mechanisms to handle ambiguities and under-specification in natural language instructions. From the learning theory perspective, feedback loops and continuous learning also play important roles in enhancing the model’s capability to understand natural language instructions and the accuracy of generated robot actions. Future research directions include integrating more modalities of human instructions, such as brain signals and hand gestures, for closed-loop robot control system design, as well as leveraging community efforts and open-source resources to accelerate innovation in LLM-based robot control.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Robot Control via Natural Instructions Empowered by Large Language Model


    Beteiligte:
    Vinjamuri, Ramana (Herausgeber:in) / Wu, Zihao (Autor:in) / Shu, Peng (Autor:in) / Li, Yiwei (Autor:in) / Li, Quanzheng (Autor:in) / Liu, Tianming (Autor:in) / Li, Xiang (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    24.07.2024


    Format / Umfang :

    21 pages




    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Extraction of robot primitive control rules from natural language instructions

    Wang, G. H. / Jiang, P. / Feng, Z. R. | British Library Online Contents | 2006


    LINKs: Large Language Model Integrated Management for 6G Empowered Digital Twin NetworKs

    Jiang, Shufan / Lin, Bangyan / Wu, Yue et al. | IEEE | 2024



    Learning procedures from interactive natural language instructions

    United States; National Aeronautics and Space Administration / United States; Air Force | British Library Conference Proceedings | 1993


    Machine Learning Empowered Large RIS-assisted Near-field Communications

    Zhong, Ruikang / Mu, Xidong / Liu, Yuanwei | IEEE | 2023