The integration of Large Language Models (LLMs) into robotics has revolutionized human-robot interactions and autonomous task planning. However, these systems are often unable to self-correct during the task execution, which hinders their adaptability in dynamic real-world environments. To address this issue, we present a Hierarchical Closed-loop Robotic Intelligent Self-correction Planner (HiCRISP), an innovative framework that enables robots to correct errors within individual steps during the task execution. HiCRISP actively monitors and adapts the task execution process, addressing both high-level planning and low-level action errors. Extensive benchmark experiments, encompassing virtual and real-world scenarios, showcase HiCRISP's exceptional performance, positioning it as a promising solution for robotic task planning with LLMs.
Illustration of LLM-based robotic systems: (1) without correction, (2) correction after task completion, and (3) correction within steps. An error occurs during the execution of Action 3. Our proposed HiCRISP belongs to the system that performs corrections within individual steps.
Overview of our proposed HiCRISP framework. The user first sends a task in natural language, which is broken down by LLM into multiple sub-tasks. The planner, which leverages the semantic understanding capabilities of LLM, generates a sequence of reference actions. Translation module, furthermore, translates actions into the low level control sentences which the robot can executes easily. Perception detects environment information and judges whether the system state changes. If failure is detected, corrector generate correction actions in order to fix error.
Illustration of feedback stack structure. The corrector learns the error action and error information from the top of the stack, and then determines the appropriate action to fix the error. If the fixing error action cannot be successfully executed, it is added as a new item to the Fix Action stack. Otherwise, the error action at the top of the stack is attempted, with the goal of removing it from the stack.
HiCRISP demonstrates superior performance in a VirtualHome simulator when compared to ProgPrompt.
HiCRISP acts in Gazebo Engine.
HiCRISP operates on both the Gazebo engine and real-world AGV platform. We command the vehicle to sequentially approach specific landmarks according to a predetermined order. Action failure arises when obstacles obstruct the intended trajectory. Planning failure occur if the vehicle erroneously navigates towards an incorrect landmark. HiCRISP addresses these failures by providing corresponding corrective actions and rectifying the issues.
@misc{ming2023hicrisp,
title={HiCRISP: A Hierarchical Closed-Loop Robotic Intelligent Self-Correction Planner},
author={Chenlin Ming and Jiacheng Lin and Pangkit Fong and Han Wang and Xiaoming Duan and Jianping He},
year={2023},
eprint={2309.12089},
archivePrefix={arXiv},
primaryClass={cs.RO}
}