关键词: 大语言模型, 机械臂抓取, 人机交互, 结构化机器人接口, 监督微调

Abstract: For natural language driven object recognition and grasping in open tabletop environments,many existing methods rely on extremely large models as a central decision maker. While powerful, but they incur high memory, compute cost,large latency,and expensive deployment,making them unsuitable for real time closed loop robotic control. In addition,the lack of a unified structured interface between language level decisions,perception,and control often leads to non-executable outputs,redundant interaction,and unstable failure recovery. It proposes a function calling human robot interaction framework that uses a medium scale language model as the decision core. Visual detection,target selection,grasp point estimation,motion control,and execution feedback are wrapped as robot functions with explicit input output protocols,and the structured robot state is encoded as a text prefix. This design constrains the model to produce parsable and verifiable action calls,enabling stable closed loop execution with feedback-based recovery. Using the 4 B parameter Qwen3 model, it builds state to function call trajectory data and perform supervised fine tuning to improve tool use and task level decision quality while keeping the system lightweight for deployment. Experiments on a real Kinova Gen3 robotic arm with tabletop clutter rearrangement show that the fine-tuned model achieves higher task success,fewer action steps,and better function call validity and accuracy than the non-fine-tuned baseline,and it approaches a much larger 671 B model on key metrics. These results indicate that structured interfaces and task specific fine tuning can make medium scale models practical for real world interactive grasping tasks.

Key words: Large Language Model (LLM), robotic arm grasping, Human-Robot Interaction (HRI), structured robot inter-faces, Supervised Fine-Tuning (SFT)