Treffer: 基于语言大模型的工业机器人智能作业综合实验设计.

[Objective] With increasing integration of large language models (LLMs) and robotics, industrial robots are playing pivotal roles in smart manufacturing, particularly in responding to growing demands for flexibility and customization in modern manufacturing. This transition toward intelligent industrial robots represents an inevitable trend in industrial development. This paper aims to explore the combination of LLM applications, machine vision, and industrial robot programming, proposing a new experimental platform design to facilitate the intelligent operation of industrial robots in various scenarios. The study focuses on developing a comprehensive experimental approach that enhances our understanding of industrial robot intelligence and provides new insights for talent cultivation in the rapidly evolving field of smart manufacturing. [Methods] The study design involves an innovative "virtual-real integration" experimental platform for industrial robots that integrates hardware and software components. The platform comprises a computer, the Yuejie E6 collaborative robotic arm, an Intel D435i depth camera, and a simulated work environment created using the Gazebo platform. The experimental system is divided into three core modules: decision-making, perception, and execution. The decision-making module employs an LLM to process voice commands and plan tasks. Meanwhile, the perception module utilizes machine vision for object recognition and precise positioning. Finally, the execution module controls the motion units of the modular units to ensure reliable execution of assigned tasks. The study conducted voice recognition and task decision-making experiments to evaluate the effectiveness of the proposed task planning model in detail. Data processing for these modules was conducted using Python, with experimental environments set up under Windows and Ubuntu operating systems. [Results] Experimental validation showed the effectiveness of the platform, yielding the following notable results: First, regarding voice recognition, the Whisper-1 and Qianfan models achieved recognition accuracy of over 95%, with Qianfan delivering faster response times. Second, regarding LLM task planning, the hierarchical prompt system was highly effective in parsing complex instructions. The LLMs generated valid high-level action sequences and handled ambiguous commands by returning empty action sets. Third, regarding visual perception, hand-eye calibration achieved sufficient accuracy. Notably, traditional image processing provided stable and accurate target localization (mean error < 7 mm), making it suitable for coordinate transformation. Meanwhile, vision foundation models showed better semantic understanding but exhibited larger and less stable localization errors, making them unsuitable for precise positioning. Fourth, regarding integrated performance, in 10 bin-picking trials, voice commands were recognized correctly (>95%), LLMs generated accurate action sequences, and the vision module accurately located targets. The end-effector positioning error, attributed to calibration residuals, was consistently below 6 mm and was deemed acceptable for the task. [Conclusions] This study successfully designed and implemented a comprehensive "virtual-real integration" experimental platform for intelligent industrial robot operations that integrated LLM-based decision-making, machine vision, and modular execution. The hierarchical architecture and innovative prompt engineering strategy provided a robust approach to translating natural language into reliable robot actions. Traditional image processing outperformed vision foundation models in precise localization tasks. The proposed platform provides a practical and accessible tool for students to understand the integration of LLMs, vision, and robotics in intelligent industrial operations, providing a valuable resource for cultivating talent in the emerging field of smart manufacturing. The core design principles, particularly the layered architecture and prompt engineering, present a transferable framework for real-world "AI+" industrial applications. [ABSTRACT FROM AUTHOR]

针对工业机器人智能化作业发展趋势及人才培养需求, 该文提出了一种基于语言大模型的工业机器 人智能作业综合实验设计方案, 旨在通过虚实结合的实验平台, 整合语言大模型、机器视觉与工业机器人作 业编程技术等教学内容。平台以低成本、易搭建为设计理念, 通过模块化实验帮助学生掌握机器人智能化技 术。实验内容涵盖语音识别、任务决策规划与视觉感知等关键技术, 验证了平台在任务规划与执行中的有效性。 [ABSTRACT FROM AUTHOR]

Copyright of Experimental Technology & Management is the property of Experimental Technology & Management Editorial Office and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)