• CN: 11-2187/TH
  • ISSN: 0577-6686

Journal of Mechanical Engineering ›› 2023, Vol. 59 ›› Issue (6): 173-184.doi: 10.3901/JME.2023.06.173

Previous Articles     Next Articles

Augmented Reality-assisted Mutual Cognitive System for Human-Robot Interaction Safety Concerns

ZHENG Pai1,2, LI Chengxi1, YIN Yue1, ZHANG Rong3, BAO Jinsong3, WANG Baicun4, XIE Haibo4, WANG Lihui5   

  1. 1. Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong 999077;
    2. The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen 518057;
    3. College of Mechanical Engineering, Donghua University, Shanghai 201620;
    4. School of Mechanical Engineering, Zhejiang University, Hangzhou 310058;
    5. KTH Royal Institute of Technology, Stockholm SE-11428, Sweden
  • Received:2022-03-29 Revised:2022-12-21 Online:2023-03-20 Published:2023-06-03

Abstract: In modern manufacturing, the interaction and symbiosis between humans and the industrial robot are one of the foci of smart manufacturing. During human-robot interaction(HRI), the potential risk of any injury to workers caused by industrial robots is very critical and should be well-addressed to ensure manufacturing safety. However, in the dynamic and uncertain manufacturing environment, the current HRI safety is still based on the robot's perception of the environment to achieve collision avoidance, lacking adaptable decision-makings under mutual cognition. Therefore, to enhance the cognition of human operators in the working environment and improve the robot's collision avoidance and adaptive motion planning capabilities, this work designs and further implements a mutual cognitive HRI safety system based on augmented reality (AR) in a wearable manner. In the proposed system, the wearable AR device serves as the bridging interface to realize the virtual-real registration of the robot, the virtual-physical mapping of the working environment of the HRI process, and to collect the information of human, robot, and working space. In addition to these, a hierarchical HRI safety strategy is introduced for real-time mutual cognitive assistance to both humans and robots, namely: 1) robot motion speed control and safety area visualization based on human-robot distance, 2) virtual-physical mapping for robot motion preview and collision detection, and 3) deep reinforcement learning-driven motion planning for collision avoidance strategies generation. Lastly, a prototype system is further developed to validate the feasibility and effectiveness of the proposed strategies. By leveraging advanced artificial intelligence and human-robot interaction technologies, it is envisioned this work can bring insightful safety protection mechanisms to better achieve symbiotic human-robot collaboration.

Key words: smart manufacturing, human-robot interaction, augmented reality, reinforcement learning, safety strategy

CLC Number: