Research on Rapid Reconstruction and Collision Detection of Human Machine Point Clouds for Tightly Coupled Collaborative Spaces

doi:10.3901/JME.2025.16.057

Abstract

Abstract: High reliability human-machine collision detection is the main challenge for the safe operation of robotic arms in tightly coupled collaborative spaces. Aiming at the problems of point cloud occlusion in tightly coupled collaboration space leading to incomplete perception of human obstacle space, and obstacle envelope space redundancy leading to failure of collaborative manipulator posture planning, a combination of human-machine skeleton information and sparse sampling method is used to realize the rapid reconstruction of human-machine point cloud, and a compact axis-aligned bounding box generation and collision detection method based on voxel approximation strategy is proposed. The accuracy of the proposed method in human obstacle spatial perception was verified in a typical scenario of human-machine cooperation for a certain type of reducer. The experimental results show that the proposed method can achieve fast and complete reconstruction and collision detection of human-machine point clouds without losing efficiency. It can effectively reduce the volume of obstacle envelope space by 92.38% and expand the obstacle avoidance motion space of the robotic arm by 1.15 m³.

Key words: human-machine collaborative assembly, point cloud reconstruction, axis-aligned bounding box, voxel approximation, collision detection

CLC Number:

TG156

DONG Yuanfa, HUANG Jitao, LI Haifan, ZHOU Bin, PENG Wei, AN Youjun. Research on Rapid Reconstruction and Collision Detection of Human Machine Point Clouds for Tightly Coupled Collaborative Spaces[J]. Journal of Mechanical Engineering, 2025, 61(16): 57-69.

References

[1] BISEN A, PAYAL H. Collaborative robots for industrial tasks: A review[J]. Materials Today: Proceedings, 2022, 52: 500-504.
[2] 王耀南,江一鸣,姜娇,等. 机器人感知与控制关键技术及其智能制造应用[J]. 自动化学报, 2023, 49(3): 494-513. WANG Yaonan, JIANG Yiming, JIANG Jiao, et al. Key technologies of robot perception and control and its intelligent manufacturing applications[J]. Acta Automatica Sinica, 2023, 49(3): 494-513.
[3] 陶永,兰江波,刘海涛,等. 基于滑模和模糊算法相融合的人机协作机器人轨迹跟踪方法[J]. 机械工程学报, 2022, 58(18): 181-191. TAO Yong, LAN Jiangbo, LIU Haitao, et al. Cooperative robot trajectory tracking control method based on the fusion of sliding mode control and fuzzy algorithm[J]. Journal of Mechanical Engineering , 2022 , 58(18) : 181-191.
[4] 杨赓, 周慧颖, 王柏村. 数字孪生驱动的智能人机协作: 理论、技术与应用[J]. 机械工程学报, 2022, 58(18): 279-291. YANG Geng, ZHOU Huiying, WANG Baicun. Digital twin-driven smart human-machine collaboration: Theory, enabling technologies and applications[J]. Journal of Mechanical Engineering, 2022, 58(18): 279-291.
[5] BDIWI M, PFEIFER M, STERZING A. A new strategy for ensuring human safety during various levels of interaction with industrial robots[J]. CIRP Annals, 2017, 66(1): 453-456.
[6] 石大为. 面向机器人装配的人机协作安全防护关键技术研究[D]. 沈阳:东北大学, 2020. SHI Dawei. Research on the key technologies of human machine cooperative safety protection for robot assembly [D]. Shenyang: Northeastern University, 2020.
[7] HADDADIN S, ALBU-SCHAFFER A, DE LUCA A, et al. Collision detection and reaction: A contribution to safe physical human-robot interaction[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2008: 3356-3363.
[8] LU S, CHUNG J H, VELINSKY S A. Human-robot collision detection and identification based on wrist and base force/torque sensors[C]//Proceedings of the 2005 IEEE International Conference on Robotics and Automation. New York: IEEE, 2005: 3796-3801.
[9] DE LUCA A, ALBU-SCHAFFER A, HADDADIN S, et al. Collision detection and safe reaction with the DLR-Ⅲ lightweight manipulator arm[C]//2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2006: 1623-1630.
[10] 张建华,蔡灿,刘璇,等. 基于二阶前馈外力观测器的机械臂碰撞策略[J]. 计算机集成制造系统, 2019, 25(7): 1775-1783. ZHANG Jianhua, CAI Can, LIU Xuan, et al. Manipulator collision strategy based on second-order feedforward external force observer[J]. Computer Integrated Manufacturing Systems, 2019, 25(7): 1775-1783.
[11] 谢峰,李红杨,甄圣超,等. 基于带通滤波二阶动量观测器的机器人碰撞检测[J]. 机械工程学报, 2024, 60(21): 99-111. XIE Feng, LI Hongyang, ZHEN Shengchao, et al. Robot collision detection based on bandpass filtered second-order momentum observer[J]. Journal of Mechanical Engineering, 2024, 60(21): 99-111.
[12] LASOTA P A, FONG T, SHAH J A. A survey of methods for safe human-robot interaction[J]. Foundations and Trends in Robotics, 2017, 5(4): 261-349.
[13] 黄沿江,汪子钦,张宪民,等. 人与机器人共存中的位姿估计与碰撞检测[J]. 机器人, 2022, 44(3): 281-290. HUANG Yanjiang, WANG Ziqin, ZHANG Xianmin, et al. Pose estimation and collision detection in human-robot coexistence[J]. Robot, 2022, 44(3): 281-290.
[14] SAFEEA M, NETO P. Minimum distance calculation using laser scanner and IMUs for safe human-robot interaction[J]. Robotics and Computer-Integrated Manufacturing, 2019, 58: 33-42.
[15] 席明磊. 基于深度学习的人机协作安全方法研究[D]. 天津:天津理工大学, 2022. XI Minglei. Research on human-computer collaboration security based on deep learning[D]. Tianjin : Tianjin University of Technology, 2022.
[16] 涂武强. 基于人体动作识别的人机协作安全策略研究[D]. 哈尔滨:哈尔滨工业大学, 2020. TU Wuqiang. Research on safety strategy of human-robot cooperation based on human motion recognition[D]. Harbin: Harbin Institute of Technology, 2020.
[17] KALDESTAD K B, HADDADIN S, BELDER R, et al. Collision avoidance with potential fields based on parallel processing of 3D-point cloud data on the GPU[C]//2014 IEEE International Conference on Robotics and Automation (ICRA). New York: IEEE, 2014: 3250-3257.
[18] FABRIZIO F, DE LUCA A. Real-time computation of distance to dynamic obstacles with multiple depth sensors[J]. IEEE Robotics and Automation Letters, 2016, 2(1): 56-63.
[19] BALAN L, BONE G M. Real-time 3D collision avoidance method for safe human and robot coexistence[C]// 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2006: 276-282.
[20] LIU C, TOMIZUKA M. Algorithmic safety measures for intelligent industrial co-robots[C]//2016 IEEE International Conference on Robotics and Automation (ICRA). New York: IEEE, 2016: 3095-3102.
[21] OUYANG F, ZHANG T. Virtual velocity vector-based offline collision-free path planning of industrial robotic manipulator[J]. International Journal of Advanced Robotic Systems, 2015, 12(9): 129.
[22] KUHN S, HENRICH D. Fast vision-based minimum distance determination between known and unkown objects[C]//2007 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2007: 2186-2191.
[23] FANG C, ROCCHI A, HOFFMAN E M, et al. Efficient self-collision avoidance based on focus of interest for humanoid robots[C]//2015 IEEE-RAS 15th International Conference on Humanoid Robots(Humanoids). New York: IEEE, 2015: 1060-1066.
[24] AFAGHANI A Y , AIYAMA Y. On-line collision avoidance between two robot manipulators using collision map and simple escaping method[C]//Proceedings of the 2013 IEEE/SICE International Symposium on System Integration. New York: IEEE, 2013: 105-110.
[25] FLACCO F, KRÖGER T, DE LUCA A, et al. A depth space approach to human-robot collision avoidance[C]//2012 IEEE International Conference on Robotics and Automation. New York: IEEE, 2012: 338-345.
[26] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: Common objects in context[C]//Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer International Publishing, 2014: 740-755.
[27] LIU J, SHAHROUDY A, PEREZ M, et al. Ntu RGB+D 120: A large-scale benchmark for 3D human activity understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(10): 2684-2701.
[28] 朱瑛,谢睿,郑若池. 基于节点匹配代价优化的随机森林算法[J]. 计算机工程与设计, 2020, 41(11): 3106-3111. ZHU Ying, XIE Rui, ZHENG Ruochi. Optimization of random forest algorithm based on nodes matching cost[J]. Computer Engineering and Design , 2020 , 41(11) : 3106-3111.
[29] 李晨光. 基于 KinectV2 的人体姿态识别研究[D]. 秦皇岛:燕山大学, 2021. LI Chenguang. Research on human posture recognition based on KinectV2[D]. Qinhuangdao: Yanshan University, 2021.
[30] 于文斐. 基于虚拟现实技术的碰撞检测算法综述[J]. 民航学报, 2019, 3(4): 85-87, 96. YU Wenfei. Overview of collision detection algorithms based on virtual reality technology[J]. Journal of Civil Aviation, 2019, 3(4): 85-87, 96.
[31] 徐硕. 基于 Kinect 多视角点云的人体三维重建系统研究[D]. 武汉:武汉纺织大学, 2023. XU Shuo. Research on human 3D reconstruction system based on kinect multi-view point cloud[D]. Wuhan: Wuhan Textile University, 2023.