面向机器人操作的柔性线缆视觉分割与抓取位姿生成算法

doi:10.3901/JME.2025.09.231

机械工程学报 ›› 2025, Vol. 61 ›› Issue (9): 231-240.doi: 10.3901/JME.2025.09.231

• 机器人及机构学 • 上一篇

扫码分享

面向机器人操作的柔性线缆视觉分割与抓取位姿生成算法

王松, 蒋潇, 吴丹

清华大学机械工程系北京 100084

收稿日期:2024-05-06 修回日期:2024-09-20 发布日期:2025-06-12
通讯作者: 吴丹,女,1966年出生,博士,教授,博士研究生导师。主要研究方向为机器人加工与操作。E-mail:wud@tsinghua.edu.cn E-mail:wud@tsinghua.edu.cn
作者简介:王松,男,2000年出生,博士研究生。主要研究方向为计算机视觉、机器人加工与操作。E-mail:wangs23@mails.tsinghua.edu.cn;蒋潇,男,2000年出生,博士研究生。主要研究方向为机器人轨迹规划、机器人加工与操作。E-mail:jiangx22@mails.tsinghua.edu.cn
基金资助:
国家自然科学基金资助项目(52375019)。

Visual Segmentation and Grasping Pose Generation Algorithms for Robotic Manipulation of Flexible Cables

WANG Song, JIANG Xiao, WU Dan

Department of Mechanical Engineering, Tsinghua University, Beijing 100084

Received:2024-05-06 Revised:2024-09-20 Published:2025-06-12

摘要/Abstract

摘要： 柔性线缆的高维度以及线缆之间复杂的拓扑关系使得机器人自动操作柔性线缆具有相当的难度。以带端子柔性线缆的自动化插接任务为背景，聚焦机器人操作时线缆拓扑关系和端子空间位姿的视觉感知问题，针对已有研究存在线缆间拓扑关系解耦时前端语义分割网络能力不足、残缺点云端子空间抓取鲁棒性差的问题，提出了基于区域生长的语义分割后处理算法，利用神经网络输出掩膜作为先验信息，以超像素块作为生长单位，选择颜色、方向和形状为编码特征进行区域生长，实现了高效分割补全；提出了局部特征驱动的抓取位姿生成算法，结合插接需求设计夹爪并提取鲁棒特征，将问题简化为复杂点云成分下的平面拟合问题，通过随机抽样一致算法剔除局外点，利用主成分分析法提取主方向并生成抓取位姿。设计语义分割和空间端子抓取算法评价实验，结果表明：相比已有算法，所提算法分割效果好、抓取成功率高。

关键词: 柔性线缆, 语义分割, 位姿生成, 鲁棒抓取

Abstract: Manipulation of flexible cables is a significant challenging task for robots due to the infinite dimensionality of the state space and confusing topological relations of the cables. Based on the industrial requirement of insertion automation of cables with terminals, most studies are focused on visual perception of cable topology and terminal spatial poses estimation during robotic manipulation. However, previous work shows limited capability of front-end semantic segmentation networks for decoupling cable topology and unsatisfied robustness of grasping incomplete point cloud terminals. To address these problems, a semantic segmentation post-processing algorithm, based on region growing, is proposed. The key insight is that the neural network output masks are utilized as prior information, with super-pixel blocks serving as growth units and color, orientation, and shape selected as coding features for region growing, leading to efficient segmentation completion. Moreover, a local feature-driven grasping pose generation algorithm is proposed. Considering a type of insertion terminals, the grippers are specially designed and the robust grasping features are thus extracted. This facilitates problem simplifying to plane fitting amidst complex point cloud components. In this solution, the random sample consensus algorithm is employed to eliminate the outliers, and the principal component analysis method extracts the main direction for grasping pose generation. Finally, specially developed experiments are conducted to evaluate both the semantic segmentation and spatial terminal grasping algorithms. The results exhibit superior performance over existing methods in segmentation effectiveness and grasping success rate.

Key words: flexible cable, semantic segmentation, pose generation, robust grasping

中图分类号:

TP242

王松, 蒋潇, 吴丹. 面向机器人操作的柔性线缆视觉分割与抓取位姿生成算法[J]. 机械工程学报, 2025, 61(9): 231-240.

WANG Song, JIANG Xiao, WU Dan. Visual Segmentation and Grasping Pose Generation Algorithms for Robotic Manipulation of Flexible Cables[J]. Journal of Mechanical Engineering, 2025, 61(9): 231-240.

参考文献

[1] 黄俊锋. 用于工业机器人自动插线系统的视觉识别与定位关键技术研究[D]. 广州:华南理工大学,2020. HUANG Junfeng. Research on key technologies of visual recognition and positioning for industrial robot automatic wiring system[D]. Guangzhou:South China University of Technology,2020.
[2] 吕乃静,刘检华. 柔性线缆的机器人自动敷设关键技术与发展趋势[J]. 机械工程学报,2022,58(17):75-95. LÜ Naijing,LIU Jianhua. Key technologies and development trend of robot automatic laying of flexible cables[J]. Journal of Mechanical Engineering,2022,58(17):75-95.
[3] HE Kaiming,GKIOXARI G,DOLLÁR P,et al. Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision,Hawaii,USA. Piscataway:IEEE,2017:2961-2969
[4] KIRILLOV A,WU Yuxin,HE Kaiming,et al. Pointrend:Image segmentation as rendering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,USA. Piscataway:IEEE,2020:9799-9808.
[5] KIRILLOV A,MINTUN E,RAVI N,et al. Segment anything[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,Paris,France. Piscataway:IEEE,2023:4015-4026.
[6] CAPORALI A,ZANELLA R,De GREOGRIO D,et al. Ariadne+:deep learning-based augmented framework for the instance segmentation of wires[J]. IEEE Transactions on Industrial Informatics,2022,18(12):8607-8617.
[7] CAPORALI A,GALASSI K,ZANELLA R,et al. FASTDLO:fast deformable linear objects instance segmentation[J]. IEEE Robotics and Automation Letters,2022,7(4):9075-9082.
[8] CAPORALI A,GALASSI K,ŽAGAR B L,et al. RT-DLO:real-time deformable linear objects instance segmentation[J]. IEEE Transactions on Industrial Informatics,2023,19(11),11333-11342.
[9] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Boston,USA. Piscataway:IEEE,2015:3431-3440.
[10] CHEN L C,ZHU Yukun,PAPANDREOU G,et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European conference on computer vision (ECCV),Munich,Germany. Berlin:Springer,2018:801-818.
[11] SUN Zhaole,ZHOU Hang,LI Nanbo,et al. A robust deformable linear object perception pipeline in 3D:from segmentation to reconstruction[J]. IEEE Robotics and Automation Letters,2023,9(1):843-850.
[12] DE GREGORIO D,PALLI G,Di STEFANO L. Let's take a walk on superpixels graphs:Deformable linear objects segmentation and model estimation[C]//Asian Conference on Computer Vision. Perth,Australia. Berlin:Springer,2018:662-677.
[13] JIN Peng,LIU Shaoli,LIU Jianhua,et al. Weakly-supervised single-view dense 3D point cloud reconstruction via differentiable renderer[J]. Chinese Journal of Mechanical Engineering,2021,34:1-11.
[14] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60:91-110.
[15] BAY H,TUYTELAARS T,Van GOOL L. Surf:speeded up robust features[C]//Computer Vision-ECCV 2006:9th European Conference on Computer Vision,Graz,Austria. Berlin:Springer,2006:404-417.
[16] RUBLEE E,RABAUD V,KONOLIGE K,et al. ORB:An efficient alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision,Barcelona,Spain. Piscataway:IEEE,2011:2564-2571.
[17] BESL P J,MCKAY N D. Method for registration of 3-D shapes[C]//Sensor fusion IV:Control Paradigms and Data Structures,Boston,USA. Washington:Spie,1992,1611:586-606.
[18] 张瑾贤,吴晓峰,叶才铭,等. 一种基于点云场景分割与改进配准算法的物体位姿估计方法[J]. 机械工程学报,2023,59(22):176-185. ZHANG Jinxian,WU Xiaofeng,YE Caiming,et al. Object pose estimation method based on point cloud scene segmentation and improved registration[J]. Journal of Mechanical Engineering,2023,59(22):176-185.
[19] WEN B,TREMBLAY J,BLUKIS V,et al. Bundlesdf:Neural 6-dof tracking and 3d reconstruction of unknown objects[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver Canada. Piscataway:IEEE,2023:606-617.
[20] FANG H S,WANG C,FANG H,et al. Anygrasp:robust and efficient grasp perception in spatial and temporal domains[J]. IEEE Transactions on Robotics,2023.
[21] NIU Boshen,WANG Chenxi,LIU Changliu. Tolerance-guided policy learning for adaptable and transferrable delicate industrial insertion[C]//Conference on Robot Learning,London,UK. New York:PMLR,2021:2030-2039.
[22] ZHANG Di,AN Qichao,FENG Xiaoxue,et al. Unstructured road extraction in UAV images based on lightweight model[J]. Chinese Journal of Mechanical Engineering,2024,37(1):45.
[23] ACHANTA R,SHAJI A,SMITH K,et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[24] BERTAIMIO M,BERTOZZI A L,SAPIRO G. Navier-stokes,fluid dynamics,and image and video inpainting[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,Kauai,USA. Piscataway:IEEE,2001,1:I-I.
[25] FISCHLER M A,BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[26] MACKIEWICZ A,RATAJCZAK W. Principal components analysis (PCA)[J]. Computers & Geosciences,1993,19(3):303-342.
[27] REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster r-cnn:towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems,2015,28:1.
[28] ROBBINS H,MONRO S. A stochastic approximation method[J]. The Annals of Mathematical Statistics,1951(1):400-407.
[29] YU Mingrui,LÜ Kangchen,ZHONG Hanzhong,et al. Global model learning for large deformation control of elastic deformable linear objects:an efficient and adaptive approach[J]. IEEE Transactions on Robotics,2022,39(1):417-436.
[30] RUSU R B,BLODOW N,BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]//2009 IEEE International Conference on Robotics and Automation,Kobe,Japan. Piscataway:IEEE,2009:3212-3217.

面向机器人操作的柔性线缆视觉分割与抓取位姿生成算法

Visual Segmentation and Grasping Pose Generation Algorithms for Robotic Manipulation of Flexible Cables

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

本文评价

[1]	杨显通, 郑玲, 金彦林, 张涵柯, 曾迪, 冀杰. 基于车载视觉和动力学模型的路面附着系数融合估计方法[J]. 机械工程学报, 2024, 60(24): 254-264.
[2]	彭湃, 耿可可, 王子威, 柳智超, 殷国栋. 智能汽车环境感知方法综述[J]. 机械工程学报, 2023, 59(20): 281-303.
[3]	吕乃静, 刘检华. 柔性线缆的机器人自动敷设关键技术与发展趋势[J]. 机械工程学报, 2022, 58(17): 75-95.
[4]	林添良, 姚瑜, 许文杰, 付胜杰, 任好玲, 陈其怀. 基于环境识别的电动工程机械无人驾驶行走方法[J]. 机械工程学报, 2021, 57(10): 42-49.
[5]	游斌弟, 郑天骄, 陈军, 杨斌久, 赵阳. 拖拽下大变形柔性线缆力学特性分析与测试[J]. 机械工程学报, 2015, 51(23): 36-45.