Visual Segmentation and Grasping Pose Generation Algorithms for Robotic Manipulation of Flexible Cables

doi:10.3901/JME.2025.09.231

Abstract

Abstract: Manipulation of flexible cables is a significant challenging task for robots due to the infinite dimensionality of the state space and confusing topological relations of the cables. Based on the industrial requirement of insertion automation of cables with terminals, most studies are focused on visual perception of cable topology and terminal spatial poses estimation during robotic manipulation. However, previous work shows limited capability of front-end semantic segmentation networks for decoupling cable topology and unsatisfied robustness of grasping incomplete point cloud terminals. To address these problems, a semantic segmentation post-processing algorithm, based on region growing, is proposed. The key insight is that the neural network output masks are utilized as prior information, with super-pixel blocks serving as growth units and color, orientation, and shape selected as coding features for region growing, leading to efficient segmentation completion. Moreover, a local feature-driven grasping pose generation algorithm is proposed. Considering a type of insertion terminals, the grippers are specially designed and the robust grasping features are thus extracted. This facilitates problem simplifying to plane fitting amidst complex point cloud components. In this solution, the random sample consensus algorithm is employed to eliminate the outliers, and the principal component analysis method extracts the main direction for grasping pose generation. Finally, specially developed experiments are conducted to evaluate both the semantic segmentation and spatial terminal grasping algorithms. The results exhibit superior performance over existing methods in segmentation effectiveness and grasping success rate.

Key words: flexible cable, semantic segmentation, pose generation, robust grasping

CLC Number:

TP242

WANG Song, JIANG Xiao, WU Dan. Visual Segmentation and Grasping Pose Generation Algorithms for Robotic Manipulation of Flexible Cables[J]. Journal of Mechanical Engineering, 2025, 61(9): 231-240.

References

[1] 黄俊锋. 用于工业机器人自动插线系统的视觉识别与定位关键技术研究[D]. 广州:华南理工大学,2020. HUANG Junfeng. Research on key technologies of visual recognition and positioning for industrial robot automatic wiring system[D]. Guangzhou:South China University of Technology,2020.
[2] 吕乃静,刘检华. 柔性线缆的机器人自动敷设关键技术与发展趋势[J]. 机械工程学报,2022,58(17):75-95. LÜ Naijing,LIU Jianhua. Key technologies and development trend of robot automatic laying of flexible cables[J]. Journal of Mechanical Engineering,2022,58(17):75-95.
[3] HE Kaiming,GKIOXARI G,DOLLÁR P,et al. Mask r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision,Hawaii,USA. Piscataway:IEEE,2017:2961-2969
[4] KIRILLOV A,WU Yuxin,HE Kaiming,et al. Pointrend:Image segmentation as rendering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,USA. Piscataway:IEEE,2020:9799-9808.
[5] KIRILLOV A,MINTUN E,RAVI N,et al. Segment anything[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,Paris,France. Piscataway:IEEE,2023:4015-4026.
[6] CAPORALI A,ZANELLA R,De GREOGRIO D,et al. Ariadne+:deep learning-based augmented framework for the instance segmentation of wires[J]. IEEE Transactions on Industrial Informatics,2022,18(12):8607-8617.
[7] CAPORALI A,GALASSI K,ZANELLA R,et al. FASTDLO:fast deformable linear objects instance segmentation[J]. IEEE Robotics and Automation Letters,2022,7(4):9075-9082.
[8] CAPORALI A,GALASSI K,ŽAGAR B L,et al. RT-DLO:real-time deformable linear objects instance segmentation[J]. IEEE Transactions on Industrial Informatics,2023,19(11),11333-11342.
[9] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Boston,USA. Piscataway:IEEE,2015:3431-3440.
[10] CHEN L C,ZHU Yukun,PAPANDREOU G,et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European conference on computer vision (ECCV),Munich,Germany. Berlin:Springer,2018:801-818.
[11] SUN Zhaole,ZHOU Hang,LI Nanbo,et al. A robust deformable linear object perception pipeline in 3D:from segmentation to reconstruction[J]. IEEE Robotics and Automation Letters,2023,9(1):843-850.
[12] DE GREGORIO D,PALLI G,Di STEFANO L. Let's take a walk on superpixels graphs:Deformable linear objects segmentation and model estimation[C]//Asian Conference on Computer Vision. Perth,Australia. Berlin:Springer,2018:662-677.
[13] JIN Peng,LIU Shaoli,LIU Jianhua,et al. Weakly-supervised single-view dense 3D point cloud reconstruction via differentiable renderer[J]. Chinese Journal of Mechanical Engineering,2021,34:1-11.
[14] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60:91-110.
[15] BAY H,TUYTELAARS T,Van GOOL L. Surf:speeded up robust features[C]//Computer Vision-ECCV 2006:9th European Conference on Computer Vision,Graz,Austria. Berlin:Springer,2006:404-417.
[16] RUBLEE E,RABAUD V,KONOLIGE K,et al. ORB:An efficient alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision,Barcelona,Spain. Piscataway:IEEE,2011:2564-2571.
[17] BESL P J,MCKAY N D. Method for registration of 3-D shapes[C]//Sensor fusion IV:Control Paradigms and Data Structures,Boston,USA. Washington:Spie,1992,1611:586-606.
[18] 张瑾贤,吴晓峰,叶才铭,等. 一种基于点云场景分割与改进配准算法的物体位姿估计方法[J]. 机械工程学报,2023,59(22):176-185. ZHANG Jinxian,WU Xiaofeng,YE Caiming,et al. Object pose estimation method based on point cloud scene segmentation and improved registration[J]. Journal of Mechanical Engineering,2023,59(22):176-185.
[19] WEN B,TREMBLAY J,BLUKIS V,et al. Bundlesdf:Neural 6-dof tracking and 3d reconstruction of unknown objects[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver Canada. Piscataway:IEEE,2023:606-617.
[20] FANG H S,WANG C,FANG H,et al. Anygrasp:robust and efficient grasp perception in spatial and temporal domains[J]. IEEE Transactions on Robotics,2023.
[21] NIU Boshen,WANG Chenxi,LIU Changliu. Tolerance-guided policy learning for adaptable and transferrable delicate industrial insertion[C]//Conference on Robot Learning,London,UK. New York:PMLR,2021:2030-2039.
[22] ZHANG Di,AN Qichao,FENG Xiaoxue,et al. Unstructured road extraction in UAV images based on lightweight model[J]. Chinese Journal of Mechanical Engineering,2024,37(1):45.
[23] ACHANTA R,SHAJI A,SMITH K,et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[24] BERTAIMIO M,BERTOZZI A L,SAPIRO G. Navier-stokes,fluid dynamics,and image and video inpainting[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,Kauai,USA. Piscataway:IEEE,2001,1:I-I.
[25] FISCHLER M A,BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[26] MACKIEWICZ A,RATAJCZAK W. Principal components analysis (PCA)[J]. Computers & Geosciences,1993,19(3):303-342.
[27] REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster r-cnn:towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems,2015,28:1.
[28] ROBBINS H,MONRO S. A stochastic approximation method[J]. The Annals of Mathematical Statistics,1951(1):400-407.
[29] YU Mingrui,LÜ Kangchen,ZHONG Hanzhong,et al. Global model learning for large deformation control of elastic deformable linear objects:an efficient and adaptive approach[J]. IEEE Transactions on Robotics,2022,39(1):417-436.
[30] RUSU R B,BLODOW N,BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]//2009 IEEE International Conference on Robotics and Automation,Kobe,Japan. Piscataway:IEEE,2009:3212-3217.