• CN: 11-2187/TH
  • ISSN: 0577-6686

Journal of Mechanical Engineering ›› 2026, Vol. 62 ›› Issue (18): 215-225.doi: 10.3901/JME.260740

Previous Articles    

Multi-modal Fusion Shield Segment Recognition and Positioning Method Based on Visual Large Model

XIAO Yanqiu1,2, WEN Qing1,2, WANG Pengpeng1,2,3, CUI Guangzhen1,2, JIA Lianhui4, CHEN Zhijun1,2, JIAO Jianqiang1,2, YAO Lei1,2, CHEN Huizi1,2   

  1. 1. Henan Excellent Engineer College, Zhengzhou University of Light Industry, Zhengzhou 450002;
    2. Henan Collaborative Innovation Center of Intelligent Tunnel Boring Machine, Zhengzhou University of light Industry, Zhengzhou 450002;
    3. State Key Laboratory of Geomechanics and Geotechnical Engineering Safety, Instituteof Rock and Soil Mechanics, Chinese Academy of Sciences., Wuhan 430064;
    4. Electrical and Intelligent Technology Research Institute, China Railway Engineering Equipment Group Co. Ltd, Zhengzhou 450047
  • Received:2025-12-23 Revised:2026-06-18 Published:2026-11-20

Abstract: In the automatic assembly of shield tunnel segments, it is difficult to accurately measure the grasping pose of segments due to the complex environment and weak target characteristics. To solve this problem, a multi-modal fusion shield segment spatial pose calculation method based on visual large model is proposed. Firstly, the YOLO-SAM model based on edge guidance and dynamic weight scheduling is constructed to detect and segment the segment under complex background, and then it is mapped to the three-dimensional point cloud. Through the cross-modal feature extraction method of point cloud registration fusion of principal component analysis (PCA) and iterative closest point (ICP), the high-precision extraction of geometric features of segment bolts is realized. Finally, combined with the multi-feature fusion space vector constraint relationship between the segment bolt and the positioning hole, the spatial pose of the segment is solved, so as to solve the pose calculation problem caused by the rotational symmetry of a single bolt. The algorithm proposed in this paper is applied to the full-scale 6-meter segment assembly experimental platform. The verification results show that the intersection over union (IoU) of the proposed segmentation method is 96.20%, the measurement position error is less than or equal to 2 mm, and the attitude error is less than or equal to 2 °. The measurement accuracy can meet the engineering accuracy requirements of shield segment assembly, and a complete technical framework for subsequent automatic segment assembly recognition is built.

Key words: shield tunnel, YOLO-SAM model, multimodal fusion, pose measurement

CLC Number: